GitLab Duo Self-Hosted with Bedrock or Azure

How GitLab Duo Self-Hosted on AWS Bedrock or Azure OpenAI works

Teams that want GitLab Duo features without buying and operating their own GPU servers can use a cloud model platform as the place where models run, but only if code and prompts stay within boundaries their cloud and data policies already approve. In this setup, the self-hosted AI gateway sits between your self-managed GitLab instance and the model platform. GitLab sends Duo requests to the gateway, and the gateway forwards them to the model endpoint you configure. The gateway is the component you deploy, place on your network and align with your GitLab version, so it is where most of the data path decisions are made.

Before any design is fixed, confirm three things against GitLab's current documentation and support matrix: which connection path is supported for AWS Bedrock and for Azure OpenAI, which models are offered on each platform for Duo, and which feature ratings apply to those models. Support for a given model family and rating can change between releases, and a feature that is rated Full for one model may be rated Partial or Limited for another. Record the GitLab version, gateway version and model identifiers you tested, because the gateway and GitLab instance must stay version aligned during upgrades.

Whether this option is acceptable is a decision for your organization, not for DeployOpen or the cloud vendor. It depends on your cloud account structure, the regions and data residency terms your contracts require, how your provider's data handling terms apply to inference requests, and what your auditors expect to see in network diagrams and logs. DeployOpen implements and documents the gateway, network paths and access controls, and prepares evidence for your compliance reviews, but it does not issue audit reports or certifications. Those come from independent auditors and certification bodies, so bring them into the design before build starts.

What you need before connecting a cloud model platform

Connecting GitLab Duo Self-Hosted to a cloud model platform touches four areas at once: the GitLab release, the Duo entitlement, the cloud account and the network path. Work through the cards in parallel with GitLab and your cloud provider, because a mismatch in any one area blocks the others.

GitLab and gateway version alignment

Confirm that your self-managed GitLab instance and the self-hosted AI gateway are on versions GitLab documents as compatible with each other and with Duo Self-Hosted. Mismatched releases are a common cause of failed model calls that look like network faults. Ask GitLab which gateway release matches your instance before you choose a target, and plan the upgrade path so both move together. Keep the GitLab upgrade stops in view when you do.

Duo subscription and add-on

Confirm which entitlement your deployment needs before you provision anything in the cloud. An online license and an offline license lead to different add-ons and billing, and the choice decides whether the gateway can reach external services at all. Ask your GitLab account team to confirm the seat type, the add-on name and how usage is metered against a cloud model platform, since those terms affect the cost model you present internally.

Cloud account and model access

Confirm that your AWS account or Azure subscription can use the model service you intend to call, and that access to the specific model is approved in the target region. Model access is often granted per model rather than per account, and a quota or approval request can take longer than the build itself. Ask GitLab which model families are supported on the platform you choose, and check the support rating for each feature before you commit.

Private network path to the endpoint

Plan how the self-hosted gateway reaches the model endpoint over private connectivity rather than the public internet. On AWS this usually means a VPC endpoint for Bedrock, and on Azure a private endpoint for Azure OpenAI, with routing and security groups or firewall rules that allow only the gateway's subnet. Confirm with your cloud provider which private connectivity options are available in your region and whether your egress controls permit them.

Certificate and DNS planning

Decide the internal hostnames for the GitLab instance and the AI gateway, and confirm which certificate authority issues them. The gateway and GitLab must trust each other's certificates, and private endpoint names must resolve to private addresses from the gateway subnet, not public ones. Ask your network team to confirm split-horizon DNS and certificate rotation ownership before the gateway is configured, because an expired or untrusted certificate fails silently in many workflows.

Credentials and ownership

Decide who owns the cloud identity the gateway uses to call the model platform, and how that credential is stored and rotated. Prefer a role or managed identity scoped to model invocation over long-lived access keys, and confirm with your security team where the secret is kept. Name an owner for each area, including GitLab administration, cloud access and network changes, so that a model version change or access revocation has a clear path to approval.

Where cloud model platforms differ from on-premises GPUs for data residency

The core difference is where prompts and model outputs are processed. With on-premises GPUs, inference runs inside your own facilities, so prompt data never leaves infrastructure you control. With AWS Bedrock or Azure OpenAI, the model runs in a cloud provider's regions and processes your prompts under that provider's terms, not yours. Both options can be valid, but they answer the residency question differently, and the decision has to be made before the AI gateway is pointed at a cloud endpoint.

Start with the region. Confirm which region or regions will host the model you select, and whether your organization can restrict processing to a specific geography. Then check what happens to inputs and outputs. Ask the provider whether prompts or completions are retained, for how long, whether they are logged for abuse monitoring, and whether they are ever used to train or improve models. Map the answers to the data classes you plan to send through Duo, such as source code, merge request content and issue text, since those classes often contain regulated or confidential material.

Next, review the contractual and data processing terms with your legal and privacy teams. Identify the data processing agreement, sub-processor list, breach notification commitments and audit rights that apply to the cloud service you would use, and confirm whether they match the obligations your regulator or internal policy imposes on third-party processors. Some organizations accept a private cloud processor for code assistance but not for regulated records, so consider whether the scope can be limited by project or group. If offline or air-gapped operation is a requirement, this option is ruled out, because it depends on a cloud endpoint. In that case, follow the fully offline Duo topic instead.

Configuring the gateway to call your cloud model endpoint

This sequence connects a self-managed GitLab instance to a cloud-hosted model through a self-hosted AI gateway, with private connectivity and credentials handled inside your boundary. Each step closes with a check you can run, so a failure is traced to the layer where it occurs.

Confirm the model and endpoint type
The platform owner checks GitLab's current self-hosted model support matrix for the model family and the Bedrock or Azure OpenAI endpoint type you intend to use, then records the rating (Full, Partial or Limited) for each Duo feature you plan to enable. Ask GitLab or your account contact to confirm that the chosen model is supported with your GitLab and gateway versions, since support can differ by release. Done when a dated decision record lists the model, endpoint type, region constraints and feature ratings, and the security and compliance owners have signed it.
Create model access in the chosen region
In the cloud account that will own the model, enable access to the chosen model in one region, or create the Azure OpenAI resource and deployment, following the provider's current steps. Keep the account, subscription and resource group inside the same boundary as your other regulated workloads, and confirm the region is one your data residency rules allow. Create a dedicated identity for the gateway with only the permissions needed to invoke that model. Check: a test prompt sent from outside GitLab, using the provider's console or CLI, returns a completion, and the request appears in the provider's audit log.
Set up private connectivity and DNS
The network team places the gateway and the model endpoint on a path that stays off the public internet, typically with a VPC interface endpoint for Bedrock or a private endpoint for Azure OpenAI, and creates an internal DNS record that resolves the endpoint name to its private address. Confirm with your provider that the endpoint type supports private connectivity in your chosen region. Restrict public network access on the model resource where the provider allows it. Check: from the gateway host, the endpoint name resolves to a private address, and a connection test from a host outside the approved subnet fails.
Deploy the gateway at the matching version
The platform team installs the self-hosted AI gateway on a host or cluster inside the same network boundary, using the version that matches your GitLab instance. Confirm the alignment requirement in GitLab's gateway guidance before installing. Record the version in the change ticket and repeat this check at every GitLab upgrade. Check: the gateway reports the expected version, and its health endpoint returns a healthy status when called from the GitLab instance host, which shows the two components can reach each other before any model call is made.
Store provider credentials in your secrets manager
Where the platform supports it, grant the gateway an IAM role or managed identity so that no static key exists. Where a key or secret is required, store it in your organization's secrets manager, such as AWS Secrets Manager, Azure Key Vault or OpenBao, and have the gateway read it at start-up from an environment variable or file that the secrets manager populates. Name the rotation owner in the runbook. Check: the credential is absent from gateway configuration files, shell history and logs, and the gateway starts successfully after the secret is loaded.
Point the gateway at the model endpoint
Set the gateway's model and endpoint settings to the private endpoint name, region and deployment or model identifier from the earlier steps, using the configuration keys documented for your gateway version. Keep these settings in version control with the rest of the deployment, and limit who can change them. Restart or reload the gateway as its documentation requires. Check: the gateway log shows a successful upstream request to the cloud endpoint, and a direct call to the gateway API from the GitLab instance host returns a completion for the chosen model.
Enable Duo features and run test requests
In the Duo self-hosted settings of the GitLab Admin Area (confirm the menu path for your version), connect the instance to the gateway, assign the model to each feature you enabled, and confirm the licensing prerequisites for your deployment are in place. Then run one test request per enabled feature from a non-production user and compare each result with the rating recorded in the first step. Check: each feature returns output, the gateway log records each request against the expected model, and any feature rated Partial or Limited is documented for its users.

Mapping models, endpoints, identities and data between the systems

This table maps each element of a Duo Self-Hosted deployment on Bedrock or Azure OpenAI to its counterpart on each platform, and shows what to confirm for each one. Read it as a checklist for design reviews, and confirm every mapping against the feature support matrix, gateway documentation and platform documentation for your GitLab release before you commit to a design.

Element	AWS Bedrock	Azure OpenAI	What to confirm
Duo feature	Duo feature routed to a model hosted in Bedrock	Duo feature routed to a model deployed in an Azure OpenAI resource	Which Duo features the chosen model supports on this platform and release
Model deployment or endpoint	Model invoked through the Bedrock runtime API in one AWS account and region	Named model deployment inside one Azure OpenAI resource	Which model families are available and supported on each platform
Gateway configuration key	Provider settings in the self-hosted AI gateway that point to the Bedrock endpoint	Provider settings that point to the Azure OpenAI resource endpoint and deployment name	Exact key names, format and version-specific behavior in the gateway documentation
Credential or managed identity	IAM role or access credentials the gateway uses to call Bedrock	Microsoft Entra ID managed identity or API key for the resource	Which authentication methods the gateway accepts and where secrets are stored
Cloud region	Region where the model is offered and where the request is processed	Region of the Azure OpenAI resource and its deployment	Model availability per region and whether any cross-region routing occurs
Data that leaves the instance	Prompts, code context and responses sent to the Bedrock endpoint	Prompts, code context and responses sent to the Azure OpenAI endpoint	Full list of payloads per enabled feature, plus logging, retention and abuse monitoring settings

Choosing between AWS Bedrock and Azure OpenAI

If your organization already runs its workloads and identity in AWS, Bedrock keeps the model platform inside the account structure, network design and audit tooling your teams already operate. The main work is confirming that the account, VPC and endpoint design meet your private connectivity rules, and that the people who own the account accept the model traffic path.

Account ownership: Confirm which team owns the AWS account that hosts the models, who approves quota changes and who can grant model access in the console.
Network controls: Check whether your existing VPC endpoint, egress and firewall patterns can reach the Bedrock endpoints from the subnets where the AI gateway runs.
Identity and access: Map how the gateway authenticates to Bedrock, such as IAM roles, and whether your existing federation and least-privilege policies cover it.
Logging and keys: Decide whether invocation logs go to your current log archive and whether encryption uses your existing KMS keys.
Model availability: Verify in the AWS console and documentation which models are offered in your chosen region and whether they match the Duo support matrix.

Before committing, confirm the exact model list, regional availability and quota for your account with AWS and with GitLab's current support matrix.

Securing credentials, network paths and prompt data

A cloud-hosted model platform adds identities, keys, network routes and logs that sit outside GitLab. Each one needs an owner and a documented setting before the gateway goes live. The cards below cover the decisions that most often leave gaps between the GitLab side and the cloud side.

Least-privilege gateway cloud identity

The AI gateway should call only the model operations it needs in your AWS Bedrock account or Azure OpenAI resource. Scope the identity to the specific model deployments or foundation models approved for Duo, and deny administrative actions such as creating new endpoints or changing logging settings. Confirm with the provider's current documentation which permissions each Duo feature requires, because the list changes between releases.

Keys and managed identities

Prefer a managed identity or IAM role over a static access key when the gateway's platform supports it. If a provider key must be stored, keep it in a secrets manager or the gateway's secret mechanism, not in gitlab.rb or plain configuration files where it can leak through backups and support bundles. Ask who can read the secret, how it is rotated, and what happens to the gateway if rotation fails.

Outbound paths limited to the model endpoint

Allow the gateway to reach only the regional model endpoint it uses, plus the GitLab instance it serves. Block general internet egress from the gateway host and from the GitLab nodes that do not need it. Test this with a deliberate blocked request, and record the allowed destinations in your change management system so reviewers can see the boundary.

Who can enable features and models

Decide which GitLab administrators can turn Duo features on, change the model selection and enable beta or experimental features for groups or projects. Restrict these changes to a small role with logged approvals, and review them after each GitLab upgrade, since new features may appear with default settings that expand what data leaves the instance.

Gateway and cloud-side logs

Turn on logging in both places: the gateway's request logs in your infrastructure and the provider's activity and model invocation logs in the cloud account. Decide where each log is stored, who can read it, and how long it is kept, and align retention with your own policies. Prompt content may appear in some logs, so treat those stores as sensitive data.

Gradual rollout of hardening settings

GitLab's hardening guidance is a work in progress and recommends rolling settings out gradually. Apply changes in a staging instance first, record the before and after state, and widen the rollout only after Duo features have been tested. Keep in mind that these settings reduce risk but do not make a deployment compliant on their own, because auditors assess them independently against your chosen framework.

Why the gateway cannot reach the cloud model and how to fix it

This table lists the failures teams most often hit when the self-hosted AI gateway calls AWS Bedrock or Azure OpenAI, with the likely cause and a verification step for each fix. Work through the rows in order, since version and network problems hide the symptoms of the rows below them. Confirm each fix with the provider and GitLab documentation for your exact release before changing production settings.

Symptom	Likely cause	Fix and verification step
Gateway logs show requests failing after an upgrade of GitLab	Gateway and GitLab instance run different versions	Align the AI gateway version with the GitLab release, then compare the gateway version output with the GitLab version in the Admin area before retesting
Model calls return access denied or model not found	Model access was not granted in the chosen AWS region or Azure deployment	In the AWS console, confirm model access for the exact region the gateway calls. In Azure, confirm the model is deployed in the same resource and region, then send a test request from the gateway host
Requests time out or reach a public address	Private DNS zone missing, or a resolver sends the endpoint name to a public IP	Resolve the Bedrock runtime or Azure OpenAI endpoint name from the gateway host with nslookup or dig, and confirm it returns the private endpoint address you configured
Connection refused or no route to host	Egress route or security group blocks the private endpoint port	Run a TCP connection test to port 443 on the private endpoint address from the gateway subnet, then review the route table and security group rules for that subnet
Authentication errors with a previously working setup	Access key or Azure API key expired, rotated or belongs to the wrong account	Check the key age and identity in the secrets store, then call the provider with that credential from the gateway host to confirm it authenticates outside GitLab
Access denied on some features but not others	IAM role or Azure role scoped to one model or action only	Compare the policy attached to the gateway identity with the models Duo uses for each feature, and test each model invocation separately
TLS handshake or certificate verification errors	Private endpoint certificate chain not trusted by the gateway host	Add the internal certificate authority to the trust store the gateway uses, then run an HTTPS request with certificate verification enabled and confirm the chain validates
Requests succeed in testing but Duo returns model errors	Model name in gateway configuration does not match the deployed model identifier or deployment name	Copy the exact model identifier from AWS or the Azure deployment name into the gateway configuration, restart the service, and send one request for each configured model

What buyers ask before connecting Duo to Bedrock or Azure OpenAI

How much effort and cost does connecting Duo to Bedrock or Azure OpenAI involve?

The cost has two parts: a one-time build and recurring operation. One-time work covers design, configuration, network changes and testing, whether your team or a partner does it. Recurring costs include model consumption billed by AWS or Azure, GitLab licensing for the Duo seats or add-ons your deployment needs, staff time to run the AI gateway and watch usage, and any support contract. Consumption is usually the most variable line because it grows with the number of developers using Duo and how heavily they use it. GitLab's pricing page lists Premium at 29 US dollars per user per month billed annually, and the higher tier is quoted individually. Ask any quote to separate one-time work from monthly items and to state the user count and usage model it assumes.

Can our team connect Duo to Bedrock or Azure OpenAI without outside help?

Yes, a capable platform team can do this in-house, and many teams prefer to. The work combines cloud identity and model access, private network routing to the AI gateway, GitLab configuration and ongoing operations. The skills needed are cloud IAM, network routing and DNS, VM or container operations, and experience administering GitLab, including its required upgrade stops. The harder part is not the first connection but keeping the gateway, GitLab and model settings aligned through upgrades, quota changes and model updates. A partner usually helps when the team is stretched, the timeline is short or security review calls for independent testing. DeployOpen plans and implements this work under a fixed scope agreed in writing, so you can compare that quote with your own staffing estimate.

How long does the integration take, and what drives the timeline?

Duration depends mainly on how much of the foundation already exists. The main drivers are whether a GitLab instance is running at a supported version, whether cloud accounts, model access and service quotas are ready, how quickly network and firewall changes are approved, and how many gates security review adds. Model onboarding also takes time, because each model must be confirmed against the Duo features you plan to use using GitLab's support matrix. A small pilot group before wider rollout is a common stage that adds time but reduces risk. Ask for a timeline split into discovery, build, pilot and handover, and identify which milestones depend on client teams. A single end date with no dependencies listed is hard to check and harder to hold anyone to.

What does the work need from our cloud, network and security teams?

The work needs decisions and access from several teams before the first build day:

Cloud account owners who can grant access to AWS Bedrock or Azure OpenAI, including model enablement, service quotas and identity roles for the gateway.
Network teams to allow private routing between the GitLab instance, the AI gateway and the model endpoint, along with DNS and firewall rules.
Security reviewers who sign off on data flows, how model credentials are stored, logging and the change record.

Also name a GitLab administrator and an owner for the cloud subscription. Without named people in these roles, the schedule usually slips at the first approval gate, so confirm them before the kickoff.

Does a Bedrock or Azure OpenAI setup meet our data residency requirement?

It can, but only when the chosen region, the service configuration and your contract terms all match your requirement. Start by confirming which regions offer the model you need, where prompts and outputs are processed and stored, and whether any logging or monitoring keeps data outside that region. These details come from the cloud providers' current documentation and your agreements with them, so verify each one directly. Regulated teams should also confirm how encryption keys, access logs and the cloud account are governed under their own controls. If your rules prohibit any transfer of prompts or code to a third-party cloud service, the cloud model path is not acceptable, and you should evaluate an on-premises GPU or offline option instead.

When would a fully offline deployment be the better path?

An offline deployment is the better path when no prompt, code or model traffic may leave your network. GitLab documents running self-managed instances in offline, air-gapped environments, with separate guidance for running Duo Self-Hosted offline. That path uses self-hosted models and a self-hosted AI gateway on hardware you control, along with offline licensing through the Agent Platform Self-Hosted add-on. The trade-off is that you own GPU capacity, model updates and capacity planning, which raises upfront cost and requires specialized operations skills. A cloud model platform makes more sense when the cloud region satisfies residency and the team wants to avoid running GPUs. Confirm the licensing and feature list with GitLab before choosing between the two.

What should we request in a written quote for this work?

A useful written quote states the scope, the assumptions and the split between one-time and recurring items. Ask for:

The GitLab version, AI gateway version and model families covered, with the Duo features each model is expected to support.
A clear list of what you provide (cloud accounts, model access, quotas, network paths) and what the partner builds.
Effort by phase for discovery, build, security testing, pilot and handover, with each date tied to the client dependencies it needs.
Recurring items, including the model consumption assumptions, GitLab license terms, support and upgrade work.
Acceptance criteria and the process for handling a delayed dependency.

DeployOpen agrees scope, deliverables and timelines in writing before work starts, so the signed quote is the document the project is measured against.

How DeployOpen scopes and delivers Duo Self-Hosted integrations

DeployOpen plans, implements, integrates, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For GitLab Duo Self-Hosted on AWS Bedrock or Azure OpenAI, we start from a written scope that fixes which cloud model platform, network path and GitLab instance are in play before any build begins.

How we scope

Confirm which cloud model platform your data terms and residency rules allow, and document the decision in writing.
Map the GitLab version, AI gateway placement and private network path to each model endpoint in the scope.
Agree the written scope, deliverables and timelines before work starts, with features and models named explicitly.

How we build

Configure the self-hosted AI gateway to reach your Bedrock or Azure OpenAI endpoints using identities scoped to least privilege.
Set up private connectivity, egress controls and logging so model traffic follows the route your security team approved.
Test each Duo feature in scope against the chosen model, recording gaps against the vendor's support matrix for your version.

How we hand over

Deliver architecture notes, configuration references and runbooks for the gateway, identities, network rules and model settings.
Document rollback steps, version alignment rules and the checks to run before each GitLab upgrade stop.
Walk your platform, security and compliance owners through the evidence we produced, so your teams can operate it themselves.

How we support

Offer ongoing support, upgrades and continuous testing as a retainer, scoped separately from the initial build.
Re-test the gateway and model connections after each GitLab, gateway or cloud platform change that affects them.
Ask for a quote based on the written scope, which sets out who handles model quota, cloud spend and vendor escalations.

GitLab Duo Self-Hosted on AWS Bedrock or Azure OpenAI