Choosing a self-hosted model for GitLab Duo

Choosing a self-hosted model for GitLab Duo Code Suggestions, Chat and the Agent Platform

Choose the model family per feature, before you buy GPUs or configure the AI gateway. Code Suggestions, Chat and the Duo Agent Platform do not all work equally well with the same model, so a single model choice for the whole instance is rarely the right plan. Start from GitLab's support matrix for the exact GitLab version you run, because supported model names and their ratings change between releases. Read each rating against the feature it applies to: Full, Partial and Limited describe how well a feature works with a given model, not whether the feature is available at all. Confirm the exact wording and any footnotes for each rating in the matrix itself rather than relying on a summary.

You need this decision when you plan a self-hosted or fully offline Duo rollout, because the model determines the hardware you size, the serving stack you operate and the features your developers can rely on. If you plan to use the Agent Platform, check the agentic rows separately, since a model that rates well for Code Suggestions may not rate well for multi-step agent work. Record the chosen model, its rating for each feature and the GitLab version it was checked against in your design document, so the GPU sizing, gateway configuration and any later upgrade are based on the same evidence.

Before you commit, ask your GitLab contact or check the matrix for three things: whether your target model is listed for each feature you intend to enable, what changes when you move to a newer GitLab version, and whether any feature you need is rated Limited for that model. Also confirm that the model you select can be deployed in your offline environment, because the supported list and the deployment options for air-gapped installations are the decision points that most often change a plan late in the project.

What Full, Partial and Limited ratings mean for each feature

Code Suggestions is usually the first Duo feature developers notice, because it appears while they type in their editor. A Full rating tells you that GitLab rates the feature as fully supported for the model family you are considering, so developers should get the complete set of suggestion behavior the matrix describes. Partial and Limited ratings signal that some behavior is missing or narrower, and the matrix footnotes explain the specifics.

Full: Developers can expect the suggestion behavior the matrix lists for that model family. Confirm which editors and versions the rating covers.
Partial: Some capabilities, editors or languages may be excluded or behave differently. Read the footnote that applies before you promise the feature to a team.
Limited: The supported behavior is narrower. Treat the combination as a pilot candidate before any wider rollout.
Editor coverage: Test suggestions in every editor your developers use, on the languages that dominate your main repositories.
Response time: Measure how long suggestions take on your own GPUs with the number of developers you expect at peak.

Before you accept a Partial or Limited rating, confirm the footnote text in GitLab's support matrix and write the exact gaps into your acceptance record.

Which model family fits which situation

This table compares the options teams weigh when choosing a model for GitLab Duo Self-Hosted. Read each row against the rating GitLab publishes for every feature at your target version. The main takeaway is that a Full rating for every feature in scope is the simplest case, while Partial and Limited ratings should be treated as a scope decision and built into the rollout plan.

Situation	When it fits	Main trade-off	What to confirm
Full for every feature in scope	Code Suggestions, Chat and the Agent Platform all need to run on one model family, so one support boundary covers the rollout.	Running the model usually needs the most GPU capacity, so hardware planning starts with this choice.	The rating for each feature at your exact GitLab and AI gateway version, and whether the rating holds for fully offline installs.
Full for some features only	The features your teams need first are rated Full, and the rest can wait until a later phase.	Users may hit a gap after launch, and running two families for different features adds configuration work.	Which features are rated Full at your version, and the footnotes attached to each rating.
Partial or Limited for the features you need most	Suits a pilot, a small group of users, or a feature that is not central to the business case.	Users may get weaker results or fewer capabilities than with Full-rated features, so expectations need setting before rollout.	What each Partial or Limited footnote means, and whether GitLab has published a plan to raise the rating in a later release.
Model served through a private cloud platform	Fits when your policy allows model traffic to a cloud provider in an approved region and you prefer not to operate GPU servers yourself.	Model traffic leaves your data center even when it stays in your cloud tenancy, and quotas and billing come from the provider.	Approved regions, whether prompts and responses are retained by the provider, and the offline support status of the feature set.
Model served on on-premises GPUs	Fits when prompts, code and model traffic must stay in facilities you control, including air-gapped networks.	Your team owns procurement, driver and patch management, capacity planning and model updates, and these recur for the life of the deployment.	Supported accelerator types and driver versions, and the sizing guidance for your chosen model family and developer count.
Offline or air-gapped requirement	Fits when the instance has no internet connectivity at any point, including licensing checks.	Features that rely on outside services in hybrid mode may be unavailable, which can narrow the choice of model family.	Offline support for each feature in your scope, and the licensing path for an offline deployment.

How to shortlist and validate a model before rollout

This sequence moves from the features and users in scope to a tested model choice recorded in the written scope. Each step produces a record that the next step depends on, so the final decision can be traced back to the GitLab version and the pilot evidence.

List the Duo features and user groups in scope
The platform owner works with product and engineering leads to write down which GitLab Duo features will be enabled (Code Suggestions, Chat, and any Agent Platform use cases) and which teams or user groups will use each one. Mark features that are out of scope for the first rollout so they are not assumed later. Done means one written list where every feature is tied to named user groups and an expected usage pattern, reviewed by both sides.
Record the GitLab version running in production
The platform team confirms the exact GitLab version on the production instance, including any planned upgrade target and the date of that plan. Support matrix ratings are tied to specific versions, so this record decides which matrix you read. Check that the version string matches what the instance reports in its admin area and that the upgrade plan respects the required upgrade stops described in GitLab's upgrade paths documentation.
Pull the support matrix rows for that version
Using the version from the previous step, the team copies the support matrix rows for each in-scope feature, listing model families and their ratings. Read the rating definitions in the matrix itself rather than relying on memory or secondhand summaries. Done means a dated copy of the relevant rows, stored with the version number, that the review group can check against the vendor source.
Shortlist families rated for every in-scope feature
Keep only the model families that carry a rating your scope accepts for every in-scope feature, not just the one you care about most. If a family is strong for Code Suggestions but weak for Chat, it either goes or the scope changes. If no family covers all features, the team reduces the feature list or splits the rollout by feature. Done means a short list where each entry shows its rating per feature.
Run a pilot on each shortlisted model
Engineers run the same set of tasks on each shortlisted model, using actual repositories and developer requests drawn from your own codebase and workflows. Requests reach the models through the self-hosted AI gateway configured for the same GitLab version, so the results reflect your setup and not a demo environment. Done means every candidate has produced outputs for the identical task set, with logs kept for review.
Record quality and latency observations
Reviewers score each output against a written rubric covering correctness, usefulness, failures and response time under the concurrency you expect. Where practical, reviewers do not know which model produced each output. Done means a scored sheet per feature and per candidate, with named reviewers and notes on the failures that would block rollout. Use this sheet, not impressions from a single session, to compare candidates.
Fix the model choice in the written scope
Only after the pilot results are recorded, the team writes the chosen model family, its rating for each in-scope feature, the GitLab version and the known limitations into the written scope. The scope also names the events that require repeating validation, such as a GitLab upgrade or a change to the model. Check that the signed scope can be traced back to the matrix copy and the pilot sheet.

Why the GitLab version and upgrade plan limit your model options

The model you can run for GitLab Duo Self-Hosted depends on the GitLab version you run, so the support matrix is a version-specific document, not a fixed list. Before you pick a model family, record the exact GitLab version your instance runs today and the version you plan to reach within the next upgrade cycle. Then check the support matrix against both. A model that is rated for your current release may not carry the same rating after you move forward, and a choice made against an older matrix can quietly become a support question later.

GitLab's maintenance policy shapes how often you will need to move. Bug fixes are backported only to the current stable release, while security fixes also reach the previous two monthly releases. In practice, a team that falls behind on patching may find that it has to upgrade sooner than it planned in order to keep receiving fixes, and that the matrix it relied on has changed underneath it. Plan for version movement as a recurring operational task, and confirm the current supported release line and its security backport window in the GitLab maintenance policy before you commit to a model.

Required upgrade stops add a second constraint. GitLab requires upgrades to pass through stops such as 18.2, 18.5, 18.8 and 18.11, so a jump from one version to a later one may have to be staged. Treat each planned stop as a checkpoint: confirm which model families the matrix supports at that version, and confirm whether the self-hosted AI gateway and model server need to move with GitLab. Ask your GitLab contact or check the GitLab upgrade paths documentation for the path that applies to your instance, and rerun the matrix check before each upgrade rather than once at deployment.

Security and operating risks in the model decision

The model family you pick decides where prompts, code and model weights travel, who patches what, and how the service behaves under load. These four to six risks connect the support matrix to the controls your security and platform teams own once Duo is running.

Where prompts and code are processed

Every Code Suggestions and Chat request carries source code, file context and sometimes ticket text. Hybrid setups can send parts of that traffic to an external platform, while a fully self-hosted model keeps it inside your network. Before choosing a model, map each feature to the endpoint it calls, confirm with GitLab which requests leave your environment for that feature, and record the answer in your data flow documentation.

Keeping weights and endpoints inside the boundary

Model weights, the inference endpoint and the AI gateway are all components that must sit inside your trust boundary, with the same access control you apply to other production systems. Check who can pull or replace model files, which hosts can reach the inference API, and whether the gateway accepts requests only from your GitLab instance. Treat a model file as software you must version, verify and restrict.

Patching the serving stack with GitLab

The model server, the AI gateway and the GitLab instance must stay in version alignment, so an upgrade to one can break or silently change behavior in another. Plan a combined patch cycle that follows GitLab's release cadence and upgrade stops. Before each change, confirm the supported versions of every component in the compatibility guidance for your release and test the full Duo path in staging first.

Capacity for concurrent developers

Inference capacity is shared, so a model that answers well for a pilot group can slow to a crawl when a full engineering team starts using Chat and Code Suggestions at the same hour. Track request queue depth, time to first token and GPU memory during peak use, not only average latency. Ask your vendor or GitLab to state the concurrency each configuration was validated for, and load test your own peak before rollout.

Access control on agent actions

The Agent Platform can read repositories, propose changes and run tools on behalf of a user, so a model that handles agentic tasks needs the same authorization limits as the user it acts for. Review which projects, groups and tools an agent can reach, restrict agent actions to the permissions of the initiating user, and log every action. Confirm with GitLab which agentic features your chosen model family supports before enabling them for any group.

Monitoring and audit evidence

Security teams need to show who used the assistant, what it accessed and how the service was configured on a given date. Without logs from the model server and gateway alongside GitLab audit events, incident investigation and change records have gaps. Decide which request metadata you retain, where it is stored, who can read it, and how long it is kept, and confirm that these settings meet your own policy before go-live.

What to confirm in the support matrix and your environment

Before you commit to a model family, confirm the exact support matrix row for your GitLab version and deployment type. Ask GitLab which Duo features are rated Full, Partial or Limited for each candidate model, and whether that rating applies to Code Suggestions, Chat and the Agent Platform separately. Ask whether the rating changes when you move between minor releases, because a model that is Full on one release can be rated differently on another. Record the answers against the version you plan to run at go-live, not the version you run today. If your GitLab instance will sit on a required upgrade stop, check the matrix for the target version as well, since a model choice made for the current version may need revisiting before the upgrade.

Next, confirm whether the Agent Platform is available for your deployment type and whether your models support its agentic features. Ask GitLab directly whether the Agent Platform is offered on self-hosted deployments at your target version, and which model families it supports there. Then confirm the licensing path for self-hosted features. The choice between Duo Enterprise seats and the Agent Platform Self-Hosted add-on is a separate decision covered in its own topic, so settle it before you size hardware. Ask your account contact which license type applies to an online or offline instance, because it changes which features reach your models at all.

Finally, confirm that your environment can run the chosen model and that offline installations follow their own steps. Ask your infrastructure team whether your GPU servers, storage and network paths to the self-hosted AI gateway meet the vendor's stated minimums for the model you select, and whether the GPU sizing topic applies to your user count. For air-gapped networks, check GitLab's offline environments guidance for the separate steps needed to run GitLab Duo Self-Hosted without internet access, including how models and gateway images reach the network. Treat any gap between the matrix and your hardware as a blocker to resolve in writing with GitLab before procurement.

Questions teams ask before choosing a self-hosted Duo model

How long does choosing a self-hosted Duo model take?

The timeline depends on how much you must confirm before a pilot can start. The main driver is the number of features in scope, because each feature adds a row of the support matrix to check and a workflow to test. Pilot scope comes next. A pilot with a few developers on one team moves faster than one that covers every group. Environment constraints such as offline operation, GPU availability and data residency rules often add review cycles that take longer than the model comparison itself. DeployOpen does not publish a standard duration. Your written scope sets the phases, owners and dates once these inputs are known, so ask for them in that document.

Can our team choose the model without outside help?

Yes, many teams can, provided they have someone who administers GitLab, someone who runs GPU infrastructure and a way to test developer workflows before rollout. The support matrix is the starting point. The decision comes down to matching each feature you need with a model family rated for it in your GitLab version. Outside help is most useful when the matrix leaves gaps for your use cases, when offline operation narrows the options, or when your platform and security teams lack time to run a pilot. DeployOpen plans and implements self-hosted platforms inside your own infrastructure under fixed-scope agreements, with scope and deliverables agreed in writing before work starts. Whichever route you take, decide who approves the model and any accepted gaps before the pilot begins.

What information do you need from us to scope the work?

Before scoping a model decision, the team needs these inputs from you:

The GitLab version you run now and the version you plan to run, including the upgrade path between them.
The number of users and the number of developers who expect to use Duo at the same time.
An inventory of the GPU and server hardware you can use, with its location and network placement.
The Duo features in scope, and whether the deployment must run fully offline or can use connected services.
The named security and compliance approvers who will sign off on the chosen model and any accepted gaps.

Missing items usually extend the discovery phase, so gathering them early keeps the written scope accurate.

What drives the cost of choosing a self-hosted Duo model?

Most of the cost is effort and infrastructure rather than a single license line. Effort is largely one-time: research against the support matrix, pilot design and testing, and documentation for security review. GPU capacity is the main infrastructure cost and recurs for as long as the models run, so the model sizes and the number of concurrent developers you plan for matter most. Pilot effort rises with the number of features and teams involved. DeployOpen does not publish a price list, and a quote follows from the written scope. Ask for a quote that lists each phase, who performs it, the hardware it assumes, and what is excluded, including any GPU purchase or hosting cost you pay directly to a provider.

Is a Partial rating acceptable for the features we need?

It can be, but only as a documented decision for each feature. Start by reading the definitions GitLab gives for Full, Partial and Limited in the support matrix for your version, because the exact meaning should come from GitLab. Then test the workflows your developers will actually use instead of generic prompts. Record what works, what does not, and which team accepts each gap, with a review date tied to your next upgrade. If a feature your teams depend on is rated Limited, plan the first rollout without it. Security and engineering leads should approve the final list before any user gets access.

What changes in model support when we upgrade GitLab?

The support matrix is tied to a GitLab version, so every upgrade is a reason to check your model choice again. Re-read the ratings for your target version, then confirm that the AI gateway and model components are compatible with it. Plan the upgrade path first. Since GitLab 17.5, required upgrade stops fall at versions x.2, x.5, x.8 and x.11, and your upgrade must pass through them. Security fixes reach the previous two monthly releases, but bug fixes go only to the current stable release, so staying close to current matters. Record the ratings you accepted on the old version and confirm each one still holds after the upgrade, because a rating may change.

Does a fully offline deployment limit which models we can choose?

It can. GitLab documents offline and air-gapped operation and provides separate guidance for running GitLab Duo Self-Hosted offline, so start with that guidance and do not assume the online setup applies. Check each feature in scope for anything the hybrid option would send outside your network. Each of those features needs a self-hosted alternative or a documented gap. Confirm licensing for offline use as well, since the setup depends on the add-on and license type you hold. Ask your GitLab contact to confirm current behavior for your version before you finalize the model list.

How DeployOpen works on this

DeployOpen treats the model choice as a written scoping decision before any hardware is ordered. We map your Code Suggestions, Chat and Agent Platform needs to the support matrix for your GitLab version, then build and test the chosen model path inside your own infrastructure.

How we scope

Confirm which Duo features your teams need first: Code Suggestions, Chat or the Agent Platform
Record the GitLab version, edition and AI gateway version the model decision must match
List which features are rated Full, Partial or Limited for each candidate model family
Agree in writing which features are in scope, with engagements fixed-scope before work starts

How we build

Stand up a test path with the chosen model family behind the self-hosted AI gateway
Run each in-scope feature against representative developer tasks and record observed gaps
Document the configuration, version pairings and any feature limits accepted by your team
Check that the deployment works with no internet connectivity if your environment is offline

How we hand over

Deliver a written model decision record that names the chosen family, ratings and known limits
Provide runbooks for model updates, gateway version alignment and rollback to a prior model
Walk your platform, security and engineering leads through the test results and open risks
Hand over configuration files, access notes and a list of items to confirm with GitLab before rollout

How we support

Review the model decision when GitLab publishes a new release or support matrix changes
Available as a retainer for ongoing upgrades, re-testing and support after handover
Re-run feature checks when you add a model, change hardware or move between online and offline licensing
Ask your quote to separate one-time scoping and build work from recurring support terms

Choosing a model for GitLab Duo Self-Hosted