Measuring DORA metrics in self-managed GitLab
GitLab computes DORA metrics as an Ultimate feature, so an engineering leader who wants deployment frequency, lead time for changes, time to restore service and change failure rate from a self-managed instance needs to confirm the tier before anything else. The numbers are only as good as the production environment and incident setup behind them. If deployments are not recorded against a production-tier environment, or incidents are not tracked in GitLab, the dashboard will show figures that look precise but describe the wrong thing. Deployment frequency counts deployments, and GitLab only counts those that land in production-tier environments, so the first check is how environments are tiered in each project. Lead time for changes measures the time a change takes to reach production. Time to restore service and change failure rate both depend on GitLab incidents, so a team that handles outages in a separate tool will have no restore time to report. Before rollout, ask who owns environment tier assignments and where incidents are opened, and confirm both against your own projects.
Pull-based deployments need extra work. GitLab does not create deployment records for them automatically, so the deployment process has to call the Deployments API against a production-tier environment each time a release reaches production. Agree on the API fields the pipeline will send, test the call in a non-production project, and confirm the production record appears in the metrics before relying on it. Check the current DORA metrics documentation for the exact requirements of your GitLab version, since the figures leadership reads depend on those details.
What each DORA metric needs from your GitLab setup
Each of the four DORA metrics reads a different part of your GitLab data. Deployment frequency and lead time depend on how environments are tiered, while time to restore and change failure rate depend on incident records. A gap in either input rarely produces an error, so the dashboard simply reports less than actually happened.
Deployment frequency
Deployment frequency counts deployments only in environments marked with a production tier. If your release jobs deploy to environments named without a production tier, or use a generic name such as deploy-prod with no tier set, GitLab does not count those releases, and the metric shows a lower or zero rate. Check each production environment's tier in the project settings before you trust the trend. Confirm the tier mapping with a test deployment in a non-production project first.
Lead time for changes
Lead time measures the time from a commit to its deployment to a production-tier environment. It depends on deployments being recorded against that environment, so the start and end of the interval both come from GitLab data. If a team deploys outside GitLab, or merges without the commit reaching the deployment record, the interval can be measured from the wrong commit or skipped. Review a sample of recent changes and compare their timestamps with the actual release.
Time to restore service
Time to restore service needs GitLab incidents, because the metric measures the gap between an incident opening and its closure. Teams that handle outages in chat or a separate ticketing tool and never open a GitLab incident leave this metric empty. Agree on who opens and closes incidents, and whether an incident must be linked to a production deployment. Ask your team to test the flow with a simulated outage before you rely on the figure.
Change failure rate
Change failure rate compares deployments to production with the incidents that followed them. Without incidents, every deployment looks successful, which reports a rate near zero whether or not changes caused harm. The metric also depends on incidents being connected to the deployment that caused them. Define in writing what counts as a failed change, such as a rollback or hotfix, and confirm that your incident process records it.
Pull-based deployment records
In pull-based models such as GitOps, GitLab does not create deployment records on its own, so deployment frequency and lead time can drop to zero without any alert. Create a deployment record through the Deployments API against a production-tier environment at the moment the cluster reports a successful sync. Confirm that the pipeline or controller that calls the API handles failures, retries and environment names consistently.
How production-tier environments decide which deployments count
GitLab counts a deployment toward DORA metrics only when it targets an environment in the production tier. Staging, testing, review and other non-production environments generate deployment records that GitLab ignores for deployment frequency and lead time. Before you turn metrics on, list every environment your pipelines deploy to and decide, for each one, whether a failed or successful release there should represent a change reaching customers. Only the ones that do should carry the production tier. Environment names and tiers are set in the project's environment configuration, so the mapping is a platform decision that should be written down, not left to individual pipeline authors.
Platform owners should run a review step before any dashboard is shared. Ask each delivery team to confirm which environments are production, check that the tier on each environment matches that answer, and look for review apps or preview environments that were given a production-like name or a production URL. A single mislabeled environment distorts every metric: a staging deployment counted as production inflates deployment frequency, and a production service that was tiered as something else can hide its real change failure rate. Record the decisions and the reviewer in your change management evidence, because auditors will ask how the population of counted deployments was chosen.
Confirm two points in your own environment before the review closes. First, whether any production deployments run outside GitLab's standard deployment jobs, since those need explicit recording as described in the next section. Second, whether the people who own environment settings are the same people who can edit pipeline definitions, because a change to a pipeline that retargets a job to a different environment changes the metrics without any visible release. Restricting who can edit environment configuration and reviewing changes to it keeps the measurement stable enough to compare one quarter with the next.
Setting up DORA metrics in self-managed GitLab
This sequence turns a self-managed GitLab instance into a source of delivery measurements. Each step builds on the one before it, so the dashboard reflects production deployments and incidents that your teams can trace back to real releases.
Confirm the Ultimate license on the instance
The platform administrator checks the license on the self-managed instance from the Admin Area and confirms that the Ultimate tier is active, because DORA metrics are an Ultimate feature. Confirm the tier with your GitLab account representative if the license shows any ambiguity, and do not assume that a Premium or Free instance will show the dashboard. The step is done when the license page lists Ultimate and a project's analytics section offers the DORA metrics view for a project in a group you control.
Label production environments as production tier
The project maintainer defines each environment that represents production in the project's environment settings and sets its tier to production. GitLab counts deployments toward deployment frequency and lead time only when they land in production-tier environments, so staging and test environments should use their own tiers. The check is that each production environment appears with the production tier in the environments list, and that a test pipeline deploying to a non-production environment does not add to the production count.
Configure incident tracking for restore and failure metrics
The project team enables incidents for the projects whose production failures should count, and makes sure that someone creates an incident for each service disruption, since time to restore service and change failure rate depend on incident records. Agree a written rule on who opens an incident, when it is closed and how a failed deployment is linked to it. The check is a test incident that opens, is linked to a deployment and closes, after which it appears in the DORA metrics data for the project.
Record pull-based deployments through the Deployments API
For GitOps or other pull-based delivery, where the cluster or host pulls a change and GitLab does not run the deploy job, the pipeline or controller must call the Deployments API against the production-tier environment. GitLab does not create these deployment records on its own for pull-based flows, so the integration owner should script the call, record the release identifier and status, and handle failed attempts. Confirm the authentication method and the permissions of the token used, then check that a test deployment appears in the environment's deployment history.
Validate the dashboard against known release history
The delivery lead compares the DORA metrics dashboard with a release history that the team already trusts, such as a list of production changes from change records or the tags and deployments they expect to see for a chosen period. Check that each known production release is counted once, that failed or rolled-back deployments are represented as intended, and that incidents match the outages recorded by operations. Investigate any gap in the source records before relying on the figures, and record the validation result in your change documentation.
Push and pull deployments and how each is recorded
In a push-based model, a CI/CD job deploys the change directly to a target, such as a cluster, a virtual machine or a cloud service. Because the job names an environment, GitLab can attach the deployment to that environment and record it as the job runs. The record comes from the pipeline itself, so teams do not write extra API calls for each release. Your work is mainly in naming environments and tiers so that production deployments are counted, and in confirming that the job finishes only when the deployment has succeeded.
- Environment declaration: The job sets an
environment:keyword with a name and, where relevant, adeployment_tierof production so GitLab knows which deployments count. - Job outcome: A failed or cancelled deploy job should not look like a successful release, so review how retries and manual gates affect the recorded result.
- Protected environments: Restrict who can deploy to production, which also clarifies who is accountable for each recorded deployment.
- Multiple targets: Jobs that deploy to several regions or clusters need separate environments, or the counts will blur together.
- Rollbacks: A rollback run through a pipeline is also a deployment, so decide whether it should count toward frequency.
Check a sample production deploy in your own project to confirm it appears under the expected environment with the production tier.
In a pull-based model, a controller inside the cluster or on the target watches a Git repository and applies changes it finds there. The CI pipeline may only update a manifest, so GitLab sees a commit but not the moment the change reached production. GitLab does not create deployment records automatically for this model. Your team must create them with the Deployments API against an environment that carries the production tier, typically after the controller reports that the sync succeeded.
- Explicit API calls: Add a step, often triggered by the controller or a webhook, that creates the deployment record through the Deployments API.
- Production-tier environment: Create the environment with the production tier before the first call, otherwise the deployment will not count toward DORA deployment frequency.
- Status accuracy: Record success only after the controller reports a healthy sync, not when the manifest commit is merged.
- Ownership of the script: The integration is custom code, so assign an owner who maintains it through controller and GitLab upgrades.
- Backfill risk: Gaps appear when the integration fails silently, so alert on missed records.
Confirm how your controller reports sync status, including whether it exposes events or webhooks, before choosing how the record gets written.
The accuracy of a pull-based record depends on what your controller reports and how reliably that signal reaches GitLab. Controllers differ in the events they emit, the status fields they expose and how they handle failed or partially applied syncs. A deployment that is recorded as successful while the controller is still degraded will inflate deployment frequency and hide change failures. Treat the controller's status model as an input you need to verify, not an assumption.
- Sync state: Identify which state means the new revision is live and healthy, and which means it is pending, degraded or rolled back.
- Event source: Decide whether the record is written from a webhook, a polling job or the controller's own hook, and what happens when that source is unavailable.
- Revision identity: Store the commit SHA or revision in the deployment record so you can trace each recorded deployment to a change.
- Multiple applications: One commit may update several applications, so decide whether each one gets its own record.
- Failure paths: Define how a failed sync is written, so change failure rate and restore time can be measured from it.
Before relying on the metrics, compare a month of GitLab records against the controller's own history for the same production applications.
Whichever model you use, the metrics are only as good as the records behind them. Verification means proving that each production change appears once, under the right environment, with the right outcome. Mixed estates are common, with some services deployed by pipelines and others reconciled by a controller, so the verification plan should cover both paths in the same review. Run the check against real releases before anyone reads the dashboard for decisions.
- Sample reconciliation: Pick recent production releases from each model and confirm one deployment record exists for each.
- Duplicate detection: Check that a pipeline deploy and a controller sync do not both record the same release.
- Tier review: Confirm production environments carry the production tier and that non-production environments do not inflate counts.
- Access control: Limit who can create deployment records by hand, since manual entries weaken the audit trail.
- Documented mapping: Keep a table that maps each service to its deployment model, environment and owner.
Ask your GitLab administrator and platform team to confirm the expected record count for one sample release before the metrics go into a report.
Choosing how to record incidents and deployments
This table compares the three ways a team can feed GitLab's DORA metrics: CI-driven deployments, Deployments API calls from pull-based tooling, and incident sources for restore time and change failure rate. Read each row for when the option fits, what it requires and where it tends to break. Items marked to confirm depend on your instance version or your tooling, so check them before you commit.
| Option | When it fits | What it requires | Main risk |
|---|---|---|---|
| CI-driven deployments | Pipelines run the deploy job directly against an environment that is set as production tier | An environment in the production tier, a deploy job that targets it, and an Ultimate license for DORA metrics | Jobs that deploy to production without targeting a production-tier environment are not counted, so frequency looks lower than reality |
| Deployments API from pull-based tooling | A GitOps controller or other pull-based tool applies changes and GitLab never runs the deploy job | A token with rights to call the Deployments API, a production-tier environment, and a script or controller hook that posts a record after each sync | GitLab does not create these records automatically, so a missed hook silently drops deployments from the metrics |
| Incidents created from alerts or manual entry | Teams open a GitLab incident for each production-impacting event, so restore time has a start and end | Incident management enabled on the project, an agreed rule for when an incident is opened and closed, and an Ultimate license | Incidents opened late or closed early distort time to restore service |
| Incidents from external alerting | Monitoring tools already page on-call and you want incidents created in GitLab from those alerts | An alert integration configured for the project, alert payload mapping, and confirmation of the supported alert sources for your instance version | Alert noise creates incidents that inflate failure counts, so filter alerts before they reach GitLab (confirm supported sources with the vendor) |
| Change failure rate from linked incidents | Teams tie production incidents to the deployment that caused them so the failure rate has a numerator | A written linking convention, such as a deployment reference on each incident, and a review step in the incident process | Attribution is manual unless your tooling supports it, so the rate depends on how consistently people link records |
| Environments and tiers | Staging and production are separated, and only real production targets feed the metrics | Clear environment naming and tier assignment, with access rules for who can change tiers | Renaming or retiring environments can break history, so agree the mapping before the first measurement period |
| Instance version and tier check | Before the first report, to confirm that the DORA metrics features behave as your team expects | Confirmation with the vendor or in the release notes for your instance version of how each DORA metric is calculated | Documentation and behavior change between releases, so record the version you measured against |
Access, data quality and audit considerations
DORA figures are only as credible as the records behind them. Deployment records, incident entries and the tokens that write them all shape what a reviewer can trust, so access and evidence decisions should be made before the first dashboard is shared.
Who can record deployments
Pull-based deployments do not create deployment records automatically, so your pipelines or a service account must call the Deployments API against a production-tier environment. Decide which projects and jobs may create those records, and limit the token to the minimum role needed. Confirm in your own instance which project roles can create deployments and who can change environment tiers, because a tier change alters which deployments count toward deployment frequency.
API token ownership and rotation
Deployment records written by a personal access token appear under the owner's identity, which makes the record depend on a person staying in the organization. Prefer project or group tokens owned by a service identity, with an expiry and a named rotation owner. Confirm with your administrators how tokens are stored in CI variables and whether masked and protected variables cover the production pipelines that write records.
How incidents are entered
Time to restore and change failure rate depend on GitLab incidents, so the quality of incident entry decides those two metrics. Define who opens an incident, when the clock starts, and who closes it. Confirm whether your teams open incidents from alerts, from issues or by hand, and whether restricted visibility is needed for security-sensitive incidents without removing them from the metric.
Protecting incident and deployment history
Deployment and incident records are data that teams may later need to edit or delete. Decide who may modify environments, deployments and incidents, and whether edits should be logged. Confirm with your administrators how audit events record changes to projects and environments, since a silent edit to history undermines every trend line built on it.
Guarding against gamed metrics
A team measured on deployment frequency can split or repeat deployments, and a team measured on time to restore can close incidents early. Reviewers should compare DORA figures with the underlying pipeline and incident records, not only the dashboard. Illustratively, a deployment job that runs on every commit to a production-tier environment will inflate frequency without delivering change, so check what each counted deployment actually shipped.
Retaining evidence for change reviews
Change advisory and audit reviews usually ask for the record of who approved and deployed a change, not a chart. Keep merge request approvals, pipeline logs, deployment records and incident timelines together, and set retention on logs and artifacts so they outlive the review cycle. Confirm your retention periods with your compliance owner and test an export before an auditor asks for one.
Questions teams ask before measuring DORA metrics
Do DORA metrics in GitLab require the Ultimate tier?
Yes. DORA metrics are an Ultimate feature in GitLab, covering deployment frequency, lead time for changes, time to restore service and change failure rate. The GitLab pricing page lists Ultimate as custom priced, so the licensing question belongs in your quote conversation. Free and Premium do not give you these dashboards, so if you only need the raw signal, you can also record deployments and incidents yourself and calculate the figures in your own reporting. Confirm current tier boundaries on the pricing page before you commit, since packaging can change between releases.
What does the setup effort for DORA metrics depend on?
The effort depends mostly on how your deployments are triggered and how your incidents are tracked. Push-based pipelines that deploy to environments GitLab already knows about need the least work. Pull-based deployments, where a GitOps controller or external tool applies changes, need an extra step: GitLab does not create deployment records automatically, so you must call the Deployments API against a production-tier environment. Other drivers include how many services and environments you have, whether incidents are already tracked as GitLab incidents, and how consistent your environment naming is. Ask any partner to scope these items in writing before work starts.
Can an internal team set up DORA metrics without outside help?
Often yes, if the team already administers GitLab, runs its CI/CD pipelines and has someone who understands the deployment chain end to end. The tasks are configuration-heavy rather than deep engineering: defining production-tier environments, wiring deployment jobs, creating incident workflows and, where needed, scripting calls to the Deployments API. The harder part is agreeing on definitions across teams, such as what counts as a deployment and when an incident starts and ends. If those definitions are not settled, an outside partner can help facilitate the decision, but your own leaders must own it.
What does the work require from engineering and platform staff?
Engineering teams need to tag deployment jobs with the correct production environment and record incidents consistently. Platform engineers need access to the GitLab project and group settings, permission to configure environments, and a way to script API calls with a token that follows your access policy. Someone should own the mapping between services, environments and teams, and review the numbers each month. Plan for a short period of changed habits, because teams that previously deployed without recording an incident will need guidance on when to open one. Document the procedure so the metrics survive staff changes.
How do we validate that the DORA numbers are correct?
Start by sampling recent production deployments and confirming each one appears as a deployment record against the expected production-tier environment. Then compare a sample of incidents with the timestamps your responders recorded, checking that time to restore reflects when service was actually recovered. Look for double counting from retries, rollbacks recorded as new deployments, or staging environments accidentally treated as production. Keep a short written test plan with expected results so the check can be repeated after changes to pipelines or environment names. Independent reviewers can assess the procedure, but they do not certify the metrics.
What recurring costs should we expect after the metrics are live?
The recurring costs are the GitLab subscription at the tier you hold, the time staff spend maintaining environment definitions and deployment scripts as pipelines change, and the review effort to keep incident data accurate. If you engage a partner, ask whether the scope includes ongoing support or a retainer, since most engagements are fixed-scope with agreed deliverables. Ask any quote to separate one-time setup, validation and training from recurring support, and to state who maintains the Deployments API scripts after handover.
Do DORA metrics affect audit or compliance evidence?
They can support evidence for change management, but they are not an audit result. Deployment records and incident records give an auditor a traceable history of production changes, provided the environments, approvals and retention are configured as your control design requires. DeployOpen prepares systems, controls and evidence for assessments, but independent auditors issue any audit report or attestation. Confirm with your assessor which records they will accept and how long your instance retains them.
How DeployOpen works on this
DeployOpen plans, implements, integrates, documents and supports self-hosted open-source platforms inside your own infrastructure. For DORA metrics in self-managed GitLab, we start by agreeing which environments, incident records and deployment steps count, then build and document the measurement path so your team can run it after handover.
How we scope
- Confirm which GitLab edition and tier your organization runs, since DORA metrics require GitLab Ultimate.
- Map which environments are production-tier and which deployments must be counted for frequency and lead time.
- Agree in writing how incidents are opened and closed so time to restore and change failure rate have a reliable source.
How we build
- Configure production-tier environments and confirm that pipelines and deployment jobs report to them correctly.
- Script the Deployments API calls for pull-based deployments, such as GitOps workflows, so each production rollout creates a record.
- Test the metric pipeline against sample deployments and incidents in a non-production instance before relying on the dashboards.
How we hand over
- Document the environment naming, incident labeling and API usage rules your teams must follow to keep metrics accurate.
- Walk through how each of the four DORA measures is calculated in your instance, including what each one excludes.
- Hand over scripts, configuration and runbooks in your repositories so your own engineers can maintain them.
How we support
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
- Ongoing support, upgrades and checks after GitLab version changes are available as a retainer.
- Ask any partner to confirm how a GitLab upgrade or tier change would affect your DORA metric data before you proceed.
