Backing up GitLab secrets and configuration

Why GitLab secrets and configuration need their own backup

A working database and repository backup does not rebuild a self-managed GitLab instance on its own. The secrets file, gitlab.rb, SSH host keys and TLS material must be protected separately because the backup archive does not contain them. Every production instance needs this protection, and it should be in place before the first restore test and before any rebuild or migration to new hosts. The backup archive and the configuration files are separate artifacts with separate handling: the archive holds data, while the configuration directory holds the keys and settings that let that data be read and served. Treat them as two backup jobs with their own storage, access controls and restore steps. This page builds on the basic procedure in the GitLab backup and restore documentation.

The secrets file, /etc/gitlab/gitlab-secrets.json on Linux package installations, holds the keys GitLab uses to encrypt sensitive values stored in its database. Those values include CI/CD variables, two-factor authentication secrets and tokens for runners and integrations. If the file is lost, a restored database contains values GitLab cannot decrypt. Users can lose access to two-factor sign-in, pipelines lose their variables, and integrations must be configured again. Confirm the current list of encrypted fields for your version in the GitLab documentation, since it can change between releases. Copy the secrets file every time it changes, store the copy encrypted, and limit access to the people who perform restores. Check that the copy matches the instance it came from before you rely on it.

What gitlab-secrets.json protects

The secrets file holds the keys GitLab uses to encrypt and verify data stored in its own database and configuration. Each category below depends on a key that lives only in that file, so a restore that lacks it can leave data present but unreadable. Confirm the exact set of keys for your GitLab version before you write a restore runbook.

Database encryption keys

GitLab encrypts selected sensitive database columns, such as some token and credential fields, with keys stored in the secrets file. A database restore brings back the ciphertext but not the key that decrypts it. Ask your team to list which encrypted columns exist in your version and to confirm, in a test restore, that affected features still open their stored values.

CI/CD variable encryption

CI/CD variables marked as masked or protected are stored encrypted in the database. Without the matching key, pipelines that depend on those variables fail in ways that can look like a runner or script problem. Check whether your pipelines depend on stored variables, and confirm with a test pipeline after restore that secrets resolve as expected.

Two-factor authentication secrets

Stored two-factor authentication seeds are encrypted with a key from the secrets file. If that key is missing after a restore, users may be unable to complete two-factor sign-in, which can lock out administrators as well as staff. Plan a documented recovery path for account access before cutover, and confirm which recovery codes your users hold.

Runner and integration tokens

Tokens that link runners, Kubernetes integrations and other connected systems to GitLab rely on values held in the secrets file or stored encrypted in the database. A rebuilt instance may show these integrations as present but no longer working. Inventory each integration and plan to re-register runners or rotate tokens where your version requires it.

Secrets file as a separate artifact

The secrets file is not part of the repository or database backup archive, so it must be copied separately to protected storage. Restrict who can read it, since it contains live key material. Confirm with your GitLab version's documentation how the file is named and where it sits on each node in a multi-node installation.

Keys for multi-node installations

In a multi-node deployment, every node that reads encrypted data must use the same secrets. A node built with a freshly generated file will fail in subtle ways even when the rest of the cluster is healthy. Confirm that all application nodes hold identical copies of the secrets file, and verify this as part of each restore test.

Which configuration artifacts to back up and how

This table compares the configuration artifacts that a GitLab rebuild depends on, from the secrets file to Runner and Kubernetes configuration. Read each row for where the artifact lives on a Linux package install, how sensitive it is, and how it should be stored apart from the backup archive.

Artifact	Purpose and location	Sensitivity and change frequency	Suggested storage handling
gitlab-secrets.json	Holds keys that encrypt database columns such as CI variables and two-factor secrets. Sits in /etc/gitlab on a Linux package install.	Highest sensitivity. Changes rarely, but only after a secret rotation or a new install.	Encrypted, access-restricted store kept separate from the backup archive. Never commit it to version control.
gitlab.rb	Main Omnibus configuration for URLs, components and settings. Sits in /etc/gitlab on a Linux package install.	Moderate sensitivity because it may contain passwords or tokens. Changes with every tuning or feature change.	Version control with secret values removed or moved to a secrets store. Keep a dated copy with the backup set.
SSH host keys	Identify the GitLab server to clients using SSH Git access. Sit in /etc/ssh on the host, with GitLab Shell settings tied to them.	High sensitivity because private keys allow impersonation. Changes only on rebuild or when keys are regenerated.	Encrypted, access-restricted storage. Restoring them keeps existing clients from seeing host key warnings.
TLS certificates and private keys	Serve HTTPS for the GitLab URL and registry. Usually referenced from /etc/gitlab/ssl or a file path in gitlab.rb.	High sensitivity for private keys. Changes at renewal, which is often on a fixed schedule set by your certificate authority.	Private keys in encrypted, access-restricted storage. Public certificates can be stored with the configuration record.
Runner config.toml	Defines Runner registration tokens, executors and autoscaling settings. Lives on each Runner host, often in /etc/gitlab-runner.	High sensitivity because it can hold cloud credentials and runner tokens. Changes when fleet capacity or executors change.	Keep in encrypted storage or a secrets manager, and record the Runner list so tokens can be reissued after a rebuild.
Kubernetes values files	Hold chart settings when GitLab runs through the Helm chart. Stored in your deployment repository rather than on a server.	Moderate sensitivity, lower if secrets are referenced rather than inlined. Changes with each chart upgrade.	Version control with secrets referenced from an external store. Pin the chart version alongside the values file.
Restore record for the set	Notes which GitLab version, edition and backup ID belong with each artifact set. Ties the other rows together during a rebuild.	Low sensitivity. Must be updated after every upgrade or key rotation.	Stored with the runbook, and checked during restore tests so the artifacts and backup archive match.

How the backup changes for each installation method

The Linux package is the method most teams use on virtual machines or bare metal. It keeps the main configuration file and the generated secrets file in the standard configuration directory on the GitLab host, and the package reads both during reconfigure. Because the package manages these files, the backup plan is usually the simplest of the five methods, but it still has to be written down and tested.

Configuration file: Capture the gitlab.rb file from the configuration directory on every node, including any node-specific overrides.
Secrets file: Capture gitlab-secrets.json from the same directory, since the backup archive does not contain it by default.
TLS material: Capture the certificate and key files referenced in gitlab.rb, along with any custom certificate authority bundles.
SSH host keys: Capture the host key files under the OpenSSH server configuration path so the Git SSH endpoint keeps its fingerprint after a rebuild.
Permissions: Store copies with restricted access, because these files contain credentials and private keys.

Confirm the exact file paths against the Linux package documentation for your release, and check which files your instance actually uses before you write the backup job.

The Helm chart stores configuration in Kubernetes objects rather than in files on a host. Settings come from values files and the rendered objects, and the secrets used by GitLab are held in Kubernetes Secrets within the release namespace. A backup therefore has to capture the Secret objects and the values used to render them, not just the chart version.

Secrets as Kubernetes objects: Export the Secret objects that hold the GitLab secrets, including the one that corresponds to gitlab-secrets.json, using a method that does not print values to logs.
Values files: Keep the exact Helm values files used for each release, under version control with restricted access.
TLS and ingress material: Capture TLS Secrets, or the certificate source your certificate manager uses, along with the ingress configuration.
Stateful dependencies: Keep the secrets for external PostgreSQL, Redis and object storage connections, since the chart references them.
Encryption at rest: Confirm how the cluster encrypts Secret objects in etcd, and include the key management step in the recovery plan.

Confirm the object names and the secret keys against the Helm chart documentation for your chart version, because names change between releases.

A Docker deployment usually mounts the configuration and data directories from the host into the container. The configuration directory on the host is the source of truth for gitlab.rb and gitlab-secrets.json, and the container reads them at start. Backups should capture the host directories that are mounted into the container, not only the image tag, because the image contains no instance-specific secrets.

Mounted configuration directory: Capture the host directory mapped to the container configuration path, which holds gitlab.rb and gitlab-secrets.json.
Compose or run definitions: Keep the Compose file or run command, the image tag and the volume mappings under version control.
Host SSH keys: Capture the SSH host keys if the container exposes the Git SSH port, because the keys are generated on first start and a new container may create different keys.
Certificates: Capture the certificate and key files mounted into the container.
Image tag: Record the exact image tag and digest, so a restore starts the same version.

Confirm the mount paths and the container-side paths against the Docker installation documentation for your image.

How to back up secrets and configuration without breaking restores

This sequence produces a current, access-controlled set of GitLab secrets, configuration, SSH host keys and TLS material, each tied to the GitLab version it belongs to, so a restore or full rebuild can start from files that match the backup archive.

Inventory every installed artifact and its location
Before copying anything, list the files that make your GitLab instance work outside the database and repositories. For a Linux package install this normally includes /etc/gitlab/gitlab-secrets.json, /etc/gitlab/gitlab.rb, the SSH host keys under /etc/ssh, any TLS certificates and private keys referenced in gitlab.rb, and any custom files your team has added. Confirm the exact paths against your own installation, since they vary with install method and with Helm or Operator deployments. Record the owner, permissions and host of each file in a short inventory. The step is done when every file referenced by your configuration appears on the list and nothing on the list is unidentified.
Choose encrypted storage with restricted access
Decide where the copies will live before the first copy is made. The location should be encrypted at rest, separate from the GitLab host, and readable only by a named group of administrators, with access logged where your platform allows it. Because gitlab-secrets.json contains keys that protect data inside the database, treat the storage with the same care as the backup archive itself, and do not place copies in shared repositories or ticket attachments. Ask your security team to confirm the access model in writing. The step is done when the storage location, its encryption method and its access group are documented and a test write by an authorized administrator succeeds while an unauthorized account is refused.
Copy secrets and configuration after each change
Copy the secrets file, gitlab.rb and the other inventoried files into the restricted location. Repeat the copy after every change to these files, after every run of gitlab-ctl reconfigure that alters them, and before every upgrade. Keep each copy in a dated or sequenced folder rather than overwriting the previous one, so you can recover an earlier state if a change causes a problem. Preserve file permissions and ownership in the copy process, or record them, because a restored file with the wrong mode can stop services from starting. The step is done when a new dated copy exists for each changed file and its size and checksum match the source on the GitLab host.
Store SSH host keys and TLS material under the same controls
Copy the SSH host keys and the TLS certificates and private keys into the same encrypted location, under the same access group and retention rules as the secrets file. SSH host keys matter because clients cache them, so a rebuilt server that presents new keys will trigger host key warnings for every developer and runner. TLS private keys and their chains are needed to serve HTTPS on a rebuilt host without reissuing certificates in a hurry. Record which certificate authority issued each certificate and when it expires, so renewal is not a surprise during a rebuild. The step is done when each SSH host key pair and each certificate chain is present in the store, and you can list the expiry date of every certificate from the stored copies.
Record the GitLab version and edition beside each copy
Write a small metadata file next to every set of copies. It should contain the GitLab version and edition (Community or Enterprise) that produced the files, the date of the copy, the host name, and the identifier of the matching backup archive if one was taken in the same window. This matters because GitLab restores expect the same version and edition as the backup, and the secrets must be restored alongside the archive they belong to. Confirm the current version and edition from the running instance rather than from memory. The step is done when every copy folder contains a metadata file, and each file names a version and edition that you can confirm from the host.
Verify that copies are readable and match live files
Test each copy. Open the secrets file and confirm it is valid JSON and contains the expected top-level keys without printing the values to a terminal or log. Compare the checksum of each stored file with the checksum of the live file on the GitLab host, and confirm that the stored SSH host public keys match the host's current public keys. For TLS, confirm that each stored certificate matches its private key and that the chain loads. Run this check on a schedule and after every change, and record the result. The step is done when every checksum matches or any difference has been explained by a recorded change, and when a test read from an authorized account succeeds without exposing secret values.

What breaks when gitlab-secrets.json is lost after a restore

A database restore without the matching gitlab-secrets.json file is a partial recovery, and the instance can look healthy while it is. GitLab encrypts several values in the database, including two-factor authentication secrets, CI/CD variables, and tokens used by integrations. The restored rows still exist, but the keys that decrypt them live in the secrets file. Without that file, those references point to values the instance cannot read. Users may find they cannot complete two-factor sign-in because their enrollment cannot be decrypted, stored secrets may be unreadable, and integrations that depend on saved tokens may stop working.

The failures are rarely immediate. Login pages load, projects open, and repositories clone, so a quick smoke test passes. The problems surface when someone runs a pipeline that uses a protected variable, when a webhook or mirror authenticates with a saved token, or when a user with two-factor enabled tries to sign in from a new device. Because these paths are not exercised by a routine check, a restore test must include a functional pass that runs CI jobs, exercises each integration you rely on, and confirms two-factor sign-in for a test account. Your team should also decide who owns that test list and how often it runs.

Confirm the exact symptoms, the affected encrypted fields, and the recovery steps against the GitLab backup and restore documentation for your installed version before you write your runbook, because the list of encrypted values can change between releases. Ask your restore test to record which features were checked and which were not, so that any gap is a known risk rather than a surprise during an incident.

Protecting keys and secrets in backup storage

The secrets file and configuration files unlock everything a backup contains, so their copies need the same care as the data itself. The cards below cover where copies live, who can read them, how they are encrypted, and what to do when exposure is suspected. Each one is a decision your team should make and record before the first backup runs.

Restrict access to backup copies

Store copies of gitlab-secrets.json and gitlab.rb in a location only the people who perform restores can reach. A readable copy of the secrets file lets anyone holding it decrypt database fields and forge trusted tokens, so treat it like a private key. Confirm that file permissions on the copy match the originals, that service accounts used by backup jobs have write access only, and that no shared team drive sits in the path.

Encrypt copies at rest

Encrypt the secrets and configuration copies with keys held apart from the storage that holds them. Object storage server-side encryption protects against disk theft but not against someone with bucket read rights, so decide whether your organization needs client-side encryption with a key management service. Ask your storage team which key owner, rotation schedule and recovery path apply, and test that a restore can still decrypt the copy.

Separate secrets from database backups

Keep the secrets copy in a different store, account or credential set from the database and repository backup archives. If one compromise exposes both the encrypted database dump and the key that decrypts its sensitive columns, the encryption adds little. Illustrative example: a backup account that can write archives but cannot read a separate secrets vault would block one route to a full restore. Document which identities can reach each store.

Rotate after suspected exposure

If a secrets copy or a configuration file with embedded credentials may have been read by someone outside the restore group, plan rotation before anything else. Rotating the secrets file itself is disruptive because it protects stored tokens and encrypted fields, so the procedure should be rehearsed and tied to the GitLab version you run. Ask your GitLab administrator to confirm the documented rotation steps for that version, and record who approved each change.

Audit who reads the copies

Enable access logging on the storage holding secrets and configuration copies, then review it on a schedule your security team sets. Look for reads outside restore windows, copies made to personal locations, and service accounts used interactively. Confirm that the audit trail itself is stored where the backup administrators cannot delete it, so an insider cannot erase their own access.

Keep configuration out of version control

Do not commit gitlab.rb or gitlab-secrets.json to a Git repository, including private or internal projects, because history keeps every value ever committed and clones spread it to laptops and CI jobs. If you manage the file with configuration tooling, keep secret values in a separate vault and reference them at deploy time. Scan repositories for these file names and for common credential patterns before adoption, and confirm with your team which tool performs the scan.

Questions teams ask before backing up GitLab secrets

Is the standard GitLab backup enough to recover a self-managed instance?

No. The standard backup task captures application data such as repositories and the database, but it does not replace the secrets file and configuration. GitLab's backup documentation treats gitlab-secrets.json as a separate item that must be preserved alongside the archive. Without it, encrypted values in the database, including CI/CD variables, two-factor secrets and some tokens, cannot be decrypted after a restore. Keep /etc/gitlab/gitlab.rb, /etc/gitlab/gitlab-secrets.json, the SSH host keys and your TLS material in a separate, access-controlled store, and confirm the exact file list against the GitLab backup and restore documentation for your install method.

What does a failed secrets restore look like in practice?

Usually it does not look like a clean failure. The restore completes, the web interface loads, and the problems appear later. Common symptoms include users unable to log in with two-factor authentication, CI/CD jobs failing because protected variables cannot be read, runners or integrations rejecting stored tokens, and errors when GitLab tries to decrypt stored credentials. Because these failures surface hours or days after the restore, the only reliable check is a restore test that includes logging in as a user with two-factor enabled and running a pipeline that uses a protected variable. Treat any decryption error in the logs as a failed restore, not a cosmetic warning.

Can our own team back up and restore secrets and configuration?

Yes, if your engineers can administer the Linux hosts and already run GitLab. The copy itself is simple: a scheduled, encrypted copy of a small set of files to storage that is separate from the primary host. The effort lies in the parts around it: deciding where copies live, restricting who can read them, rotating them when secrets change, and proving a restore works. Many teams manage the copy but never test a restore on a separate host. Ask whether your staff have the time and skills to run that test on a schedule, because an untested copy gives little assurance during an incident.

What do we need to provide if we bring in a partner for this work?

A partner will need administrative access to the GitLab hosts and the configuration directory, a place to store the copies that you control, and a test host where a restore can be run without touching production. You also need to agree on change windows, because validating a restore usually means a short planned outage on the test system or a maintenance window on production if configuration changes are made. Name the people who approve secret rotation and who can read the backup store. Written access rules and a named contact on your side are the items that most often slow engagements down.

How does the effort change with the install method and number of nodes?

The main driver is how many hosts hold secrets and configuration. A single Linux package instance has one gitlab.rb and one secrets file. A multi-node deployment, such as those described in GitLab's reference architectures, needs the same secrets on every node that runs GitLab services, and consistent configuration across them. Helm chart and Operator installs keep secrets in Kubernetes objects and configuration in values files, so the backup scope shifts to cluster state and version control. Installs from Docker or source each need their own mapping of files. The more nodes and install variants, the more places must be captured, compared and restored, which increases both the initial setup and the testing workload.

How often should we refresh the secrets and configuration copies?

Refresh the copy whenever the source changes, and at least on a fixed schedule. Secrets change far less often than repository data, but gitlab.rb changes with upgrades, new integrations and hardening work, and TLS certificates renew on their own cycle. Create a new copy after every upgrade, after rotating secrets and after any change to SSH host keys or certificates. Keep older copies for a defined period so you can recover from a bad change. Pair these refreshes with a restore test, since a current copy that cannot be restored is not useful. Confirm the retention period with your security and compliance owners before setting it.

What should we ask a partner to include in a fixed-scope quote?

Ask for a written list of what is in scope: which hosts, which files, which install method, and which storage location. Ask how the copies are encrypted, who can read them and how access is logged. Ask whether the quote includes a restore test on a separate host, how the test is documented, and what evidence you receive. Ask how changes to gitlab.rb or rotated secrets are captured after go-live, and whether that is a one-time task or a recurring service. Finally, ask which items are your responsibility, such as storage capacity, access approvals and change windows. Most fixed-scope engagements are agreed in writing before work starts, so these points belong in the scope document.

Where to read the official backup and restore guidance

Before you change anything on a production instance, read GitLab's backup and restore documentation from start to finish, because it sets the rules this topic depends on. The page covers what a GitLab backup archive contains, which files sit outside it, and why the gitlab-secrets.json file and the gitlab.rb configuration file must be copied separately. Use that page as the reference for your own runbook, and check the version of the documentation against the GitLab version you actually run, since commands and file locations can change between releases. Ask your vendor or partner to confirm which steps apply to your package type, and write down the exact commands your team will use.

The second read is the maintenance policy, which decides the version your backup and your restore target must share. GitLab ships a major release each year, a minor release on the third Thursday of each month and patch releases twice a month, and it maintains a short list of recent stable lines. Restoring onto a different GitLab version or edition than the one that produced the backup is a common cause of failed recovery, so note which versions are currently maintained and plan your upgrades around the required upgrade stops. Keep the policy and your restore runbook in the same change-controlled location so that the version recorded with each secrets and configuration backup can be matched to the right package during an incident.

How DeployOpen scopes and supports GitLab backup planning

DeployOpen approaches backing up GitLab secrets and configuration as part of the wider recovery plan for your self-managed instance, inside your own infrastructure. We agree the scope in writing first, then build and document the backup and restore steps so your team can run them and prove they work.

How we scope

Inventory the installation method, GitLab version, and every host holding gitlab-secrets.json, gitlab.rb and SSH host keys.
Agree in writing which secrets, TLS material and configuration files the backup must cover, and where each copy is stored.
Confirm with you who owns key custody, access approval and retention before any work starts, since engagements are fixed-scope.

How we build

Set up a repeatable, access-controlled process that captures the secrets file and configuration on a schedule and after every change.
Document how the secrets file, gitlab.rb and TLS material are restored onto a matching GitLab version, in the order a rebuild needs.
Run a test restore on a separate host to confirm that projects, CI variables and runners still work afterward.

How we hand over

Deliver a runbook covering backup location, encryption, rotation steps and the exact checks that confirm a backup is usable.
Walk your operations and security staff through a restore drill so the procedure is tested by the people who will run it.
Provide an evidence pack listing the controls, backup records and test results that your internal or external auditors may request.

How we support

Offer ongoing support, upgrades and recurring restore testing as a retainer after handover, if your team wants it.
Review the backup procedure again before each GitLab upgrade, since upgrade stops can change configuration keys and file layout.
Keep the runbook current as your topology, storage or identity setup changes, and record each change in the document history.