Ceph RGW object storage for self-managed GitLab

Connecting self-managed GitLab to Ceph RGW uses GitLab's S3-compatible object storage settings. This page covers the AWS connection type, the consolidated form, HTTPS and what community-documented status means for support.

On this page

Ceph RGW object storage for self-managed GitLab: how the connection works

A team moving GitLab artifacts, LFS objects, uploads and packages off local disk onto Ceph RGW should know that the supported route is GitLab's object storage configuration using the S3-compatible API. Ceph RGW is listed in GitLab's documentation as a community-documented provider, which means GitLab does not test it. The connection is therefore your responsibility to validate, and any upgrade of GitLab or Ceph should be treated as a change to re-test.

GitLab stores several data classes in object storage rather than on the local filesystem. These include CI job artifacts and job logs, Git LFS objects, user uploads such as attachments and avatars, and package registry content. Each class has its own bucket or configuration key in GitLab, so you should inventory which classes are enabled and where each one lands before you cut over. GitLab's object storage documentation recommends object storage over NFS for these classes, because NFS places a shared filesystem dependency on every application node and makes scaling and failover harder to plan. Confirm the current list of supported classes in the GitLab object storage documentation for your GitLab version, since the set changes over time.

Before you configure anything, decide whether the RGW endpoint will be reached over HTTPS, since GitLab says server-side encryption with Ceph RGW requires an HTTPS connection. Ask your Ceph and GitLab administrators to confirm certificate trust on every GitLab node that talks to the endpoint, and to test a sample upload and download from each application and Sidekiq node. Also confirm which RGW bucket and access key your team will use per data class, because a single shared credential makes later rotation and access review harder. Treat the S3 compatibility label as a starting point only, since GitLab warns that claiming S3 compatibility does not guarantee a provider works correctly with GitLab.

What you need before connecting GitLab to Ceph RGW

Connecting GitLab to Ceph RGW depends on four things being ready before any configuration: a reachable object gateway, a bucket layout, a dedicated RGW user, and a TLS path that GitLab hosts trust. The GitLab installation method and version set the rest of the sequence, so confirm them first.

Ceph cluster with RGW reachable

Confirm the Ceph cluster is running an actively maintained release, currently Tentacle (20.2.x) or Squid (19.2.x) according to the Ceph releases page, and that the Ceph Object Gateway answers S3 requests from every GitLab node, including Sidekiq and Gitaly hosts. Check network paths, firewall rules and any load balancer in front of RGW, since GitLab will fail silently on intermittent connectivity.

Bucket layout per object type

Plan one bucket per object type GitLab stores, such as artifacts, LFS objects, uploads, packages and merge request diffs, or document why you are sharing buckets. Separate buckets make lifecycle rules, access policies and restore operations easier to reason about. Decide bucket names before configuration, because renaming them later means migrating objects.

RGW user and access keys

Create a dedicated RGW user for GitLab rather than reusing an administrator account. Scope it to the buckets in the layout above, store the access key and secret in your secrets manager, and plan rotation. Confirm with your Ceph administrator which RGW authentication and policy features apply to that user in your release.

TLS certificate trusted by GitLab

Server-side encryption with Ceph RGW requires an HTTPS connection, so the RGW endpoint needs a certificate that every GitLab host trusts, including the chain and any internal certificate authority. Test trust from each GitLab node with the same hostname GitLab will use, because a name mismatch will fail after configuration rather than during it.

GitLab installation method

Record which installation method you run: Linux package, Helm chart, GitLab Operator, Docker or source. The Linux package bundles PostgreSQL, Redis and Sidekiq, while the Helm chart and Operator change where object storage settings live. Confirm the exact configuration path for your method in the GitLab object storage documentation before editing anything.

GitLab version and upgrade stops

Confirm your current GitLab version against the maintenance policy and the required upgrade stops, which fall at versions x.2, x.5, x.8 and x.11 since GitLab 17.5. Object storage changes are easier to validate on a version you will keep. If an upgrade is pending, schedule it before the connection work so that you test the object storage settings on the target release.

How to configure GitLab object storage on Ceph RGW

This sequence takes a self-managed GitLab instance from a bare Ceph RGW endpoint to verified storage for artifacts, LFS, uploads and packages, with each object type tested before the configuration is considered done.

  1. Confirm the RGW endpoint and certificate

    The Ceph administrator confirms that the Ceph Object Gateway answers S3 requests on the endpoint GitLab will use, and that the endpoint presents a certificate trusted by the GitLab host. Ask the storage team to confirm the Ceph release in use and whether the endpoint is reached through a load balancer, since both affect the endpoint and addressing settings you will set later. The step is done when a plain HTTPS request to the endpoint returns an S3 response from the gateway rather than a connection or trust error.

  2. Create RGW users, buckets and access keys

    The storage administrator creates one RGW user per GitLab integration boundary, generates its S3 access key and secret, and creates a bucket for each object type GitLab will store, such as artifacts, LFS, uploads, packages and, where used, others your GitLab documentation lists. Keep the credentials out of shared notes and record which user owns which bucket. The step is done when the bucket list for that user shows every bucket and a test object written with the new key can be read back.

  3. Choose the AWS connection type and endpoint settings

    In GitLab, the platform team sets the object storage connection to the AWS connection type with the RGW endpoint, the region value your Ceph setup expects, and path-style addressing enabled, because RGW buckets are typically addressed by path rather than by subdomain. Confirm these choices against the GitLab object storage documentation for your version, since the exact keys differ between the separate and consolidated forms. The step is done when the settings file validates and no value is left at a provider default you have not checked.

  4. Decide between separate and consolidated form

    The platform team chooses the form before writing the final configuration. GitLab recommends the consolidated form, which enables direct upload automatically, but it cannot be used for backups or Mattermost, so a deployment that needs those features may keep separate settings for them. Record the decision, the buckets each form covers and the reason, so the choice survives later upgrades. The step is done when every object type maps to exactly one form and the mapping is written down for the operations team.

  5. Enable HTTPS for server-side encryption

    If server-side encryption is required, the team switches the endpoint to HTTPS, because GitLab documents that server-side encryption with Ceph RGW requires an HTTPS connection. Ask the security team which encryption requirement applies and whether the key management is handled by RGW or by an external service, since that determines what the reviewer will check. The step is done when a test upload with encryption requested succeeds over HTTPS and the encryption setting is visible on the stored object.

  6. Apply the configuration and reconfigure GitLab

    The GitLab administrator applies the object storage settings to the instance, using the configuration file method that matches the installation, such as the Linux package or the Helm chart, and runs the reconfigure or restart command that method requires. Keep a copy of the previous configuration so the change can be reverted. The step is done when GitLab starts cleanly, the admin area shows the object storage settings as active, and no errors about the storage connection appear in the logs.

  7. Verify uploads and downloads for each object type

    The GitLab administrator tests each object type separately: a CI artifact from a pipeline job, a Git LFS file pushed from a client, an avatar or attachment upload, and a package published to the package registry. For each one, confirm the upload completes, the object appears in the matching RGW bucket, and the file downloads correctly back through GitLab. Also run a GitLab health check and review the application logs for storage errors. The step is done when all four object types pass upload and download, and the bucket listing matches what GitLab reports.

Why the consolidated form matters and where it stops

GitLab recommends the consolidated form of object storage configuration, which uses one object_store block per storage type with a single connection, and it enables direct upload automatically. Direct upload lets clients send files straight to the object store instead of routing them through GitLab, which reduces load on the application servers for artifacts, LFS, uploads and packages. For a Ceph RGW deployment, the practical step is to confirm that the RGW endpoint, the access keys and the bucket names for each object type are set in one connection definition, and that the connection type is the AWS one described earlier in this page. Confirm the exact keys and the current behavior against the GitLab object storage documentation for your installed version before you change production settings.

The limit is the part teams miss. The consolidated form cannot be used for backups or for Mattermost, so a team that plans to send GitLab backups to object storage needs a separate configuration for that purpose, and the backup target is usually a different bucket, and often a different credential, from the artifact and package buckets. Keep the backup bucket outside the store that holds live data, so that a misconfigured lifecycle rule or an accidental deletion does not remove both copies. If you also run Mattermost on the same Ceph cluster, plan its bucket and its own configuration in the same review. Ask your GitLab administrator to list every object type and every connection in one table before go-live, and check each row against the backup and restore procedure in the GitLab backup and restore documentation.

Because Ceph RGW is community-documented rather than tested by GitLab, every connection in that table should also be verified in your own environment. Run an upload, a download and a deletion for each object type, and run a test restore of a backup written to its separate target. Record the RGW version, the GitLab version and the connection settings used, since a later GitLab upgrade or RGW change can alter behavior. Treat this inventory as a standing control, not a one-time setup step.

Which GitLab data goes to which Ceph RGW bucket

This table maps each GitLab object type to a suggested bucket, access scope and retention owner on Ceph RGW. It also shows whether the consolidated form can hold that type, so you can see where separate connection settings are needed. Rows marked for confirmation depend on your GitLab version and should be checked against the object storage documentation before you build.

Object typeSuggested bucket and access scopeConsolidated formRetention owner and what to confirm
CI job artifactsDedicated artifacts bucket, read and write only for the GitLab service userYesGitLab administrators own expiry settings. Confirm artifact expiry and lifecycle behavior for your version.
Git LFS objectsDedicated LFS bucket, same service user, no direct end-user credentialsYesProject owners and administrators decide retention. Confirm how deleted LFS objects are purged in your version.
Uploads (attachments and avatars)Dedicated uploads bucket, service user onlyYesGitLab administrators. Confirm how orphaned uploads are cleaned up before relying on bucket lifecycle rules.
Packages (container, npm, Maven and similar)Dedicated packages bucket, service user onlyYesPackage registry owners. Confirm which package types your version stores in object storage.
Backups and Mattermost dataSeparate backup bucket, restricted to backup operators and kept apart from runtime bucketsNoBackup owner. The consolidated form cannot hold these, so configure separate connection settings and confirm with the GitLab backup documentation.
Server-side encryption settingsApplies to the buckets above, requires an HTTPS endpointNot applicableSecurity team. Confirm the HTTPS endpoint and certificate chain with the Ceph RGW and GitLab documentation.
Bucket naming and per-type separationOne bucket per object type keeps access and retention independentNot applicableStorage team. Agree naming and ownership in writing before the first upload so that migrations stay simple.

Security settings for GitLab and RGW object storage

Object storage moves GitLab artifacts, uploads and packages off the local disk and onto a network path that carries credentials and data. The cards below cover the transport, identity, secret and trust decisions that connect GitLab to Ceph RGW, from the endpoint certificate to the cluster's own traffic.

HTTPS for server-side encryption

GitLab's object storage documentation says server-side encryption with Ceph RGW requires an HTTPS connection. Plan the RGW endpoint with a valid certificate before enabling encryption in GitLab, and confirm that the endpoint URL in the connection settings uses the https scheme. A plain HTTP endpoint may appear to work for uploads while encryption options fail, so test each object type after the change.

Least-privilege RGW users per bucket

Create a separate RGW user for each GitLab bucket or purpose, such as artifacts, LFS, uploads and packages, rather than sharing one administrative account. Grant that user access only to its own bucket. A narrow identity limits the blast radius if one set of keys leaks and makes it clear which GitLab function produced a given access in the RGW logs.

Protecting access keys in GitLab

The S3 access key and secret key sit in GitLab's object storage configuration. Store them in the managed secret mechanism your GitLab deployment uses, restrict who can read the configuration files on each node, and avoid copying keys into tickets, chat or shell history. Plan a rotation procedure that updates every GitLab node and then retires the old key on the RGW side.

Certificate trust on every GitLab node

Every node that reads or writes objects must trust the CA that signed the RGW endpoint certificate, including Sidekiq workers, Rails application nodes and any backup host. A node missing the CA will fail in a way that looks like a storage outage. Confirm trust on each node individually, and add the CA to the update process so that a renewed or rotated certificate is not a surprise.

Messenger v2 secure mode in the cluster

Ceph's messenger v2 protocol uses port 3300 and offers a secure mode that encrypts all post-authentication traffic with AES-GCM and adds integrity checks. Enabling it protects traffic between the Ceph daemons and clients, which is separate from the HTTPS listener GitLab uses. Confirm that every client path your cluster serves supports the mode before you require it, and test a GitLab upload after the change.

Next steps: hardening guides

Use GitLab's hardening recommendations to review the GitLab application, host and access settings, and Ceph's hardening guidance for client authentication, encryption in transit and cephadm host access. These guides cover much more than the object storage path, so treat them as a checklist to work through with your security team. DeployOpen can review these settings as part of a planned deployment, and the written scope should name which controls are in it.

What community-documented status means for support

GitLab lists Ceph RGW under its community-documented object storage providers. That means GitLab does not test this provider in its own test pipeline, so the behavior you see with Ceph RGW is not something GitLab has verified on your behalf. GitLab also warns that claiming S3 compatibility does not guarantee a provider works correctly with GitLab. S3 compatibility covers the API surface a gateway implements, but GitLab exercises specific calls, multipart behavior, signing and encryption paths, and those can differ in edge cases. Plan your build around that gap.

  • Connection type: Use the AWS connection type or the consolidated form as GitLab documents, and confirm the exact options against the current object storage page before you write configuration.
  • Encryption: GitLab says server-side encryption with Ceph RGW requires an HTTPS connection, so plan certificates and trust chains for the RGW endpoint first.
  • Feature coverage: Test artifacts, LFS, uploads and packages separately, because each object type uses its own bucket and access pattern.
  • Upgrade interaction: Retest object storage after each GitLab upgrade stop and after each Ceph RGW upgrade, since neither vendor's test suite covers the other's release.
  • Known gaps: Record any feature you cannot use, such as a consolidated form limitation for backups or Mattermost, so it does not surprise operators later.

Before you rely on this design, confirm each combination of GitLab and Ceph versions you intend to run against the current GitLab object storage documentation and your own test results.

Common GitLab and Ceph RGW failures and their fixes

This table pairs common failures when GitLab writes to Ceph RGW with the likely cause and the check to run first. Read each row as a triage starting point, and confirm the setting names and behavior against the Ceph release and GitLab version you run, since both change over time.

FailureLikely causeCheck to run
TLS errors on the HTTPS endpointRGW presents a certificate GitLab does not trust, often an internal CA or a self-signed certificate missing from the GitLab host trust storeConfirm the certificate chain with a TLS client from the GitLab host, then verify the CA bundle GitLab uses for its object storage connection
Server-side encryption rejectedServer-side encryption is enabled while the connection is plain HTTP, which GitLab says requires HTTPS for Ceph RGWConfirm the endpoint scheme is https and that encryption is enabled only after the HTTPS connection works
Requests fail with path-style addressing errorsVirtual-hosted bucket addressing is used, but the RGW endpoint or DNS does not resolve bucket subdomainsSet path-style addressing for the connection and confirm the setting name for your GitLab version in the object storage documentation
Signature mismatch or wrong regionRegion or signature version does not match what RGW expects, or the clock on the GitLab host is skewedCheck the region value GitLab sends, confirm the signature version your Ceph release accepts, and compare host time with NTP
Access denied on uploads or downloadsThe RGW user lacks bucket permissions, the bucket policy denies the action, or the keys belong to a different userTest the same keys with an S3 client against the bucket, then review user caps and any bucket policy for the actions GitLab needs
Backups or Mattermost uploads fail under consolidated formThe consolidated form cannot be used for backups or Mattermost, so separate per-type connections are requiredReview the object storage settings for each data type and confirm which connection each one uses before the next backup run
Uploads fail after a GitLab upgradeA changed default, a deprecated connection key, or a Ceph release change in signing or API behaviorCompare the GitLab upgrade notes and object storage settings before and after, then test a small upload and download in staging
Bucket or lifecycle objects missing after moveData was written to a bucket or prefix that the new configuration no longer reads, or a bucket name changedList objects per bucket with an S3 client and reconcile bucket names against the GitLab configuration before deleting anything

Questions buyers ask before connecting GitLab to Ceph RGW

What drives the effort of connecting GitLab to Ceph RGW?

The effort depends mostly on how many GitLab object types you move, how many buckets you plan, and how cleanly your Ceph RGW endpoint already works with S3 clients. Moving only CI artifacts is a small change. Moving artifacts, LFS, uploads, packages and Terraform state, with HTTPS, a private certificate chain and a tested restore, is a larger job. Also count the time needed to size and secure the object store itself, since GitLab's storage settings are the last step, not the first. Ask any partner to split the quote into object store preparation, GitLab configuration, and verification.

Can our team connect GitLab to Ceph RGW without a partner?

Yes, if you have a Ceph administrator who can create buckets and access keys, a GitLab administrator who can edit the object storage settings, and someone who can issue and trust the HTTPS certificate. GitLab's object storage documentation covers the connection types and the settings involved. The harder part is testing: you need to confirm uploads, artifact downloads, backups and failover behave correctly against your own RGW version. A partner adds value when the team lacks one of those three skills or wants the test evidence documented for an audit.

What does the work need from our Ceph and GitLab teams?

You need four things from your side. A Ceph administrator must provide RGW endpoints, bucket names and dedicated access keys, and confirm the RGW version. A GitLab administrator must decide the connection type and whether to use the consolidated form. Your certificate owner must provide an HTTPS endpoint with a chain that every GitLab node and runner trusts, since GitLab requires HTTPS for server-side encryption with Ceph RGW. Finally, someone must plan buckets for each object type, retention and backup scope, so that the GitLab backup and restore process is designed rather than assumed.

Is Ceph RGW supported by GitLab?

GitLab lists Ceph RGW among community-documented object storage providers, which means GitLab does not test it. That is different from a vendor-supported integration. GitLab also warns that claiming S3 compatibility does not guarantee that a provider works correctly with GitLab. For your decision, this means the support boundary must be written down: GitLab does not test the RGW version you run, and the test evidence for your version comes from your own environment or a partner's test plan. Confirm the status with GitLab for your deployment and tier before you rely on it in production.

How do GitLab and Ceph RGW upgrades affect this configuration?

Two upgrades change the picture: GitLab upgrades and Ceph upgrades. GitLab must pass through required upgrade stops, and bug fixes are backported only to the current stable release, so the object storage settings must be retested after each planned step. Ceph has its own release lifecycle, and RGW behavior can change between releases, so a Ceph upgrade is a separate test event. Plan a staging environment that matches production, re-run uploads, artifact downloads, and a backup restore after each upgrade, and record the RGW and GitLab versions together.

What does a fixed-scope engagement cover for this integration?

A fixed-scope engagement sets out the agreed scope, deliverables and timeline in writing before work starts. For this integration, the scope usually names the GitLab object types in use, the RGW endpoints and buckets, the connection type and certificate approach, the test cases, and the documentation handed over. It can also include a restore test and an upgrade runbook. Ongoing support, upgrades and continuous testing are available as retainers. DeployOpen does not publish a price list, so ask for a quote based on the written scope. Note that DeployOpen prepares systems and evidence for compliance assessments but does not issue audit reports or certifications.

Does GitLab support plan tiers change what we get for object storage?

Object storage configuration is available on self-managed installations regardless of tier, since GitLab Free, Premium and Ultimate are all available for self-managed use. The tiers differ in the features built on top. GitLab's pricing page lists advanced CI/CD and priority support for Premium, and application security testing, software supply chain security, vulnerability management and compliance features for Ultimate. If your team relies on priority support for a storage incident, check the tier you hold before you assume that coverage applies. Premium is listed at 29 US dollars per user per month billed annually, and Ultimate is custom priced.

How to test GitLab on Ceph RGW before go-live

Before go-live, run a test plan that exercises every object type GitLab writes to the Ceph RGW buckets, not just a single upload. Work through each category in a staging instance that uses the same endpoint, TLS certificate, bucket names and credentials as production: uploads and downloads of CI job artifacts, LFS objects, user uploads, and packages (for example, publishing and pulling a generic or container package). For each type, confirm the object lands in the expected bucket, can be downloaded through the GitLab UI and API, and survives a re-download after the RGW gateway is restarted. Also push and pull a large Git LFS file from a developer workstation and from a runner, since those paths use different client behavior. Because GitLab lists Ceph RGW as a community-documented provider that GitLab does not test, treat these results as your own evidence and record the GitLab version, Ceph release and object store settings used.

Test failover of the RGW endpoint as a separate step. If you place a load balancer or multiple gateways in front of RGW, stop one gateway during a CI pipeline and a package publish, then confirm that GitLab retries cleanly and no artifact is recorded as complete when its object is missing. Run a restore drill if you use GitLab backups: follow the GitLab backup and restore guidance for your installation method, restore into a separate environment, and verify that object storage content is reachable from the restored instance. Be aware that the consolidated form cannot be used for backups, so confirm which configuration your backup path requires before you run the drill. Finally, rehearse an upgrade against a copy of production, because required upgrade stops apply to every upgrade plan, and confirm that object storage settings still work after each stop.

How DeployOpen scopes and delivers this integration

Each GitLab and Ceph RGW integration starts with a written, fixed scope that names the GitLab object types, buckets, endpoints and checks to be delivered. The cards below cover what that agreement includes, where the work happens, and how support continues after go-live.

Fixed scope agreed in writing

The written scope names which GitLab object types move to RGW (artifacts, LFS, uploads, packages, and any others you need), the buckets and users for each, and the GitLab and Ceph versions in play. Deliverables and timelines are agreed before work starts. Ask for the scope to list what is excluded, such as backup or Mattermost storage, which the consolidated form cannot cover.

GitLab and Ceph configuration

The work covers the object_storage settings in GitLab, including the AWS connection type or consolidated form, endpoint and path-style addressing, and bucket mapping, along with the RGW users, bucket policies and keys on the Ceph side. HTTPS is configured on the gateway so server-side encryption works. Confirm which GitLab version and connection form your instance will use before the scope is signed.

Testing against your instance

Testing checks each object type end to end: CI artifact upload and download, LFS push and pull, avatar and attachment uploads, package publishing, and object deletion by GitLab's cleanup jobs. Because GitLab does not test Ceph RGW, these checks are the evidence your team relies on. Ask for the test cases and their pass results in the deliverables.

Documentation for operators

The handover covers the configuration as deployed, the bucket and user inventory, the HTTPS certificate chain, the steps for rotating keys, and the known gaps for backups. Your team keeps a record of which GitLab release and RGW version were tested together, which matters because GitLab requires upgrade stops and the RGW version may change independently.

Infrastructure stays with you

All work runs inside your own infrastructure, including your Ceph cluster, GitLab instances, network and certificate authority. DeployOpen does not host your data or hold your storage keys beyond the engagement. DeployOpen prepares configurations, tests and evidence for your assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those.

Retainers after go-live

Ongoing support, GitLab upgrades that re-test the object storage path, and Ceph patch reviews can run under a retainer agreed separately from the project scope. Before signing, confirm what the retainer covers, how upgrade testing is scheduled around GitLab's monthly releases, and who handles incidents involving storage. Quotes follow from the written scope, and DeployOpen does not publish a price list.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.