What the Cloud Native Hybrid architecture on Kubernetes is and when to choose it
A platform team that wants GitLab on Kubernetes has often been told that stateful components cannot run in the cluster, and that is correct for GitLab. The GitLab reference architectures documentation states that running stateful components such as PostgreSQL and Redis inside Kubernetes is not supported. The Cloud Native Hybrid architecture resolves this by running the stateless application tier, Webservice and Sidekiq, in Kubernetes, while PostgreSQL, Redis, Gitaly and object storage stay on virtual machines or managed services outside the cluster. Before you commit, confirm the current support statement in the reference architecture documentation for your target release, because the split is a documented design decision and not a preference you can change per workload.
The split matters for operations. Webservice and Sidekiq are the components you scale out and replace frequently, so they benefit from Kubernetes scheduling, rolling replacement and horizontal scaling. Gitaly holds repository data on disk, PostgreSQL holds the metadata, and Redis holds cache and queue state, so their recovery, storage performance and backup procedures stay with the teams that already run those services. Plan the cluster's network path to each external service early, including DNS, TLS, firewall rules and the connection limits of the database tier, because those are the failure points that surface after go-live. Ask your GitLab contact and your Kubernetes provider which external endpoints and credentials the chart expects for each stateful role.
Choose this architecture over a pure virtual machine layout when your organization already operates Kubernetes with monitoring, GitOps and cluster upgrades, and when stateless scaling of web and background workers is a real operational need. Choose the all-in-cluster Helm chart only when you accept the vendor's current guidance on stateful components for your scale, which is a narrower path than the hybrid design. If your team does not yet run Kubernetes in production, a VM-based reference architecture is usually the simpler first step, and the cluster can be added later. Confirm the Kubernetes version and distribution support for your GitLab release before you size the cluster, since both change on their own schedules.
Which managed or VM service fits each stateful role
PostgreSQL holds GitLab's relational state for projects, merge requests, users, pipelines and permissions, so its availability and recovery settings define how much of the platform survives an outage. In the Cloud Native Hybrid layout it must run on VMs or a managed database service outside Kubernetes, because the reference architecture documentation does not support running PostgreSQL inside Kubernetes. The choice is between a self-managed VM cluster and a managed PostgreSQL offering from a cloud provider.
- Version and extension support: Confirm the PostgreSQL major versions listed for your GitLab release in the reference architecture and installation documentation, since managed services may default to a newer major version.
- Managed service features: Check whether the service supports the extensions and connection settings GitLab expects, and whether you can control maintenance windows for major upgrades.
- Connection pooling: Plan how connections are pooled between the Webservice and Sidekiq pods and the database, and confirm what the managed service provides.
- Recovery: Verify point-in-time recovery, backup retention and restore time in the service, and align them with GitLab backup procedures.
- Network placement: Keep the database private to the cluster network, with access limited to the GitLab components.
Confirm the exact PostgreSQL versions for your target GitLab release before you select a managed plan.
Redis holds GitLab's caching and queue-related data, and Sidekiq depends on it for background jobs. The reference architectures use Redis with Sentinel for high availability, and the Cloud Native Hybrid layout keeps Redis outside the cluster for the same reason as PostgreSQL. Teams often assume that any managed Redis-compatible service will fit, but GitLab's support statements limit which engines, cluster modes and serverless offerings you can use.
- Engine and version: Confirm which Redis or Valkey versions your GitLab release supports, as well as the compatible versions for the managed service you select.
- Cluster mode: Check whether the service offers the topology GitLab documents, since Redis Cluster and some serverless offerings are not supported for GitLab.
- High availability: Verify how failover works in the managed service and whether it matches the Sentinel-based design in the reference architectures.
- Eviction policy: Confirm that you can set the eviction policy GitLab recommends, because a wrong setting can cause data loss in queues.
- Network and authentication: Confirm TLS, password handling and private network access from the cluster.
Check these items against the Redis guidance for your GitLab version before you choose a managed offering.
Gitaly serves Git repository data over RPC, and it is the stateful role most sensitive to storage performance and layout. Repository storage cannot be placed in a managed database, so the choice is usually between VMs with attached block storage, a sharded Gitaly layout, or Gitaly Cluster with Praefect for repository-level replication. Each option changes backup, failover and sizing decisions, so the layout must be settled before the Kubernetes components are sized.
- Layout choice: Compare sharded Gitaly with Gitaly Cluster, since Gitaly Cluster has its own limitations and its status on Kubernetes differs from VM deployments.
- Storage performance: Confirm disk throughput and latency for your repository size and CI load, using the reference architecture for your user and request targets.
- Backups: Confirm how repository data is backed up, because some Gitaly Cluster configurations do not support snapshot backups.
- Resource sizing: Plan CPU and memory for Gitaly nodes separately from the Kubernetes worker pools.
- Network path: Confirm the latency and firewall rules between Webservice, Sidekiq and Gitaly.
Review the Gitaly topic guidance and your reference architecture before you select a storage class or VM image.
Object storage holds artifacts, LFS objects, uploads, packages, registry data and backups, and it is the stateful role most often moved to a cloud provider. In the Cloud Native Hybrid layout, GitLab reads and writes these objects through an S3-compatible or cloud API, so the storage choice depends on which GitLab object types must be supported and how the cloud provider's API behaves. Teams should confirm each object type rather than assume the whole platform works with one bucket.
- Object types: List artifacts, LFS, uploads, packages, registry and backup buckets, and confirm which ones your GitLab release supports on the chosen service.
- Credentials: Plan how the cluster authenticates to buckets, such as workload identity or stored keys, and where those secrets are kept.
- Lifecycle and retention: Set retention rules for artifacts and backups, and confirm that versioning and deletion behavior match your recovery plan.
- Encryption and access: Confirm encryption at rest, key ownership and bucket policies against your data classification.
- Regional placement: Keep buckets in the same region as the cluster to limit transfer cost and latency.
Confirm the supported object storage providers and configuration keys for your GitLab version in the official documentation before you commit to a provider.
Choosing between Cloud Native Hybrid, VM reference architectures and the full Helm chart
This table compares the three deployment shapes by who runs the stateful tier, who carries upgrade and backup work, and which team skills each one needs. Read each row as a question to answer for your own environment before choosing a shape.
| Decision area | Cloud Native Hybrid | VM reference architectures | Full Helm chart |
|---|---|---|---|
| Stateful tier | PostgreSQL, Redis, Gitaly and object storage run on VMs or managed services outside the cluster | PostgreSQL, Redis, Gitaly and object storage run on VMs you provision and operate | Stateful services run inside Kubernetes, which the reference architecture documentation does not support for PostgreSQL and Redis |
| Who owns stateful upgrades | Your database, cache and Gitaly teams, on their own schedule, alongside GitLab upgrade stops | Your platform team across all VMs, following GitLab required upgrade stops | Your cluster team, but the stateful components sit outside the documented support boundary |
| Backup responsibility | Split: GitLab backup tasks plus database and object storage backups owned by the teams that run them | Split across GitLab backup tasks and VM-level or database-level backups you define | Confirm with GitLab which backup procedures apply to in-cluster stateful components before relying on them |
| Kubernetes skills needed | Operating a cluster for stateless Webservice and Sidekiq workloads, plus etcd and control plane care | Standard Linux administration and configuration management, with no required cluster team | Deep Kubernetes operation, including stateful workloads, storage classes and persistent volume recovery |
| Managed services fit | Managed PostgreSQL, managed Redis-compatible caches and object storage can take the stateful roles if they meet GitLab's requirements | Managed services can be attached, but the VM layout remains yours | Limited, because the stateful roles are meant to run in the chart |
| Best fit | Teams with an existing Kubernetes platform who want stateless scaling and managed data services | Teams without a container platform who need a supported, well-documented layout | Evaluation or small environments where the team accepts the support boundary in writing |
| Confirm before choosing | Which managed database and cache versions GitLab supports, and who signs off on upgrade stops | Node counts for your reference architecture size and the zero-downtime upgrade procedure | Current support statement for stateful components in the chart with GitLab's documentation |
Build sequence for a Cloud Native Hybrid GitLab
This sequence takes a Cloud Native Hybrid GitLab from sizing to a tested restore, with each stateful service ready before Kubernetes depends on it. The output is a validated instance whose data, secrets and backups have been proven to work together.
Size the deployment against the reference architecture
The platform lead and GitLab administrators choose the closest reference architecture for the expected user count and API requests per second, then check whether CI concurrency, large monorepos or a Geo secondary raise the sizing. They record node counts for Webservice, Sidekiq and Gitaly, the capacity each stateful role needs, and the owner of each service. Done when the sizing sheet is approved and every stateful role has a named host or managed service, with sizing figures your vendor or managed provider has confirmed in writing. Start with the GitLab reference architectures documentation as the baseline.
Provision stateful services outside the cluster
The infrastructure team builds PostgreSQL, Redis, Gitaly and object storage on virtual machines or managed cloud services, because GitLab's reference architectures do not support running PostgreSQL or Redis inside Kubernetes. Confirm that each service version and feature set matches what GitLab documents for the release you will install. Enable automated backups and restrict network access to the cluster's subnets. Done when each service accepts connections from a test host on the private network, credentials are stored in your secrets manager, and backups have run at least once.
Prepare the Kubernetes control plane and nodes
The cluster team builds the control plane with at least three nodes behind a TCP load balancer on port 6443, or uses a managed control plane that meets the same availability need. Where etcd runs as an external cluster, use an odd number of members, and Kubernetes documentation recommends five members for production. Schedule etcd snapshots with
etcdctl snapshot saveand confirm the restore path withetcdutlbefore any workload is deployed. Done when all nodes report Ready, etcd reports healthy members, and a first snapshot exists in storage outside the cluster.Configure Secrets handling and access control
The security engineer enables encryption at rest for Kubernetes Secrets, because they are stored unencrypted in etcd by default. Restrict list and watch permissions with least-privilege RBAC, and either keep sensitive values in an external store reached through the Secrets Store CSI Driver or document why native Secrets are acceptable. Apply the Pod Security Standards to the GitLab namespace and connect human access through your identity provider. Done when a test service account cannot list Secrets outside its namespace, encryption is confirmed active, and the GitLab namespace enforces the chosen Pod Security level.
Install the chart with external connections
The platform engineer installs the GitLab Helm chart with
values.yamlpointing Webservice and Sidekiq at the external PostgreSQL, Redis, Gitaly and object storage endpoints, and disables any bundled stateful components. Confirm the chart version against GitLab's Cloud Native Hybrid guidance, and keep the kubelet version within the Kubernetes version skew policy relative to the API server. Done when all pods are running, the GitLab health endpoints respond, a user can sign in, and a test repository can be cloned, pushed and built by a runner.Validate backups and prove a test restore
The operations team runs a full GitLab backup, stores it away from the source environment, and restores it into a separate instance built from the same chart and values. They also restore an etcd snapshot into a test cluster and confirm that the GitLab secrets needed to decrypt restored data are recovered alongside the database and repositories. Compare restored projects, merge requests and pipeline definitions with the source. Done when the restored instance serves the same repositories, the signed-off recovery steps are written down, and the team has a dated record of the test.
How requests and jobs flow through the hybrid split
A Git request or CI job in this layout passes through the load balancer to Webservice pods in Kubernetes, and each step after that depends on a component that sits outside the cluster. A git push or a web page request lands on Webservice, which authenticates the user and checks permissions against PostgreSQL. Repository reads and writes are then forwarded to Gitaly, which holds the repository data on VMs or in its own storage, so the request depends on both PostgreSQL and Gitaly before it completes. If PostgreSQL becomes unreachable, authentication and metadata lookups fail for every user at once, while users who only need an already cached page may see partial results. If a single Gitaly node fails, only the repositories stored on that node are affected, which is why the storage layout you choose matters more than pod count.
Background work follows a different path. Webservice and Sidekiq both use Redis, which the Cloud Native Hybrid design keeps outside the cluster, to queue jobs and hold cache and session state. Sidekiq pods running in Kubernetes pull jobs from those queues, then read and write PostgreSQL and call Gitaly to do their work. A Redis outage stops new background jobs from being queued and can log users out or slow pages, but jobs already in progress may finish or retry depending on their design. A Sidekiq pod restart delays its jobs without losing the queue, so pipeline status updates, mirror syncs and notifications typically arrive late rather than disappearing. Confirm with GitLab's reference architecture guidance how your Redis deployment handles eviction and persistence before you rely on these recovery paths.
Plan your monitoring and runbooks around this map. For each external service, decide which user-facing symptom it produces, which alert identifies it, and who owns the response. Test the flow end to end after installation by pushing to a repository, running a pipeline and confirming that the job status updates, so you can see the dependency chain working before go-live. Refer to the GitLab reference architectures documentation for the supported topology and the services that must remain on VMs or managed platforms.
Security boundaries and Kubernetes hardening to plan
A Cloud Native Hybrid deployment places GitLab's stateless Webservice and Sidekiq pods in a cluster that holds credentials, tokens and configuration for the whole platform. The controls below decide how far a compromised pod or stolen credential can reach, so each one needs a named owner and a check before go-live.
Secrets in etcd and encryption at rest
Kubernetes stores Secrets unencrypted in etcd by default, so anything GitLab keeps there, including the secrets it generates during install, is readable by anyone with direct etcd access. Plan encryption at rest for the API server, restrict list and watch on Secret objects, and decide whether an external store fed through the Secrets Store CSI Driver holds the database and object storage credentials. Confirm who can read etcd snapshots, since backups carry the same data.
RBAC scope for GitLab namespaces
Give the GitLab release its own namespace and bind only the roles its service accounts need, rather than cluster-wide permissions. Operators who deploy and upgrade the chart need rights that differ from the pods that serve traffic, so split those identities. Review any role that can create pods, exec into containers or read Secrets, because those permissions reach the database and Redis credentials indirectly.
Pod Security Standards for Rails and Sidekiq
Apply a Pod Security Standard level to the GitLab namespace and set securityContext values on Webservice and Sidekiq pods, such as running as non-root and dropping privileges. Sidekiq processes handle job payloads from CI and repositories, so they carry the most untrusted input. Test the chosen level against the chart version you deploy, because some defaults may need an explicit exception that should be documented and reviewed.
Authentication to the cluster
Kubernetes does not manage ordinary user accounts. Decide whether administrators authenticate with OpenID Connect tokens from the corporate identity provider, client certificates or a proxy, and keep break-glass certificate access documented and rarely used. Deployment pipelines, including GitLab Runner jobs that deploy to the cluster, need scoped service accounts with short-lived credentials instead of long-lived kubeconfig files.
Network paths to stateful services
PostgreSQL, Redis, Gitaly and object storage sit outside the cluster, so every pod-to-VM or pod-to-managed-service path becomes a firewall and routing decision. Allow only the ports Webservice, Sidekiq and the toolbox need, restrict egress from the GitLab namespace, and confirm that managed database endpoints accept traffic only from the cluster's address range. Gitaly paths deserve particular attention because they carry repository data.
CIS benchmark and GitLab hardening guidance
Use the CIS Kubernetes Benchmark as the baseline for control plane, node and policy settings, and the GitLab hardening documentation for application, CI/CD and configuration file settings such as gitlab.rb. Map each recommendation to an owner and mark any that conflict with a chart default. GitLab describes its hardening guidance as a work in progress to be rolled out gradually, so record which items you adopted and when you reviewed them.
Questions buyers ask before choosing this architecture
What drives the cost of running GitLab Cloud Native Hybrid?
Cost comes from four places: the Kubernetes cluster that runs Webservice and Sidekiq, the VMs or managed services that hold PostgreSQL, Redis, Gitaly and object storage, the people who design and operate both layers, and the GitLab subscription if you buy Premium or Ultimate. The subscription is priced per user, and the pricing page lists Premium at 29 US dollars per user per month billed annually, with Ultimate custom priced. Infrastructure costs recur monthly and scale with user count, API load and CI volume. The one-time costs are design, build, load validation and migration. Ask any quote to separate infrastructure, licensing, and labor, and to state which stateful services are self-run and which are managed.
Who pays for what in a Cloud Native Hybrid build?
Your organization pays its cloud or data center provider for compute, storage, networking and the managed database or cache services it selects. It also pays for the GitLab subscription if it uses a paid tier, and for any managed Kubernetes control plane. The implementation partner is paid for the written scope: architecture decisions, installation and configuration, integration with identity and object storage, backup and restore testing, and handover documentation. Ongoing operation, upgrades and testing can be placed under a separate retainer. Make sure the quote states which items are handed over to your team and which remain with the partner.
How long does a Cloud Native Hybrid implementation take?
Duration depends on the number of users and the reference architecture size you target, how many environments you need, whether the cloud accounts and network paths already exist, and how much integration work sits on top, such as SSO, runners and object storage. The reference architectures range from 1,000 users at 20 requests per second to 50,000 users at 1,000 requests per second, so the bigger end of that range adds sizing and load testing time. Your partner should give a duration in the written scope, with the assumptions behind it. Ask which dependencies sit outside the project, such as approvals for new database services or firewall changes.
What does our team need to provide during the project?
You need named owners for the Kubernetes platform, the database and cache tier, identity, DNS and certificates, and the network. Your team should supply cloud or data center access with least-privilege roles, a decision on the object storage service, the identity provider configuration, and a maintenance window policy. Someone must approve the backup retention and restore test plan, and someone must own the GitLab Runner fleet once it exists. Missing any of these owners usually moves the schedule more than the technical work does.
Can an internal platform team build and run this alone?
A team with strong Kubernetes operations, PostgreSQL and Redis administration, Linux and network skills, and GitLab experience can build it alone, but the stateful tier is where most gaps appear. The architecture keeps PostgreSQL, Redis, Gitaly and object storage outside the cluster, so your team must operate them as well as the cluster. Check whether your staff can run a tested point-in-time recovery, a Redis failover, and a Gitaly restore without guidance. If any of these are new skills, plan a partner for the build and a knowledge transfer phase before go-live.
How do GitLab and Kubernetes upgrades get coordinated?
Treat them as two release trains. GitLab requires upgrades to pass through required upgrade stops, which since GitLab 17.5 fall at versions x.2, x.5, x.8 and x.11, and the maintenance policy ships a minor release on the third Thursday of each month. Kubernetes releases about three times a year, and the version skew policy requires the kube-apiserver to be upgraded one minor version at a time and allows kubelets to be up to three minor versions older. Plan a combined calendar where each GitLab stop is checked against the Kubernetes version currently supported by your distribution, and rehearse the upgrade in a non-production environment first.
When should we choose VMs instead of Cloud Native Hybrid?
Choose VMs when your organization does not already run Kubernetes in production, when your team lacks capacity to operate a cluster alongside stateful services, or when the Linux package install fits your operating model better. GitLab describes the Linux package as the most mature and scalable installation method, and the reference architectures for VMs are the standard path for many self-managed deployments. Cloud Native Hybrid suits teams that already operate Kubernetes well and want Webservice and Sidekiq to scale on it. Decide by comparing the staffing and upgrade load of each option, not by preference alone.
Does this architecture meet our compliance requirements?
The architecture does not meet or certify any requirement by itself. It gives you a layout that you can document, harden and test, including the encryption and access settings your auditors will ask about. Kubernetes stores Secrets unencrypted in etcd by default, so encryption at rest and least-privilege RBAC must be configured. GitLab's hardening documentation maps to NIST SP 800-53 and is described as a work in progress. Independent auditors and certification bodies issue any attestation, and you should confirm the required controls with them before the design is fixed.
Running upgrades and day-two operations without breaking the split
Build one upgrade calendar that covers GitLab, Kubernetes and the external stateful services together. In a Cloud Native Hybrid layout, the Webservice and Sidekiq pods are only part of the change. GitLab requires upgrades to pass through required upgrade stops, which since GitLab 17.5 fall at versions x.2, x.5, x.8 and x.11, so a jump from 18.2 to 18.5 is a planned stop, not a shortcut. Minor releases ship on the third Thursday of each month and patch releases twice a month, and only the current stable release receives ordinary bug fixes, while security fixes also go to the previous two monthly releases. Overlay the Kubernetes side: the kube-apiserver must move one minor version at a time and ahead of the kubelets, and kubelets may trail by up to three minor versions. Each cluster upgrade should be tied to a GitLab stop, and the chart version, Helm values and the PostgreSQL, Redis and Gitaly versions the chart expects should be checked against the same release notes before anyone changes a node pool.
Back up every stateful piece on its own schedule, and test the restore as one system. GitLab's backup task covers repositories, the database and uploads, but it does not capture the components that live outside the cluster. PostgreSQL needs its own point-in-time recovery, Redis data is usually treated as cache-like state that GitLab can rebuild, and the Gitaly repository storage needs a snapshot or file-level plan because the backup task alone is not a complete recovery design. Object storage holds artifacts, LFS objects, packages, registry data and uploads, so the bucket versioning, replication and lifecycle rules are part of the GitLab backup scope. Confirm with your platform team that the gitlab-secrets.json file and the Helm values that reference external endpoints are stored outside the cluster, because a restore without them is incomplete. Schedule a full restore into a separate namespace at least once per upgrade cycle and record how long it took.
Official GitLab and Kubernetes sources to read first
Hybrid designs depend on two vendor baselines: which components GitLab supports outside Kubernetes, and how Kubernetes versions, control planes and secrets behave over time. Read these sources before sizing nodes, planning upgrades or writing your backup runbook.
GitLab reference architectures
This page sets the rules for which components can run in Kubernetes and which must stay on VMs or managed services. It states that stateful components such as PostgreSQL and Redis are not supported inside Kubernetes under the Cloud Native Hybrid pattern. Check the user and requests-per-second target that matches your organization before you pick node counts for Webservice and Sidekiq.
GitLab upgrade paths
Hybrid deployments still have to pass through required upgrade stops before moving to a newer release. Since GitLab 17.5 these fall at versions x.2, x.5, x.8 and x.11. Map your current version against this page before you schedule a Helm chart or Linux package upgrade, and confirm the order in which your VM-hosted and cluster-hosted components must move.
GitLab backup and restore
In this architecture, backups cover PostgreSQL, Gitaly repositories, object storage and the secrets file, and each lives on a different system. Confirm who owns each backup job, where copies are stored, and how often a restore is rehearsed. The restore order matters, because repositories and database records must match after recovery.
Kubernetes version skew policy
Webservice and Sidekiq pods run on a cluster that you upgrade on Kubernetes' own schedule. This policy requires kube-apiserver to move one minor version at a time, before controller manager, scheduler and kubelets. Kubelets may be up to three minor versions older than kube-apiserver. Check these limits against the Kubernetes minor releases you plan to run.
kubeadm high availability guide
If your cluster is self-built with kubeadm, this guide sets the control plane baseline. It describes a stacked etcd topology or an external etcd cluster, with at least three control plane nodes and a TCP load balancer on port 6443 in front of kube-apiserver. Decide which topology your team can operate, then plan etcd snapshot backups and restores before go-live.
Kubernetes Secrets good practices
GitLab's connection secrets, registry credentials and database passwords often end up as Kubernetes Secrets. By default these are stored unencrypted in etcd, so confirm that encryption at rest is enabled. Also restrict list and watch access with least-privilege RBAC, and decide whether an external store such as the Secrets Store CSI Driver should hold them instead.
How DeployOpen works on this
DeployOpen scopes the Cloud Native Hybrid layout for your GitLab instance before any infrastructure is provisioned, then builds it inside your own cloud accounts, data centers or air-gapped network. Each stage ends with written deliverables your platform team can run without us.
How we scope
- Confirm which Webservice and Sidekiq pods run on Kubernetes and which stateful services stay on VMs or managed services.
- Map Gitaly, PostgreSQL, Redis and object storage to hosting choices you already operate or approve.
- Agree the written scope, deliverables and timeline in writing before any build work starts, since engagements are fixed-scope.
How we build
- Provision the cluster, ingress and network paths so the stateless pods can reach the external stateful services.
- Configure the GitLab Helm chart values, secrets handling and Kubernetes RBAC and Pod Security settings against your standards.
- Test backups and restores of the external data stores and run load validation against the reference architecture targets before go-live.
How we hand over
- Deliver architecture diagrams, configuration repositories and runbooks covering pod scaling, upgrades and etcd or database recovery.
- Walk your platform, database and security teams through the upgrade stops and version skew rules that apply to your cluster.
- Document the ownership boundary between cluster, VM and managed-service components so each team knows what it operates.
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after handover, scoped in writing.
- Plan Kubernetes and GitLab upgrade windows against the published maintenance and upgrade stop schedules together.
- Confirm the support scope, response expectations and reporting for your environment as part of the retainer quote.
