Why a production Argo CD install needs the HA manifests
The HA manifests are the upstream starting point for running Argo CD components with redundancy, and they are not a complete production design. When a platform team relies on Argo CD to decide what runs in production clusters, a single pod or node failure can stop deliveries, so the question is whether the installation survives routine failures without someone intervening. The HA manifests change the default layout to run multiple replicas of the components, but the surrounding decisions about node capacity, storage, network paths and cluster access are still yours to make and document.
Argo CD is largely stateless. Its data lives in Kubernetes objects such as Applications, AppProjects and repository Secrets, and Redis is a disposable cache that Argo CD can rebuild. That shifts recovery work away from restoring a database and toward the design of the cluster that hosts Argo CD and the Git repositories it reads. Before you plan backups, confirm what the control plane of that cluster can restore, how quickly you can recreate Argo CD from Git and exports, and who holds the access needed to do it.
Two gating questions should be settled before anyone copies the manifests. First, the HA manifests need at least three nodes because of pod anti-affinity rules, so confirm that your target cluster has three schedulable nodes that can be spread across failure domains, and that your capacity plan allows for losing one. Second, the HA documentation states that IPv6-only clusters are not supported, so verify the address families of the cluster before you start. Answering these early avoids discovering a blocked rollout during a change window, and the Argo CD high availability documentation is the source to check against your own cluster.
What the HA manifests change from the default install
The HA manifests change how many replicas run for each Argo CD component, where those pods land, and how Redis fails over. Read each card as a question to answer for your cluster: what gets redundant, what still has a single point of failure, and what you must set yourself. Exact replica counts and Sentinel settings belong to the high availability page for the release you deploy.
API server replicas
The API server serves the web UI, the CLI and the gRPC and REST calls that automation makes. The HA manifests run more than one API server replica behind a Service, so a pod restart does not take the UI or CLI offline. What stays the same is that login, sessions and the admin account still depend on the configuration in the argocd-cm and argocd-secret objects, which every replica shares. Confirm the replica count and any login rate limit settings on the high availability page for your release before you size ingress and TLS termination.
Repo-server replicas and Git access
The repo-server clones Git repositories and runs Helm and Kustomize to render manifests, so it is the component most exposed to slow or failing renders. The HA manifests run multiple repo-server replicas, which spreads that work and keeps rendering available if one pod is busy or restarting. What does not change is that every replica needs network access to your Git hosts and Helm registries, and rendering still runs tools on repository content. Check the replica count and the tool timeout settings for your release.
Application controller and sharding
The application controller reconciles live cluster state against Git and is the component that runs out of capacity first as clusters grow. The HA manifests can run it with more than one replica, and the high availability documentation describes sharding the controller so each replica owns a subset of clusters. Replicas do not simply add throughput for one cluster: each managed cluster is handled by one shard. Confirm the sharding mode your release supports and how a cluster is assigned before you rely on replicas for capacity.
Redis as a disposable cache
Redis holds cached application state and does not hold the source of truth, which lives in Git and in Kubernetes objects. The HA manifests move Redis to a highly available layout with Sentinels so a primary failure triggers a failover instead of a cache outage. Losing the cache forces a rebuild and slower first syncs, not data loss. Confirm the Sentinel quorum, the number of Redis replicas and the Service name your applications use against the high availability page for your release.
Dex or the external OIDC setup
Login depends on either the bundled Dex or a direct OIDC connection to Okta, Keycloak, Microsoft or another provider. The HA manifests make the Argo CD side of the login path redundant, but Dex and your identity provider are separate systems with their own availability. If you run bundled Dex, check its replica settings in the HA manifests. If you connect directly to an external provider, the identity provider's uptime becomes the login ceiling for every user, so plan a break-glass path before you disable the admin account.
Pod anti-affinity and node count
The HA manifests apply pod anti-affinity rules so replicas of the same component do not share a node. The high availability documentation states that these rules need at least three nodes, so a two-node cluster cannot schedule the HA layout as written. Spreading pods helps only if your nodes sit in separate failure domains. Confirm the topology labels your cluster uses, and check that your cluster does not run IPv6-only, which the same documentation says is not supported.
Choosing between the default install, the HA manifests and a sharded controller
This table compares three Argo CD layouts: the default manifests, the HA manifests on one cluster, and HA combined with application controller sharding. Read each row as a decision: find the layout that matches your fleet size and availability needs, then check what it requires from the cluster and what it costs to run.
| Layout | When it fits | What it requires | Main trade-off |
|---|---|---|---|
| Default manifests | Evaluation, a lab, or a small non-critical deployment where a short outage is acceptable | A single cluster with standard capacity, and no need to survive a node loss | Components run as single replicas, so a node failure stops deployments until the pods reschedule |
| HA manifests | One production cluster that must keep syncing through a node loss | At least three nodes, since the HA manifests use pod anti-affinity rules, and an IPv6-only cluster is not supported | More pods and Redis Sentinels to run, with extra node capacity and upgrade steps to plan |
| HA plus controller sharding | Large fleets with many applications or clusters that load one application controller heavily | HA layout plus a plan to assign clusters to controller shards and to watch their balance | Highest operational complexity, since shard assignment and balance must be monitored and adjusted over time |
| Who decides | Platform lead with the application and cluster inventory | Node count, cluster inventory and an upgrade window agreed up front | Confirm the replica and sharding settings for your Argo CD version in the official HA documentation before you size nodes |
Why three nodes are a hard requirement for HA Argo CD
The HA manifests need at least three nodes because they use pod anti-affinity rules to spread each replicated component across different nodes. The Argo CD high availability documentation states this requirement directly. Replicas of the API server, repo-server, application controller and the Redis HA components are meant to land on separate nodes, so that losing one node does not take out every copy of a component. With only two nodes, the scheduler cannot place all the required replicas under those rules, so some pods stay in Pending and the installation looks healthy only until a node fails. With a single node, the anti-affinity rules cannot be satisfied at all, and the cluster offers no node-level redundancy for Argo CD.
The failure is easy to miss in managed clusters. Node pools with autoscaling may start small, and a pool that scales to zero or to two nodes during quiet periods will leave HA replicas unschedulable exactly when a rollout or a node replacement needs them. Taints on dedicated node pools can also block Argo CD pods unless matching tolerations are set, and a pool limited to a single availability zone defeats the purpose of spreading replicas. Resource requests can be a hidden blocker too: if the nodes are already reserved by other workloads, the replicas will be Pending even when the node count looks correct. Before installing, confirm that the target cluster has at least three schedulable nodes in the pools Argo CD will use, that those nodes carry no taints the manifests lack tolerations for, and that the spread across failure domains matches what your availability requirements need.
Treat the node check as part of the acceptance test, not as a one-time setup step. After installation, run kubectl get pods -n argocd -o wide and confirm that replicas of each component sit on different nodes and that none remain Pending. Then drain one node in a non-production cluster and confirm that the UI, CLI and syncs keep working while the replacements schedule. Ask your platform team or managed-service provider to confirm the node pool minimums, autoscaler settings and any admission policies that could block scheduling, since those settings change outside the Argo CD manifests.
Installing the HA manifests in a production cluster
This sequence takes a cluster from a node check through a verified HA Argo CD install, with pod placement, Redis and Sentinel health and SSO confirmed before the built-in admin user is turned off.
Confirm nodes and cluster networking
The platform team confirms that the target cluster has at least three schedulable nodes, because the HA manifests use pod anti-affinity rules that need three nodes to place replicas. Also confirm the cluster is not IPv6-only, since the HA documentation says IPv6-only clusters are not supported. Run
kubectl get nodes -o wideand check that three or more nodes show Ready, that none carry a taint that blocks the Argo CD workloads, and that the node addresses include IPv4. The step is done when the list shows at least three Ready nodes that can take the workloads and you have written down the cluster name and node count for the change record.Select the Argo CD release and check support
Choose the minor release you will install and confirm it is within the window the project supports with patch releases. The release process page says only the three most recent minor versions receive patch releases and that a minor release ships every three months. Compare your chosen version with the latest patch releases on the Argo CD GitHub releases page, and pin the exact patch version in your change record. Ask your vendor or DeployOpen engineer to confirm the upgrade path if you are replacing an older installation. The step is done when the target version is named, dated and marked as inside the patch support window.
Apply the HA manifests for that release
Apply the high availability manifests that match the pinned release, using the paths listed in the official HA documentation rather than the default install manifests. Keep the manifests in Git so the exact source is recorded and can be reapplied. Namespace, CRDs and the HA components (application controller, API server, repo-server, Redis HA and Sentinels) should be created in one pass. Check the apply output for errors and confirm the Argo CD custom resource definitions are present with
kubectl get crd | grep argoproj.io. The step is done when the apply completes without errors and the release version reported by the API server matches the pinned version.Verify pod placement across nodes
Confirm that each replicated component runs on different nodes so one node failure does not take out a whole tier. List the pods with
kubectl get pods -n argocd -o wideand check the NODE column for the application controller, API server, repo-server, Redis and Sentinel pods. Each component should have its replicas spread across at least three nodes, with no two replicas of the same component sharing a node. If a replica is pending, describe the pod and read the scheduling events, which usually point to anti-affinity or resource limits. The step is done when every replicated component shows Running pods on distinct nodes.Confirm Redis and Sentinel health
Check that the Redis HA layout has a primary and replicas, and that the Sentinel pods agree on which Redis instance is primary. List pods with
kubectl get pods -n argocdfiltered to the Redis and Sentinel workloads, then check their logs or the Sentinel status for the current primary and quorum. Argo CD treats Redis as a disposable cache, so a failover should not lose application state, but you still need to see that failover completes. Ask your team to confirm how the Sentinel status is read in your release, because the output format depends on the version. The step is done when the Sentinels report a single primary, the replicas are attached, and the application controller can reach Redis.Log in and configure SSO
Reach the Argo CD UI or CLI through its ingress, log in with the initial admin account, and configure SSO through the bundled Dex connectors or a direct OIDC provider such as Okta, Auth0, Microsoft, Keycloak or Google Workspace. The user management documentation describes both options. Map identity provider groups to Argo CD roles, then sign in with a named user from each group and confirm the expected permissions. Store the initial admin password in your secrets process and record who has access to it. The step is done when at least one SSO user has logged in and the role assignment behaves as designed.
Disable the built-in admin user
Once SSO or local users work, set admin.enabled to false in the argocd-cm ConfigMap, as the user management documentation recommends. Apply the change through your Git-managed configuration, then confirm the change took effect by attempting an admin login, which should be refused, and by checking that SSO users still sign in and can perform their assigned tasks. Remove the initial admin secret only after you have confirmed a recovery path, such as a documented break-glass process. The step is done when the admin login is refused, SSO users still have access and the change is recorded in version control.
Adjusting the HA install for your environment
On a managed Kubernetes service, the main risk is that autoscaling node pools shrink below the number of nodes the anti-affinity rules need. The HA manifests require at least three nodes, so the node pool that hosts Argo CD must keep a minimum size that the autoscaler cannot go under, and it should span failure domains where the provider offers them.
- Node pool minimums: Set the minimum node count of the pool that runs Argo CD to at least three, and confirm the autoscaler respects that floor during scale-down.
- Taints and tolerations: If you reserve a pool for platform workloads, add a toleration to the Argo CD pods and confirm the pool carries the matching taint, or the pods will stay pending.
- Zone spread: Ask whether the provider places nodes across zones in a region and whether the pool can be pinned to those zones, so a single zone outage does not remove a majority of replicas.
- Storage classes: Argo CD keeps its state in Kubernetes objects, so most managed installs need no persistent volume changes. Confirm this against your own Redis and add-on choices.
- Upgrades: Ask the provider how node upgrades drain nodes and whether a pod disruption budget is honored during the drain.
Before go-live, confirm the node pool minimums, zone behavior and drain rules with your cloud provider's documentation and your support contact.
On bare metal or virtual machines, nothing schedules nodes for you, so the three-node requirement becomes a capacity and placement decision made by your team. Two physical hosts can run three virtual machines, but then a single host failure takes out more than one node, which defeats the purpose of anti-affinity.
- Node count and placement: Provision at least three nodes and confirm they do not share a single hypervisor host, power feed or top-of-rack switch unless you accept that failure domain.
- Taints and tolerations: If the cluster hosts other workloads, dedicate or label nodes for platform services and add matching tolerations and node selectors to the Argo CD workloads.
- Storage classes: Confirm which storage class your Redis and any supporting volumes use, and whether it replicates data across hosts. Argo CD itself is largely stateless, so storage mainly matters for add-ons and backups.
- Hypervisor HA: If the nodes are virtual machines, ask whether the hypervisor's own high availability will move a node that fails, and confirm that this does not restart all three nodes together.
- Load balancing: Plan how the API server and UI are reached through a load balancer or ingress that is itself highly available.
Confirm the placement of your nodes against your hardware inventory and the vendor documentation for your hypervisor and storage platform.
In segmented or air-gapped networks, the HA install itself does not change, but the images and manifests must come from inside the boundary. Every container image the HA manifests reference, including Argo CD, Redis and any Dex or proxy images, has to be mirrored to a registry the cluster can reach, and the manifests must be updated to point at that registry.
- Image mirroring: Mirror every image version you deploy into an internal registry, and check each manifest for image references that point outside your network.
- Manifest review: Store the mirrored manifests in an internal Git repository, and record the image digests you approved so that later upgrades pull the same artifacts.
- Egress and DNS: Confirm which internal endpoints Argo CD needs for Git, the identity provider and clusters it manages, and open only those paths.
- Certificates: Use an internal certificate authority for TLS between components and for the external endpoint, and confirm that every client trusts it.
- Upgrade channel: Decide how patch releases move across the boundary, since the project publishes patches on its releases page and your team will need a controlled way to import them.
Confirm the image list, registry trust and upgrade path with your security team and against the official high availability and installation documentation before the first install.
Security controls that stay in place after the HA install
Moving to the HA manifests adds replicas and endpoints, but it does not change how Argo CD authenticates, encrypts or handles credentials. The cards below cover the controls the security documentation describes and the review steps that keep them intact after the install.
JWT-only API authentication
Argo CD authenticates API requests only with JSON Web Tokens, so every UI, CLI and automation call depends on how those tokens are issued, expired and revoked. Confirm which identity path issues tokens for your users and pipelines, and check that the admin login is turned off once SSO works. With several API server replicas, verify that every replica validates tokens the same way behind your load balancer.
TLS between components
Argo CD uses TLS for traffic, including traffic between its own components. The HA layout adds more connections, such as API server to repo-server, controller to Redis and Sentinel traffic, so each path should be checked rather than assumed. Confirm certificate issuance, rotation and trust bundles for every internal connection in your environment, and test that a replica restart does not break a connection.
Credential redaction in responses and logs
Cluster, Git and OAuth2 credentials are redacted from API responses and logs. Redaction protects the common leak paths, but it does not cover values you paste into annotations, custom plugins or notification templates. Review log shipping and the access your team has to Argo CD logs, and search a test deployment's logs for known test secrets after each upgrade.
Repo-server exposure to untrusted content
The repo-server runs Helm and Kustomize on repository content, so a malicious chart, Kustomization or out-of-tree file reference can reach the cluster through manifest generation. Limit which repositories are registered, restrict the sources each project may use, and review which files and paths a Kustomize or Helm build can read. Scale of HA replicas does not reduce this risk, so treat the repo-server as the highest-review component.
Admin user and HA endpoint exposure
The built-in admin user should be used only for initial setup, then disabled with admin.enabled set to false once local users or SSO are configured. Each HA replica also adds an endpoint, so the ingress, load balancer and network policies that front the API server need the same review as the single-replica install. Confirm which namespaces and networks can reach each service, and test that access rules still hold after a scale change.
Questions teams ask before running HA Argo CD in production
How much effort does an HA Argo CD installation take?
The effort depends less on the manifests than on what surrounds them. Applying the HA manifests is quick once the cluster is ready, but the work that takes time is confirming the cluster has at least three schedulable nodes for the pod anti-affinity rules, deciding how SSO and RBAC will map to your groups, setting up network access and TLS for the API server, and testing failover of the components and Redis. Ask any partner to split its quote into these parts so you can see which ones your team already owns.
Can a platform team install HA Argo CD without outside help?
Yes, many platform teams do, provided they have Kubernetes operations experience and time to test. The manifests are documented in the Argo CD high availability guide, and the main risks are configuration gaps: IPv6-only clusters are not supported, Redis HA with Sentinels needs careful testing, and identity settings from the upgrade notes change between versions. Outside help is most useful for the design decisions, a rehearsed failover, and a written runbook your on-call staff can follow.
What do we need to provide before the work starts?
Expect to provide access that is scoped to the work, not broad administrator rights. Typical inputs include:
- Kubernetes cluster access and node capacity, with confirmation of the three-node minimum
- Git hosting details, repository layout and credentials for the Git server
- Identity provider details, such as an OIDC client or the group claims you want mapped to Argo CD roles
- Network rules, DNS names, certificates and any proxy requirements
- Change windows for the install, cutover and failover tests
Confirm with your team how secrets will be handed over before sharing any of these.
How long does an HA Argo CD project take?
Duration follows the written scope. A single cluster with the HA manifests, one identity provider and a basic failover test is a much smaller piece of work than a project that adds several clusters, Dex connectors, repository credentials migrated into Kubernetes Secrets, and a restore rehearsal. Your change control process can also add time, because approvals and change windows are often the longest waits. Ask for the timeline as milestones tied to deliverables so you can track progress.
What recurs after an HA Argo CD goes live?
Most of the cost after go-live is recurring operational work. The project supports only the three most recent minor releases with patch releases, and a new minor ships roughly every three months, so someone must plan regular upgrades and patching. Redis is a disposable cache, but its health, controller health and sync backlog still need monitoring and alert thresholds. Budget for on-call coverage, reviewing release notes before each upgrade, and periodic failover tests.
How does fixed-scope work differ from a retainer?
Fixed-scope work covers an agreed set of deliverables, such as the HA installation, SSO configuration and a failover test, with scope and timeline agreed in writing before work begins. A retainer covers ongoing work over time, such as upgrades on the supported minor releases, patch review, monitoring and responses to incidents. Many teams start with a fixed-scope build and then move to a retainer for day-two operations. DeployOpen does not publish a price list, and a quote follows from the written scope for either option.
What does the cost of an HA install depend on?
The main cost drivers are the number of clusters, the identity model, the state of your Git and network setup, and the amount of testing you require. Each cluster that needs its own controller shards, every additional identity connector, and each recovery test adds work. Costs are one-time for the build and recurring for upgrades, patching and monitoring. When you request a quote, ask for each line item to name who does the work, what it delivers, and which parts recur.
Which Argo CD settings do we need to verify after installation?
Check that the admin account is disabled with admin.enabled set to false once local users or SSO work, that the Redis HA components are running on separate nodes, and that the application controller replicas are healthy. Confirm that repositories are defined as Kubernetes Secrets, since Argo CD 3.0 removed them from argocd-cm. Review RBAC against the federated_claims.user_id claim if you use Dex. Record each check in your runbook so an auditor or on-call engineer can repeat it.
Limits to confirm before you commit to the HA layout
The HA manifests carry constraints that shape where and how Argo CD can run. Each card names one limit to verify in your own environment before you size hardware, write the deployment plan or start an upgrade.
IPv6-only clusters are unsupported
The HA documentation states that IPv6-only clusters are not supported. If your clusters, nodes or service networks are IPv6-only or depend on IPv6 for pod addressing, the HA layout is not a fit as documented. Confirm the address families on every cluster that will host Argo CD and the clusters it deploys to, and raise any gap with your platform team before design starts.
Three schedulable nodes minimum
The HA manifests use pod anti-affinity rules, which need at least three nodes to schedule. Confirm that the target cluster has three schedulable worker nodes after taints, node pools and resource reservations are applied. Also confirm that a single node failure leaves enough capacity for all Argo CD replicas, because a two-node cluster will leave pods pending.
Redis Sentinel and API server replicas
Redis HA with Sentinels and the API server replica settings change between releases. Do not copy replica counts or Sentinel settings from an older guide. Read the high availability page for the exact release you plan to deploy, render the manifests, and compare the Redis and API server objects against what your change record approves.
Repo-server sizing depends on your manifests
The repo-server runs Helm and Kustomize to render manifests, so it is usually the most resource-hungry component. Its CPU and memory needs depend on chart size, the number of applications and how often Git changes. Measure render time and memory on a representative set of applications in a non-production cluster before you fix the replica count and resource limits.
Patch support covers three minor versions
Only the three most recent minor versions receive patch releases. Confirm which minor version your HA install targets and when the next minor release is due, and plan a regular patch window. A deployment pinned to an older minor will stop receiving security fixes, so the HA setup needs a standing upgrade schedule owned by a named team.
RBAC and repository changes in 3.0
The 2.14 to 3.0 upgrade notes change how update and delete permissions apply, require an explicit logs permission for log access, and move repository definitions from argocd-cm into Kubernetes Secrets. Dex RBAC also uses the federated_claims.user_id claim. Existing HA installs must review roles, repository objects and SSO mappings before the upgrade, or access and syncs may change unexpectedly.
How DeployOpen scopes and delivers a production Argo CD install
DeployOpen plans and builds the HA installation inside your own clusters and cloud accounts, starting from a written scope. The work covers the HA topology, identity, access, backup and handover, so your team can run the installation after the engagement ends.
How we scope
- Engagements are fixed-scope, with the HA topology, clusters and components agreed in writing before work starts.
- We confirm your cluster count, node capacity and whether the three-node minimum for anti-affinity is met.
- We confirm your IPv6 requirements early, since IPv6-only clusters are not supported by the HA manifests.
- The written scope names SSO, admin account handling, repository credentials and who owns each decision.
How we build
- We deploy the HA manifests with replicas, anti-affinity and Redis HA with Sentinels set to match your scope.
- We configure identity through Dex or a direct OIDC provider and map groups to Argo CD roles.
- We disable the built-in admin user once local users or SSO are working, and test that access still holds.
- We test failover of API server, repo-server, controller and Redis pods, and record the results.
How we hand over
- You receive the configuration as version-controlled manifests, with the decisions and their reasons documented.
- We document the upgrade path for the current minor release line, including the 3.0 changes your team must check.
- We walk your operators through health checks, backup and export steps, and failover behavior in your environment.
- Runbooks cover what to do when a component is unavailable, so your team does not depend on us to recover.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped separately from the build.
- Retainers can cover patch releases for the three most recent minor versions, as set by the project's release process.
- Scaling work such as controller sharding or Redis tuning can be scoped as a follow-on engagement.
- Ask us to confirm which support items, response arrangements and recurring costs apply before you sign.
