When teams need the Argo CD Source Hydrator
A platform team that wants reviewers to see the exact manifests reaching a cluster before merge has a different problem from the usual GitOps setup. In the default flow, Argo CD renders Helm or Kustomize at sync time, so the Git diff shows chart inputs and values but not the Kubernetes objects those inputs produce. The Source Hydrator closes that gap by rendering Helm, Kustomize, plain directories or plugins ahead of time and committing the output to Git, so Argo CD syncs files that a reviewer can read in a pull request. Teams that need that diff, or that must prove which rendered manifest was approved, are the typical reason to consider it.
The Source Hydrator is a Beta feature since v3.5.0, and it is disabled by default. Enabling it means running the commit server and setting hydrator.enabled in argocd-cmd-params-cm. Because it is Beta, plan your rollout around that status: confirm the current behavior against the Source Hydrator guide for the exact Argo CD version you run, and check the release notes before each minor upgrade. Ask whether your organization accepts a Beta component in change-controlled production paths, and who signs off on that choice.
Rendering at deploy time remains the simpler option when reviewers are satisfied with reviewing chart values and when no separate rendered history is needed. The hydrator adds a second Git history of generated output, a second set of credentials and a commit step that must succeed before every sync. Compare those costs against your review requirements and your operating model before committing to the pattern. If you proceed, your partner or internal team should agree on the operating limits early, since the guide documents that hydration runs only on new dry-source commits and that stale generated files are not cleaned up automatically.
Which Source Hydrator options fit which setup
This table compares the main ways to run Argo CD with or without the Source Hydrator, from direct sync to a staging branch that gates promotion through a pull request. Read each row for the setup it fits, what it adds to Git history and review, and how much credential scope it demands. The main takeaway is that hydration buys reviewable rendered output at the cost of a second branch, a second credential, and a commit server you must run and monitor.
| Option | When it fits | Cost in Git history and review | Credential scope |
|---|---|---|---|
| Direct sync, no hydration | Small estates where Helm or Kustomize output is stable and reviewers read chart values, not rendered manifests | No generated commits. Reviewers see only source changes, so rendered drift is hard to inspect before it reaches the cluster | One read credential for the source repository |
| Hydrator enabled with the commit server, pushing to the same branch | Teams that want rendered manifests committed to Git but accept direct promotion from the hydrated output | Generated commits land on the branch Argo CD syncs from, so every dry-source commit produces a visible rendered commit with no separate review step | A write credential for the commit server plus a read credential for Argo CD |
| Hydrator with separate repository-write and repository secrets | Setups where the component that writes rendered output should not hold the same access as the component that reads it | Same Git history as above. Credential separation adds rotation work because two secrets must be maintained and labeled correctly | Write credential limited to the commit server, read credential limited to Argo CD sync |
| Hydrator with hydrateTo and a staging branch | Regulated change flows where a pull request must approve rendered output before promotion to the live branch | Adds a staging branch and a pull request per promotion. Argo CD does not open that pull request, so your pipeline or team must create and merge it | Write access for the staging branch, plus read access for the live branch sync; review permissions sit in your Git host |
| Hydrator with a staging branch and retained generated files | Teams that need to know exactly what changed in rendered manifests between releases | Stale generated files are not cleaned up, so reviewers must check deletions in each hydrated commit manually | Same as staging branch setup; no extra credential needed |
| Hydrator with secret-injecting tools | Generally not a fit. The guide advises against SOPS and similar tools during hydration | Secrets can end up in rendered commits, which then persist in Git history | Broadens exposure of secret material; keep secrets out of the hydration path |
How to enable the Source Hydrator step by step
This sequence turns on the Source Hydrator in a running Argo CD instance, connects it to Git with separate write and read credentials, and produces a verified hydrated commit that Argo CD syncs from a non-root path.
Confirm the Source Hydrator prerequisites
The platform team checks that the Argo CD instance runs a version that includes the Source Hydrator, which the feature documentation describes as Beta since v3.5.0, and that the team accepts a Beta feature for this workload. Confirm the Git host, the branch naming and the repository layout with the application owners before changing anything. Run
argocd versionagainst the cluster and record the server version. You are done when the version is recorded, the Beta status is approved in writing, and the target repository and branch names are agreed.Enable the commit server and hydrator flag
An administrator with access to the Argo CD namespace enables the commit server component and sets hydrator.enabled in the argocd-cmd-params-cm ConfigMap, which is disabled by default. Restart the affected Argo CD components so they read the new parameter. Check the change by confirming the commit server pod is running and that the parameter appears in the live ConfigMap with
kubectl get configmap argocd-cmd-params-cm -n argocd -o yaml. You are done when the commit server pod is ready and the flag reads true.Create the repository-write secret
Create a Kubernetes Secret in the Argo CD namespace that holds the credentials Argo CD uses to push hydrated output, and label it repository-write so Argo CD treats it as push-only. Use a machine account or GitHub App scoped to the rendered-manifests repository, and grant write access to the staging branch only where your Git host allows it. Store the token in the Secret, not in any Git file. Check it by listing secrets with the repository-write label and confirming the Secret exists, then reviewing its repository URL in the Argo CD settings view.
Create the separate pull credential secret
Create a second Secret labeled repository for the credentials Argo CD uses to read the hydrated commits and the dry source. Keep it separate from the write Secret so the read path has no push rights, which limits what a compromised sync path can change. Use a read-only token where the Git host supports one. Check the separation by confirming two distinct Secrets exist, one with each label, and that the read token fails when used to push to the test repository.
Configure hydrateTo and the staging branch
In the Application spec, add the sourceHydrator block so that the source reads the dry manifests and hydrateTo names a staging branch where Argo CD writes the rendered output. Keep the live sync branch separate from the staging branch. Argo CD does not open the promotion pull request, so your team must create it through the Git host or automation. Check the configuration by applying the manifest and confirming the Application shows the hydrator source, and that the staging branch name matches what the pull request process expects.
Point syncSource.path at a non-root directory
Set syncSource.path in the Application to a subdirectory of the hydrated output, not the repository root, because the feature requires a non-root path. Choose a path per environment, for example a folder named for the environment, so each Application reads only its own rendered files. Check the choice by listing the staging branch contents after a hydration run and confirming the target path contains the expected manifests and nothing from other environments.
Run a test commit and verify hydration
Make a small, reviewable change to the dry source, such as a single value in a Helm values file, and push it to the source branch. Argo CD hydrates only on new commits to the dry source, so an unchanged repository produces no new output. Watch the Application until a new hydrated commit appears on the staging branch, then open the commit and confirm the rendered manifests show your change. You are done when the commit is visible, the rendered file matches the source change, and the sync status reflects the new revision.
Record the hydrated commit trail
Record the dry source commit, the hydrated commit and the pull request that promotes it, so an auditor or on-call engineer can trace a deployed revision back to its source. Note that hydrated commits are not signed, so the trail depends on your Git host audit log and the linked pull request rather than commit signatures. Confirm the record by picking one deployed revision and tracing it to its dry commit and approving pull request. Also note that stale generated files are not removed automatically, so plan a cleanup step in the promotion process.
Security controls for hydration credentials and rendered output
The Source Hydrator adds a write path to Git, so credentials, the repo-server and the branches that feed syncs all become part of your trust boundary. The cards below cover the controls to set before enabling hydration, and each one points to the Argo CD security documentation or the Source Hydrator guide for the detailed configuration.
Split push and pull credentials
Hydration needs two identities: a write credential that the commit server uses to push rendered output, and a read credential that Argo CD uses to sync from the hydrated path. Register the write credential as a repository secret labeled repository-write and the read credential as one labeled repository, so each can be scoped and rotated separately. Check that the pull credential cannot push, and that the write credential is limited to the hydrated branches it needs. Argo CD redacts Git credentials from API responses and logs, but scope is still your control.
Treat repo-server rendering as code execution
The repo-server runs Helm and Kustomize against repository content, so a chart, Kustomize overlay or referenced file that you did not review can influence what gets rendered. Limit hydration to repositories your organization controls, and block out-of-tree file access in Kustomize and Helm inputs where your tooling allows it. Confirm which repositories the commit server can read before you enable it, since hydration reads from those sources on every dry-source commit.
Keep secret-injecting tools out of hydration
Tools such as SOPS decrypt values during rendering, which would write plaintext secrets into the hydrated commit and then into Git history. The Source Hydrator guide advises against using them during hydration. Keep secrets out of the rendered output entirely and have the destination cluster resolve them at deploy time through an operator, so the committed manifests contain only references.
Plan for unsigned hydrated commits
Commits created by the hydrator are not signed, so a commit in the hydrated history does not prove who authored the rendered output. Compensate with branch protection, required reviews on the staging branch, and a record of the dry-source commit each hydrated commit came from. Ask your Git host whether it can enforce signed commits on the hydrated branches, and document that the gate is the review, not the signature.
Restrict who can merge to staging
When hydrateTo pushes to a staging branch, a pull request becomes the promotion gate, and whoever can merge that pull request controls what reaches the sync branch. Limit merge rights on staging and hydrated branches to the release roles you name in your change process, and separate the people who author application source from those who approve rendered output where your controls require it. Argo CD does not create the pull request itself, so confirm the process and approvers in your Git host.
Tighten Argo CD access to hydrated paths
Argo CD's RBAC decides who can view and sync the applications that read hydrated output. Since the Argo CD 3.0 upgrade notes say update and delete rights now apply only to the application itself, and logs need an explicit permission, review which roles can change the source path or sync those applications. Disable the built-in admin user with admin.enabled set to false once SSO is configured, so hydration changes are traced to named people.
Operating hydrated repositories and commit tracing
Tracing a running revision back to its source is the first operational habit to build. Each hydrated commit in the environment repository is generated from a dry commit, so the dry commit SHA is the record that explains what the cluster is running and why. Before go-live, confirm with the hydrator's commit trail that every hydrated commit you can see in the Application's sync history points back to a dry commit in your source repository, and that your change-approval records reference the dry commit rather than only the generated one. Hydrated commits are not signed, so the link between the two commits is an operational convention your team has to enforce, not a cryptographic guarantee. Write a short runbook entry for this trace so an on-call engineer can follow it without knowing how the hydrator is wired.
The most common long-term problem is stale generated files. The hydrator does not clean up output that no longer comes from a template, so a deleted Helm template, a renamed Kustomize overlay or a retired environment can leave manifests in the hydrated path that still apply to the cluster. Plan a periodic review of generated paths against the current dry sources, and treat any file that no current source produces as a defect to remove through a reviewed change. The same review should check that syncSource.path still points to the intended non-root directory and that no secret-injecting tool has been added to the render step, since the hydration guide advises against that pattern.
Repository growth and promotion ownership need explicit decisions. Each hydrated commit adds history to the environment repository, and staging branches accumulate unless someone prunes them, so decide how long promotion branches and their pull requests are retained and how old hydrated history is archived. Argo CD does not create the promotion pull request for you, so name a single owner for the promotion pull requests who approves changes to each environment branch and monitors the queue. Confirm the expected commit volume with your own pipeline before you set retention, because the guide does not give a growth figure for your repositories.
Current limitations to confirm before adoption
The Source Hydrator is a Beta feature, so its behavior, defaults and limits can change between releases. The cards below cover the limitations that most affect how you design the commit flow, and each one lists what to confirm against your Argo CD version and the feature roadmap before you commit to the pattern.
Beta status since v3.5.0
The Source Hydrator has been a Beta feature since v3.5.0 and is disabled by default. Enabling it means the commit server and hydrator.enabled in argocd-cmd-params-cm are part of your platform, so a change to the feature could affect your pipelines. Confirm the exact Beta behavior for the minor version you run, check the release notes before each upgrade, and decide who owns re-testing when the feature changes.
Hydrated commits are unsigned
Commits that the hydrator writes to Git are not signed. If your change control relies on signed commits as evidence of who produced a rendered manifest, the hydrated branch will not provide that signal on its own. Decide whether the protected branch, the pull request review on the staging branch, or the dry-source commit signature is your record of approval, and confirm how your Git host enforces that control.
Stale generated files stay in place
Argo CD does not clean up stale generated files. When a chart or overlay stops producing a file, the old manifest can remain in the hydrated output and keep being synced. Plan a review step that compares the rendered tree against the previous commit, and test removal of a resource in a non-production environment before you rely on the pattern for pruning.
Hydration runs only on new dry-source commits
Hydration runs only when a new dry-source commit arrives. A change to something the renderer reads without a new commit to the dry source, such as an external value or a rebuilt dependency, will not regenerate the output. Confirm which inputs live in the dry source, and document how a team forces re-rendering when an input changes outside Git.
The syncSource.path must not be root
syncSource.path must be a non-root directory. A repository layout that places hydrated output at the top level will fail this rule, so the folder structure has to be settled before you wire up applications. Confirm the directory convention with your team, and check that the layout still works when you add environments or clusters later.
No pull request creation and no SOPS
Argo CD does not create the pull request that gates promotion from a staging branch, so your CI or Git host automation must open and merge it. Separately, secret-injecting tools such as SOPS should not be used during hydration, which means secrets need a different path, such as an operator on the destination cluster. Confirm both responsibilities with your delivery and security owners before design starts.
Questions buyers ask about the Source Hydrator
Can our internal platform team set up the Source Hydrator without a partner?
Yes, if the team already runs Argo CD in production and has someone comfortable with Helm or Kustomize, Git branch protection and Kubernetes secrets. The setup itself is configuration: enabling the commit server, setting hydrator.enabled in argocd-cmd-params-cm, and creating repository secrets with separate write and read credentials. The harder part is the repository design and the promotion process, which depend on your change control rules. Ask your team to name an owner for the hydrated repository and a test environment before they start, because the feature is Beta and you will be the first to find its edges.
What does the work need from our team?
Plan for a named Git administrator who can create branches and protection rules, someone who can create Kubernetes secrets in the Argo CD namespace, and an engineer who knows how each application renders today. You will also need a pull request reviewer for staging branches, since Argo CD does not open that pull request for you. Access to your Git host's audit log and a non-production cluster for testing round out the list. Confirm with your Argo CD version which hydrator settings are available before you commit to a design.
What drives the cost and duration of a Source Hydrator project?
The main drivers are the number of applications, how many repositories hold the source and the rendered output, and how many environments each application promotes through. A single application with one environment is a small job. A fleet with several clusters, staging branches and pull request gates is larger because every promotion path needs testing. Credential design, the cleanup of stale generated files, and the decision about whether to use secret tools during hydration also add effort. Ask any partner for a fixed-scope written plan that lists the applications, environments, branches and test cases included.
Is the Source Hydrator safe to use in production while it is Beta?
It can be used in production only with a clear fallback and tight scope. The feature has been Beta since v3.5.0 and is disabled by default. Start with a low-risk application, keep plain Helm or Kustomize syncs running for critical services, and document how to disable the hydrator if it misbehaves. Check the current Beta status and any changes in the Argo CD release notes before each upgrade, because Beta behavior can change between releases.
Why are hydrated commits unsigned, and what does that mean for our audit trail?
Hydrated commits are not signed, so the signature on your change record cannot prove who approved the rendered output. Your evidence should come from the dry source commit, the pull request that promotes the staging branch, and the commit trace that links the two. Decide early whether your change process needs signed commits from another control, such as branch protection that requires reviewed pull requests. Confirm this gap with your auditor before the design is final.
When is a plain Helm or Kustomize sync the better choice?
A plain sync is usually better when you do not need reviewable rendered manifests in Git, when your team wants Argo CD to render on every sync, or when the feature's limits would block you. Hydration only runs on new dry-source commits, does not clean up stale generated files, and rules out secret-injecting tools like SOPS during hydration. If your reviewers rely on the rendered diff, the hydrator is worth the extra repository. If they only review chart values, a plain sync with a good test pipeline is often simpler.
How do we troubleshoot a hydrated commit that looks wrong?
Start with the commit trace, which links each hydrated commit to its dry source commit, then re-render the same source locally with the same Helm or Kustomize version to compare output. Check whether a stale generated file is still in the path, because the hydrator does not remove files it no longer produces. Confirm syncSource.path is a non-root directory and that the commit was actually new in the dry source. Keep a runbook for this, since the failure often sits between the hydrator, Git permissions and the repo server rather than in one place.
How DeployOpen scopes and delivers Source Hydrator work
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, including Argo CD. For the Source Hydrator, we start from how your teams render manifests today, decide which branches and credentials the hydrator will use, and agree the work in writing before any change to your clusters or repositories.
How we scope
- Inventory which applications use Helm, Kustomize, plain directories or plugins and which need rendered output committed to Git
- Agree the hydrator scope in writing, covering enabled instances, repositories, destination branches and promotion steps
- Define separate push and pull credentials with repository-write and repository secrets, with ownership named for each
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts; ongoing support is available as a retainer
How we build
- Enable the commit server and hydrator.enabled in argocd-cmd-params-cm on a non-production instance first
- Configure hydrateTo staging branches and confirm that a pull request, created by your own process, gates promotion to the live branch
- Check that syncSource.path is a non-root directory and that generated files are handled, since stale output is not cleaned up automatically
- Test hydration on new dry-source commits and confirm that secret-injecting tools such as SOPS are kept out of the render step
How we hand over
- Document the commit tracing workflow so reviewers can link each hydrated commit back to its source commit
- Record the known limitations, including unsigned hydrated commits and the Beta status of the feature since v3.5.0
- Provide runbooks for credential rotation, failed hydration and branch promotion, reviewed with your platform and security owners
- Deliver written test results and configuration evidence that your own change management and auditors can read
How we support
- Review each Argo CD patch and minor upgrade against the Source Hydrator guide, since the feature is in Beta and may change
- Monitor hydration failures and stale branches as part of agreed operational checks, with alerting wired to your existing tools
- Revisit the rendered manifests pattern when your Helm or Kustomize versions, repositories or promotion rules change
- Continued support, upgrades and testing are available under a retainer agreed in writing
