Deploying Apache Airflow with Argo CD or Flux: when GitOps fits Airflow
A platform team that wants Airflow chart versions, values files and secret references reviewed in Git and applied by a controller has a workable path, but GitOps adds moving parts that a plain helm upgrade does not have. Argo CD and Flux both reconcile a desired state from a Git source into a Kubernetes cluster. Argo CD runs as a Kubernetes controller with a web UI and manages an Application that points at a chart, a repository path or both, comparing live objects with what Git declares. Flux splits the same job into GitOps Toolkit controllers: a source controller fetches Git repositories and Helm repositories, and a helm controller installs and upgrades the release. Either tool can drive the Airflow Helm chart, but each reads its hook and rollout behavior differently, so the choice affects how you test changes before they reach production.
Airflow 3 makes the release more than a single pod. Since Airflow 3, which was first released on April 22, 2025 and has a current patch release of 3.3.2, the API server and the DAG processor run as separate components alongside the scheduler, workers and triggerer, so a chart upgrade changes several Deployments at once. The chart also runs database migration and user creation as jobs. Those jobs depend on Helm hook behavior that Argo CD and Flux do not reproduce in the same way, which is why the chart documentation asks for useHelmHooks: false and applyCustomEnv: false on createUserJob and migrateDatabaseJob when a controller other than Helm applies the release. Confirm that setting against your chart version before you rely on the pipeline.
A GitOps path is worth its setup cost when several people must review changes to values, when you need an audit trail of which chart and image version ran in each environment, or when you manage more than one Airflow instance and want the same promotion path for each. It is less clearly worth it for a single small instance that one team upgrades by hand. Plan for the secret boundary early: the production guidance asks for a static API secret key held in a Kubernetes Secret and different for each instance, so Git should hold references to secrets, never the values. Ask your team to test one promotion from a staging cluster before deciding, and to confirm how your chosen controller handles drift when someone edits a live object by hand.
How Argo CD and Flux differ for an Airflow release
Argo CD is an Apache-2.0 licensed project that reached CNCF Graduated status on December 6, 2022. It runs as a Kubernetes controller with a web UI and manages multiple clusters, so teams that already watch application state in a dashboard get a familiar view of each Airflow sync. Its identity model suits organizations that want operators working in a console and a clear record of who synced which change.
- Interface: The web UI shows sync and health state for the Airflow release, which helps operators who do not work primarily in the command line.
- Single sign-on: SSO runs over OIDC, OAuth2, LDAP and SAML 2.0, either through the bundled Dex or directly with an existing OIDC provider.
- Local admin: The built-in admin user is meant for initial setup. Once SSO or local users exist, the user management guidance is to disable it by setting admin.enabled to false.
- Upgrade behavior: The 2.14 to 3.0 upgrade notes narrowed update and delete RBAC to the application itself and made annotation-based resource tracking the default, so review existing project rules before upgrading.
- Patch support: Only the three most recent minor versions receive patch releases, so the platform team should confirm which minor line it will track for Airflow delivery.
Confirm which Argo CD minor version your platform team supports before you pin the Airflow application to it.
Flux is also Apache-2.0 licensed and reached CNCF Graduated status on November 30, 2022. It is built from GitOps Toolkit controllers: the source controller fetches Git repositories, Helm repositories and S3-compatible buckets, while the helm controller reconciles the Airflow release on the cluster. Teams that prefer Kubernetes objects and controllers over a separate UI will find Flux fits that model, provided they accept that visibility comes from the Kubernetes API and their own tooling.
- Controllers: The helm controller reconciles the Airflow chart and the source controller pulls it, so the whole release is described as Kubernetes objects in Git.
- Pull-based design: The cluster fetches changes from the source instead of a pipeline pushing them in, which keeps the deployment path under cluster control.
- Install paths: flux bootstrap is idempotent. The Flux Terraform provider and the Flux Operator are also available, and the installation guide covers air-gapped installs.
- Tenancy: Controllers use Kubernetes RBAC with impersonation, so each reconciliation runs with the permissions of a service account the platform team defines.
- Kubernetes versions: The installation guide sets minimum patch levels, such as 1.34.1 or later for Kubernetes 1.34, and advises against running end-of-life Kubernetes in production.
Check the Flux installation guide against your cluster version and air-gapped requirements before choosing an install path.
The Airflow Helm chart runs database migration and admin user creation as jobs that rely on Helm hooks by default. When Argo CD, Flux, Rancher or Terraform renders and applies the chart, useHelmHooks must be false and applyCustomEnv must be false on both createUserJob and migrateDatabaseJob, or the migrations will not run. Both controllers apply the chart the same way from this point of view, but how each one orders hook-annotated resources and handles sync waves should be confirmed in the controller version you run.
- Migration job: migrateDatabaseJob must complete before the new scheduler and API server pods use the metadata schema, so check the sync order the controller produces in a staging cluster.
- User creation: createUserJob needs the same two settings, and a fresh install can be left without an admin user if they are missing.
- Custom environment: With applyCustomEnv set to false on those jobs, confirm in the chart values that the metadata database connection still reaches them through another route.
- Upgrade order: The Airflow 3 upgrade guide asks you to be on Airflow 2.7 or later first, so do not let an automated sync perform the major upgrade without a tested migration run.
Render the chart for both controllers in a staging cluster and review the job resources and their ordering before you enable automated sync.
Argo CD controls access through its own RBAC over applications, projects and logs, and that model changed in the 2.14 to 3.0 line. Flux uses Kubernetes RBAC and impersonation, so access is defined in the same place as the rest of your cluster permissions. Both models need the same answers: who may change the Airflow release, who may read pod logs that can contain sensitive values, and which identity performs each sync.
- Argo CD application rights: Update and delete permissions now apply to the application itself, and reading logs needs an explicit logs, get permission.
- Dex claims: Argo CD 3.0 uses the federated_claims.user_id claim for Dex-based RBAC, so existing group mappings must be checked against that claim.
- Repository definitions: Argo CD 3.0 removed repository configuration from the argocd-cm ConfigMap, so the chart and DAG repositories must be defined as Kubernetes Secrets.
- Flux service accounts: Impersonation ties each reconciliation to a service account, so the platform team has to grant that account only the namespaces and resources the Airflow release needs.
- Secret placement: The static API secret key the chart expects should live in a Kubernetes Secret under either controller, never in a values file committed to Git.
List who holds sync, delete and log-read rights in each model before the first automated Airflow release.
Configuration options for the GitOps Airflow release
This table compares the chart and cluster settings you decide before the first sync, from migration hooks and secret handling to database placement and version pinning. Read each row with the tool you run, since the hook and secret settings differ in how they interact with Argo CD and Flux. The main takeaway is that the hook settings are not optional for GitOps delivery, while the other choices trade control against maintenance effort.
| Option | What it controls | When it fits | Trade-off |
|---|---|---|---|
| Hook settings on migrate and createUser jobs | Set useHelmHooks to false and applyCustomEnv to false on both jobs so Argo CD or Flux runs them as normal resources | Any GitOps release managed by Argo CD, Flux, Rancher or Terraform | Without these settings database migrations do not run, so the first sync can leave Airflow unusable until corrected |
| Static API secret key from a Kubernetes Secret | Supplies a fixed API secret key, unique per instance, outside the values file | Every production release, so API and log-serving components keep the same key across restarts | Rotating the key requires coordinated restarts, and a sync that changes it can break existing sessions |
| Metadata database in the chart | Lets the Helm chart deploy a bundled database for the metadata store | Short-lived test environments where data loss is acceptable | Not suitable for production, and the database lifecycle becomes tied to the release |
| External metadata database | Points Airflow at a PostgreSQL or MySQL instance you operate or a managed service | Production clusters that need backups, failover and upgrades managed separately from the chart | You own connection secrets, network policy and database sizing, and the cluster must reach the database during sync |
| Chart version pinning | Fixes the Helm chart version in the Argo CD Application or Flux HelmRelease | Teams that promote chart changes through environments and review each change | Pinning delays fixes until someone updates the version, so the upgrade path needs an owner |
| Image tag pinning | Fixes the Airflow image tag used by all components | Production releases, where the same image digest is promoted across environments | Tags must be rebuilt for each Airflow patch, and mutable tags can drift from what was tested |
| Argo CD repository secrets | Defines Git and Helm repositories as Kubernetes Secrets | Argo CD 3.0 and later, where repositories cannot be declared in argocd-cm | Credentials sit in cluster Secrets, so access to the argocd namespace must be restricted |
| Argo CD repository settings in argocd-cm | Declares repositories in the argocd-cm ConfigMap | Only for versions that still support repository configuration there, so confirm your version first | Settings in this ConfigMap are not supported in Argo CD 3.0 and later, so this option will fail on upgrade |
Setup sequence for an Argo CD or Flux managed Airflow release
This sequence moves the Airflow Helm release into Git-driven delivery in an order that keeps secrets out of version control, lets database migrations and the admin user job run under the GitOps controller, and leaves a clear health check at each stage.
Create the namespace and secrets outside the values file
The platform team creates the Airflow namespace and the Kubernetes Secrets for the metadata database connection, the Fernet key, the static API secret key and the admin user password before the release is synced. Each instance gets its own API secret key, and the values file references these Secrets by name rather than containing their contents. Done means the Secrets exist in the cluster, the values file contains no credential material, and the Git repository holds only references. Confirm the exact Secret key names against the Helm chart documentation for your chart version.
Set the hook flags for both jobs
The platform engineer sets useHelmHooks to false and applyCustomEnv to false on both createUserJob and migrateDatabaseJob in the values file. Argo CD and Flux do not run Helm hooks the way the Helm CLI does, so without these flags the database migration and the create-user job may never run. Done means a rendered template of the release shows both jobs as ordinary resources, and the flags are committed to the same Git path the controller watches.
Pin chart and image versions in Git
The team pins the Helm chart version and the Airflow image tag explicitly in the GitOps repository, using a tag that matches the Airflow 3 patch level the team has tested, rather than a floating tag such as latest. Confirm the Airflow release against the version life cycle table before pinning, since Airflow 2 has reached end of life and gets no fixes. Done means the pinned chart version, image tag and values file are merged to the main branch and the controller shows them as the desired state.
Sync the release and watch the rollout
The operator triggers the first sync in Argo CD or lets the Flux helm controller reconcile the HelmRelease, then watches the rollout in the controller's UI or with its CLI. The controller reports the application as synced and healthy only when the workloads it manages are ready. Done means the controller shows no pending hooks, no failed resources and no drift between Git and the cluster. If the sync stalls, check the controller's events for the migration and create-user jobs first.
Confirm migrations and the create-user job completed
The operator checks that the database migration job and the create-user job both reached Completed status and read their logs for errors. The migration job must finish before the scheduler and API server run against the metadata database, so a failed migration blocks the rest of the release. Done means both jobs show a successful completion, the metadata database reports the expected Airflow schema version, and an admin login is possible through the configured method.
Verify the scheduler, DAG processor, API server and triggerer
The platform team confirms that the scheduler, DAG processor, API server and triggerer pods are running and ready. In Airflow 3 the DAG processor runs as its own component, and the API server replaces the old webserver, so each must appear as a separate workload. Done means all four components report ready, the API server health endpoint responds, and a test DAG parses and runs to success. Confirm the component names and health endpoints against the Helm chart documentation for your version.
Confirm log access and clock alignment
The team runs a task that writes a log line, then opens that log from the API server to confirm the shared secret key and the cluster clocks work together. The production guide warns that unsynchronized clocks can cause forbidden errors when task logs are fetched. Done means the log opens in the UI without a permissions error, and the nodes report synchronized time. Record this check in the runbook so later upgrades repeat it.
Keeping the Fernet key, API secret and database credentials out of Git
The Airflow Helm chart production guide calls for a static API secret key, set through a Kubernetes Secret and different for each Airflow instance. In a GitOps setup this matters more than usual, because anything written into a values file is committed, reviewed, cached by the repo server and visible to everyone who can read the repository. The values file should only reference Secret names and keys for the metadata database connection, the Fernet key and the API secret key, and those Secrets should be created by a process outside the Git flow, such as a sealed secrets controller, an external secrets operator or a secrets manager with an approved sync path. Confirm with your security team which of these fits your controls before the first sync, because changing the source of a secret later means rotating it.
Argo CD 3.0 makes the same point for its own configuration: repositories must be defined as Kubernetes Secrets rather than in the argocd-cm ConfigMap, so Git credentials also belong in the cluster, not in the manifests Argo CD reads. Treat the Argo CD repository credentials, the Flux Git deploy key and the Airflow Secrets as one inventory with owners and rotation dates. Clock sync is the less obvious dependency: the production guide warns that unsynchronized clocks can cause forbidden errors when task logs are fetched, because the signed log requests depend on time. Confirm that every node running Airflow pods, and the Argo CD and Flux controllers, use NTP or an equivalent source before you rely on log viewing in the UI.
Where drift appears on Airflow upgrades
Drift on an Airflow upgrade rarely comes from one change. It comes from the chart, the image, the cluster state and the Airflow 3 architecture moving at different speeds. Each card below names one place where Git and the running release can disagree, and the check that catches it before the next sync.
Chart version changed without image change
A chart bump can alter templates, hook behavior and default values while the image tag stays the same. The Git diff then shows a chart update that looks harmless, but the rendered manifests differ. Compare the rendered output of the old and new chart versions with helm template before merging, and confirm that useHelmHooks: false and applyCustomEnv: false still sit on both createUserJob and migrateDatabaseJob.
Floating image tag
A tag such as latest or a minor-only tag lets the same manifest pull different images on different nodes or after a rebuild. The cluster then runs code that Git does not describe, and rollback no longer restores the earlier build. Pin the image to a full Airflow version tag, and confirm that the running pods report the same version as the one in the Git repository.
Manual kubectl edits reverted
An engineer who patches a Deployment or ConfigMap during an incident creates drift that the controller will undo on its next reconciliation. The change disappears without warning, sometimes during a scheduler restart. Ask the team to route every emergency change through Git, and set the sync policy and self-heal behavior deliberately so the reverts are expected rather than surprising.
Hook jobs skipped
If the GitOps tool does not run Helm hooks, migrations and user creation are skipped silently, and the new scheduler starts against an old metadata schema. The release reports healthy while Airflow fails on the first query. After each sync, check the migration job's completion status and confirm the schema version in the metadata database before releasing traffic.
Airflow 2 to 3 component changes
Airflow 3 replaces the webserver with an API server started by airflow api-server, and the DAG processor must run as its own component. A values file that still defines the old webserver layout can drift from what the upgrade guide requires. Compare the component list in the rendered chart with the Airflow 3 upgrade guide and confirm that the DAG processor and API server each have a running deployment.
Argo CD tracking and RBAC defaults
Argo CD 3.0 made annotation-based resource tracking the default, and update and delete RBAC now apply only to the application itself. Teams that relied on the older label-based behavior or broader permissions may find that some resources stop syncing or that users lose actions they used before. Review the 2.14 to 3.0 upgrade notes, then test RBAC for logs access with an explicit logs get permission in a non-production project.
Questions teams ask before deploying Airflow with GitOps
What does a GitOps deployment of Airflow cost, and what drives the price?
The cost comes from the scope agreed in writing before work starts, and DeployOpen does not publish a price list, so a quote follows from that scope. The main drivers are how many clusters and environments the release must reach, whether Argo CD or Flux is already running, how many secrets and external systems need wiring, and whether the metadata database and secret store already exist. Ask any quote to separate one-time work (chart values, Git layout, secret design, first sync, cutover) from recurring work (chart and image promotion, upgrades, and support if you retain it). Ask which items are included and which are your team's responsibility.
How long does it take to deploy Airflow with Argo CD or Flux?
Duration depends mainly on how mature your GitOps practice already is. A team with a working Argo CD or Flux pipeline, a secret store and a supported metadata database mostly needs the Airflow release and its hook settings added. A team starting from scratch also needs repository layout, RBAC, secret synchronization and promotion rules, which adds planning and review time. No timeline is given here because none is supplied for this work. Ask for a timeline in the written scope, with the assumptions about existing tooling listed, so you can check them against your environment.
What must our team provide before the engagement starts?
Plan to provide the following before work begins:
- Cluster access in the environments you control, with a supported Kubernetes version. The Airflow 3.3.2 prerequisites list Kubernetes 1.30 to 1.35.
- A Git hosting location for the Helm values, manifests and DAG repositories, with branch protection and review rules.
- A secret store that the cluster can read, such as your existing vault or an external secrets operator.
- A PostgreSQL or MySQL metadata database. The Airflow prerequisites list PostgreSQL 14 to 18 and MySQL 8.0, 8.4 and Innovation releases, and MariaDB and SQLite are not supported for this role.
- Named owners who can approve changes and handle access requests.
Can our in-house platform team deploy Airflow with GitOps on its own?
Yes, a platform team with Kubernetes and Helm experience can do this without a partner, and many do. The parts that tend to take the most experience are the Helm hook settings, keeping secrets out of values files, and planning chart and image upgrades under GitOps sync. Consider a partner if no one on the team has run Helm releases through Argo CD or Flux before, if you need the work finished within a fixed window, or if you want a documented runbook and an independent review of the design. Your team should still own the decisions about access, data and change approval.
Should we choose Argo CD or Flux for Airflow?
Choose on operating model and team fit, not on which tool is more popular. Argo CD runs as a Kubernetes controller with a web UI, manages multiple clusters and integrates SSO over OIDC, OAuth2, LDAP and SAML 2.0 with RBAC. Flux is built from GitOps Toolkit controllers, uses Kubernetes RBAC with impersonation for multi-tenancy, supports Helm, Kustomize and OCI artifacts, and offers flux bootstrap for installation. Both are Apache-2.0 licensed with no paid edition from the project. For Airflow, the deciding factors are usually your existing tooling, whether operators need a UI, and how you want to handle tenancy and secret synchronization. Confirm your choice against your own cluster count and approval workflow.
Which Helm hook settings must be changed for Argo CD or Flux?
Set useHelmHooks: false and applyCustomEnv: false on both createUserJob and migrateDatabaseJob. The Airflow Helm chart documentation states that database migrations will not run without these settings when Argo CD, Flux, Rancher or Terraform manages the release. Without them, the migration and user creation jobs can be skipped or misapplied during sync. Test these values in a non-production cluster first, then check that the migration job completes before the scheduler and API server roll out. Keep the same settings in every environment so promotion does not change them.
How do we verify an Airflow chart or image upgrade under GitOps?
Verify each upgrade in a lower environment before it reaches production, and make the check part of the promotion pipeline. Confirm the Airflow version support status first, since Airflow 2 reached end of life on April 22, 2026, and Airflow 3 is the line to plan for. For Airflow 3, the upgrade guide requires being on Airflow 2.7 or later and recommends checking DAG code with Ruff AIR301 and AIR302 rules. After sync, confirm that the migration job completed, the API server and DAG processor are running as separate components, DAGs parse, and a test run finishes. Keep the previous chart and image version in Git so a rollback is a commit, not a rebuild.
How do we keep secrets out of the Airflow values file under GitOps?
Keep passwords, the metadata connection string, the Fernet key and the API secret key in Kubernetes Secrets, and reference those secrets from the chart rather than writing values into the Git repository. The Helm chart production guide recommends a static API secret key that is different for each instance. Sync the secrets from your secret store through a controlled process so that no plaintext value is committed. Also check that cluster clocks are synchronized, because the chart guide warns that unsynchronized clocks can cause forbidden errors when task logs are fetched. Ask whether your secret store and sync tool are already approved for this workload.
What does DeployOpen do after the GitOps deployment goes live?
After go-live, the work falls into two groups. Handover covers documentation of the repository layout, chart and image promotion steps, the upgrade checklist and the rollback procedure, so your team can operate the release. Ongoing support, upgrades and continuous testing are available as retainers if you want them. Both are scoped in writing, and neither includes audit reports or certifications, which independent auditors and certification bodies issue. Ask for a written list of what is delivered at handover and what a retainer covers before you agree the scope.
How DeployOpen scopes and supports a GitOps Airflow release
DeployOpen plans, implements, documents and supports self-hosted platforms inside your own infrastructure, and a GitOps Airflow release is scoped the same way. The work runs in your cloud accounts, data centers or air-gapped network, against your own Git repositories and Kubernetes clusters, so the Argo CD or Flux configuration, the Helm release values and the secret references all stay under your control. Most engagements are fixed-scope: the written scope and deliverables are agreed before work starts. For this topic, that scope usually names the Argo CD or Flux instance being used, the Helm chart and Airflow image versions to be promoted, the hook settings for the migration and user creation jobs, the external Secret objects that hold the database connection and keys, and the environments that receive each promotion.
The written scope should also state who owns each decision. Confirm with your team which cluster hosts the GitOps controller, who approves a promotion from staging to production, whether drift is alerted or corrected automatically, and how a rollback is run if a migration job has already changed the metadata database. A quote follows from that scope, so ask for each deliverable to be listed separately, including the runbook and the handover session, and ask which items are one-time work and which recur. Your existing change management and any audit evidence requirements should be written into the scope too, because they shape what the delivered documentation must contain.
Ongoing support is available as a retainer. For a GitOps release, a retainer typically covers reviewing new Airflow patch releases and Helm chart versions, testing them in a non-production environment, updating the promoted versions in Git, and watching for drift after upgrades. Ask for the retainer terms in writing, including response expectations, which Argo CD or Flux and Airflow versions are covered, and who performs the work when a release changes the chart. DeployOpen does not issue audit reports, attestations or certifications. Where a release needs evidence for an assessment, DeployOpen prepares the controls and records, and your independent auditor decides what they establish.
