How to upgrade the Airflow Helm chart for Airflow 3 without losing your migration
Moving a Kubernetes Airflow 2 deployment to Airflow 3 is a chart and runtime change, not just an image bump. The official Helm chart moves webserver settings to apiServer, the webserver process is replaced by an API server started with airflow api-server, and the DAG processor must run as its own component started with airflow dag-processor. The supported starting point is Airflow 2.7 or later, and the upgrade guide recommends moving to the latest 2.x first. Airflow 2 reached end of life on April 22, 2026, so a deployment still on 2.x receives no fixes and should not wait longer than it must. Chart 1.22.0 supports Airflow 2.11 and later and 3.0 and later, so pairing the chart version with the image version is the first check you run. Read the chart's own upgrade guide, linked from the Airflow 3 upgrade guide, before editing any values file.
Your values file is where most of the work sits. Review every webserver key and map it to its apiServer equivalent, then remove settings the new components no longer read. Give the DAG processor its own deployment and confirm that the scheduler, API server and DAG processor all point at the same metadata database and DAG source. Each of these components needs its own resource requests and probes, so plan capacity for the extra pod rather than assuming the old webserver footprint is enough. If you run GitOps tools such as Argo CD, Flux, Rancher or Terraform, set useHelmHooks: false and applyCustomEnv: false on both createUserJob and migrateDatabaseJob, or the database migration will not run. Confirm this in a staging release with the exact chart and image versions you intend to deploy, and keep the migration job output as evidence for your change record.
What changes in the chart and runtime for Airflow 3
This table lists the Airflow 3 changes that affect a Kubernetes deployment managed with the official Helm chart, with the impact of each and the action your team needs to take. Read each row against your current 2.x values file and DAG code, because the chart values and the runtime components change together. The main takeaway is that the chart upgrade and the runtime split are one change, so both need to be tested in the same staging release.
| Change | Impact | Action needed |
|---|---|---|
| Webserver settings move to apiServer | Values that configured the 2.x webserver are not read the same way in the Airflow 3 chart, so a copied values file can deploy with the wrong settings. | Map every webserver key in your values file to apiServer and diff the rendered manifests against your 2.x release. |
| Webserver is replaced by airflow api-server | The UI and REST API are served by a new API server process started with the airflow api-server command, so the old webserver deployment no longer exists. | Confirm the API server deployment, its service and any ingress or probe paths that pointed at the webserver. |
| DAG processor runs as a separate component | The DAG processor must run as its own process started with airflow dag-processor, so a chart that does not deploy it leaves DAGs unparsed. | Enable and size the standalone DAG processor in the values file and confirm DAGs appear in the UI after rollout. |
| SubDAGs removed in favor of TaskGroups | DAGs that use SubDagOperator fail to load on Airflow 3, which can stop scheduling for those pipelines after the image changes. | Rewrite SubDAG usage as TaskGroups before the image tag changes, and check DAG parsing in staging. |
| Task code loses direct metadata database access | Tasks that open Airflow database sessions or import models directly stop working, and worker pods can fail at runtime. | Move task database calls to the REST API or Task SDK, and confirm worker pods have only the connections they need. |
| Simple auth manager is default and only one runs at a time | Authentication behavior changes on upgrade, and a 2.x FAB setup does not carry over unless the auth manager is configured explicitly. | Set the auth_manager option deliberately, plan role mapping and test sign-in in staging before cutover. |
| CeleryKubernetesExecutor and LocalKubernetesExecutor no longer supported as of 3.0.0 | Deployments that used these hybrid executors cannot run on Airflow 3 as configured, which affects how tasks are routed to workers. | Choose a supported executor configuration, such as KubernetesExecutor or several executors, and retest task routing before upgrading. |
Which migration job settings GitOps tools need
Argo CD, Flux, Rancher and Terraform do not run Helm the same way a person running helm upgrade does. Their renderers and reconciliation loops can skip or defer Helm hook resources, and the Airflow chart relies on hooks to run its database jobs. The chart documentation says that with these tools, both createUserJob and migrateDatabaseJob must be configured with useHelmHooks: false and applyCustomEnv: false. If these values are left at their defaults, the migration job may not run, and the release can appear healthy while the metadata database is still on the Airflow 2 schema.
- Set both jobs explicitly: Apply
useHelmHooks: falseandapplyCustomEnv: falsetocreateUserJobandmigrateDatabaseJobin the values file that the GitOps tool renders. - Run migration as a separate step: Plan how the migration job is triggered and ordered relative to the new API server and DAG processor rollout, since the tool no longer runs it as a hook.
- Watch sync status and job completion: Confirm that the job object exists, completes and logs a successful migration before treating the sync as finished.
- Keep values in version control: Store the overrides in the same repository as the application manifest so reviewers can see the change.
- Check Terraform providers separately: If Terraform manages the release, confirm how the Helm provider handles hooks in your provider version.
Confirm the exact key names and defaults against the documentation for the chart version you deploy, because these settings are version-specific.
When an operator runs Helm directly, hooks behave as the chart designers intended. The Helm release runs the database jobs as hook resources at the right point in the install or upgrade, and the default useHelmHooks: true and applyCustomEnv: true usually work without overrides. This is the simplest path for a small team with a single cluster, but it still depends on the operator running the upgrade in the correct order and waiting for the migration to finish.
- Keep the defaults unless a reason exists: Do not copy GitOps overrides into a plain Helm values file, since disabling hooks without a delivery tool to run the jobs will skip migration.
- Use a single upgrade command in staging first: Run the same
helm upgradein a staging release to observe hook execution and job logs. - Confirm the release revision: Check that the new revision reports success only after the migration job has finished.
- Record the values used: Save the exact values file and chart version with the change record for audit and rollback.
- Plan the rollback path: A Helm rollback does not reverse a database migration, so the metadata database backup must exist before the upgrade.
Before relying on defaults, confirm with the chart documentation for your chart version that hook-based migration is the intended behavior for your setup.
Whichever tool you use, the setting is correct only if the migration actually runs against the metadata database. A successful sync or release status does not prove that migration completed. Verification should check the rendered manifests, the job lifecycle and the resulting database schema. Because a skipped migration can look like a successful upgrade until the new components start, this check belongs in the staging run before any production change.
- Inspect rendered output: Render the chart with the same values the tool uses and confirm which job manifests are produced and whether hook annotations appear.
- Confirm job execution: Check that the migration job runs to completion in the target namespace and that its logs show the migration finished.
- Compare schema state: Verify the database version recorded by Airflow after the job, not just the pod status.
- Test a failed migration: Confirm that a failed job blocks or flags the release, so the team learns about it before users do.
- Document the evidence: Keep rendered manifests, job logs and the staging result as change evidence.
Ask whoever supports your GitOps tool to confirm how it treats Helm hooks in the version you run, and record that answer with the chart version.
What to prepare before touching the staging release
The staging release only tells you something useful if the inputs are settled first. These six preparation items cover the data you must be able to restore, the code and configuration that the new chart will read, and the cluster conditions that must hold before the chart runs.
Metadata database backup with restore test
Take a consistent backup of the metadata database on PostgreSQL or MySQL before any migration job runs. SQLite is only for testing and MariaDB is not supported as a metadata database, so a backup taken from either is not a valid safety net. Restore the backup into a separate database and confirm that the DAG runs and task history load. A backup you have not restored is not yet a rollback plan.
DAG code inventory and Ruff findings
List every DAG repository, shared module and dependency the Airflow release reads, then run Ruff with the AIR301 and AIR302 rules across all of them. Record each finding with its file and owning team. The chart upgrade will not fix DAG code, so a finding left open here becomes a task failure in staging after the release is already running.
Custom plugins and webserver_config.py changes
Inventory each custom plugin, menu item, blueprint and webserver_config.py change, and mark which ones depend on the Flask AppBuilder webserver. Airflow 3 replaces the webserver with an API server, so these items need a port or a removal decision before the staging release can start cleanly. Ask each plugin owner for the target behavior, not only the current code.
Values file diff against the chart version
Compare your current values file with the chart release you plan to deploy, and move webserver settings to apiServer. Check the chart's own upgrade notes for the chart version you select, because chart 1.22.0 supports Airflow 2.11 and later and 3.0 and later. Keep the diff in version control so that reviewers can see every change and the rollback is a single revert.
Staging namespace that mirrors production
Build a staging namespace that matches production on executor type, secret backends, DAG delivery method and external database. For each executor in use, confirm the staging workers can run the same images and reach the same systems. If the production secrets come from an external store, staging should read from a separate store with equivalent keys, so you can test the migration jobs without touching production credentials.
Kubernetes version within the supported range
Confirm that the cluster version falls between 1.30 and 1.35, which is the range listed for Airflow 3.3.2. Check the cluster version in staging and in production, because the upgrade may land on a version that is outside the range after a planned cluster upgrade. If your cluster runs a version outside this range, resolve the cluster upgrade first, before debating the Airflow chart.
The upgrade sequence for the chart in staging, then production
This sequence moves a Kubernetes Airflow deployment from the 2.x chart to Airflow 3 in a staging release first, with a rollback point before each production step, so the team knows the chart values, migration job and runtime components work before production is touched.
Move to the latest 2.x and run the chart on 2.11
Upgrade the staging release from its current 2.x version to the latest 2.x line, which the upgrade guide recommends, and stop there as a stepping stone. The chart release that supports Airflow 2.11 is the one to pin for this stage. Your platform team performs the change in staging and confirms that the scheduler, workers, webserver and DAG parsing all run cleanly on 2.11. This step is done when the staging release is healthy on 2.11 and the production release is still untouched. Record the chart version and image tag so you have a known-good point to return to.
Run Ruff AIR checks on the DAG repository
Before changing any chart values, run the Ruff AIR301 and AIR302 rules against the DAG repository to find code that will break under Airflow 3. DAG authors and the platform team review the findings together, separating automatic fixes from changes that need a person to decide. Fix the flagged code on a branch and wire the check into CI so new DAGs are checked. This step is done when the check passes on the main branch with no unresolved findings and the list of remaining manual changes is written down and assigned.
Back up the metadata database and record a rollback point
Take a consistent backup of the PostgreSQL or MySQL metadata database used by the production release, and confirm that the backup restores into a separate instance. Store the current Helm values, the rendered manifests and the image tags in version control so the previous state can be reapplied. The database backup and the restore test are the rollback point for the production migration. This step is done when a restore test has completed in a separate environment and the team has written down who can trigger a rollback.
Render the 3.x values with apiServer and dag-processor
Render the chart with the Airflow 3 values in staging and review the output before applying it. Rename the webserver settings to apiServer, as the chart's upgrade guide describes, and enable the standalone DAG processor as its own component, since Airflow 3 requires it to run separately from the scheduler. Keep the Helm values in Git and diff the rendered manifests against the 2.x output. This step is done when the rendered manifests show an API server deployment, a DAG processor deployment and no leftover webserver resources.
Set the GitOps flags on the migration jobs
If Argo CD, Flux, Rancher or Terraform manages the release, set useHelmHooks to false and applyCustomEnv to false on both createUserJob and migrateDatabaseJob. Without these settings, the chart's database migration does not run under those tools. Your GitOps owner checks the values file and confirms the migration job appears as a managed resource in the sync view. This step is done when a dry-run sync in staging lists the migration job and shows the flags set as intended.
Run the metadata database migration in staging
Let the migration job run against the staging metadata database, using the Airflow 3 image, and watch the job logs until it completes. Check the job status and the migration output for errors before continuing. Reserialize DAGs and trim history only if your runbook calls for it. This step is done when the job finishes successfully and the staging database reports the Airflow 3 schema, with the rollback point from the backup step still valid.
Start the API server and verify DAG parsing and tasks
Bring up the API server and DAG processor in staging, then confirm that DAGs parse and appear in the UI, that a scheduled DAG run executes its tasks on the configured executor, and that task logs are readable. Test the REST API with an authenticated request and confirm that existing scripts and integrations behave as expected. Check that the scheduler runs with the expected concurrency. This step is done when parsing errors are cleared, a test run completes end to end and the REST API returns expected responses.
Repeat the sequence in production with a rollback point
Run the same steps in production in the same order, with a fresh database backup and a current copy of the 2.x values before each change. Define the rollback point before each production step: the backup before the migration, the previous release revision before the API server and DAG processor switch. Hold the change window open until the team has confirmed parsing, task runs and API calls. This step is done when production passes the same checks as staging and the rollback owner has signed off on the final state.
How to test the chart upgrade in a staging release before production
Test the chart upgrade in a staging release that mirrors production values, not a generic test install. Copy the production values file, the same image tag, the same external metadata database type and the same secret references, then change only what the upgrade requires: apiServer replacing the webserver settings and the standalone DAG processor component started with airflow dag-processor. If the staging values drift from production, the test proves little about the production rollout. Run the staging upgrade on a copy of recent production metadata or a restored backup so that migrations act on realistic data, and confirm the chart's own upgrade notes for the chart version you are moving to, which are linked from the Helm chart documentation.
Verify that the migration jobs actually run under your delivery tool. If you deploy with Argo CD, Flux, Rancher or Terraform, confirm that createUserJob and migrateDatabaseJob have useHelmHooks: false and applyCustomEnv: false set, because otherwise database migrations silently do not run. Check the job logs and the airflow db migrate result in staging, and confirm the database schema version before and after. Then run a sample of DAGs end to end through the new scheduler, DAG processor and API server, covering at least one DAG that uses a TaskGroup and one that reads its own task state, so that breaking changes surface before production. Compare the rendered manifests from the old and new chart versions to catch unexpected changes in probes, service names, ingress paths or resource requests.
Write the rollback plan before the staging run, and rehearse it. It should restore the database backup taken immediately before the upgrade, redeploy the previous chart release with its previous values file, and confirm the previous image tag is still available in your registry. Ask your team to confirm how long a restore takes on your database platform, since no duration is assumed here. Keep the staging results, rendered manifest diffs and migration logs as change evidence, which is useful for change management reviews. Treat a failed rollback rehearsal as a blocker for the production change window.
What makes a Helm chart upgrade take longer
This table lists the factors that expand the work of moving a Kubernetes Airflow release from the 2.x chart to the Airflow 3 chart, with the reason each one adds effort and what to request in a scoped quote. Read it before estimating, because most of the duration comes from the items you already run rather than from the chart itself. The main takeaway is that a clean values file and a rehearsed staging release shorten the work more than anything else.
| Driver | Why it adds effort | What to ask for |
|---|---|---|
| Number of custom values overrides | Each override set for the webserver must be checked against the apiServer equivalents in the chart upgrade guide, and anything that no longer maps needs a decision. | A line-by-line map of current overrides to the Airflow 3 values, with unsupported keys flagged. |
| GitOps tool and hook configuration | With Argo CD, Flux, Rancher or Terraform, migration jobs do not run unless useHelmHooks and applyCustomEnv are set to false on both createUserJob and migrateDatabaseJob, so the release must be tested under the actual sync tool. | Confirmation of the sync tool version in use and a staging sync that shows migration jobs completing. |
| Custom plugins and Flask AppBuilder UI code | Airflow 3 replaces the webserver with an API server, so plugins and webserver_config.py changes may need porting before the release can start cleanly. | An inventory of plugins and UI customizations, and a list of which ones need rewriting versus removal. |
| DAG count and TaskGroup rewrites | SubDAGs are removed in Airflow 3 in favor of TaskGroups, and DAG code must pass Ruff AIR301 and AIR302 checks, so a large DAG repository adds review and test time. | A Ruff AIR report across the repository and a count of SubDAG usages to rewrite. |
| Metadata database size and history cleanup | Large history tables lengthen the migration window and raise the risk of a failed migration, so backups and trimming must come first. | A current database size, a backup and restore test, and a plan for cleanup before the migration job runs. |
| Executor mix | CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported in Airflow 3, so deployments on these need an executor change, not just a chart change. | Confirmation of the executors in use and a decision on the replacement for each hybrid setup. |
| Secrets stored in Kubernetes Secrets and their encryption at rest | Kubernetes stores Secrets unencrypted in etcd by default, so the upgrade must also confirm how connection and variable secrets are protected and how the chart references them. | Evidence of etcd encryption at rest or an external secrets store, and a list of secret references the chart will use. |
| Number of environments | Each environment needs its own values file, its own migration run and its own verification, so the work scales with every extra cluster or stage. | A list of environments, the promotion order and whether staging mirrors production settings. |
Questions teams ask before upgrading the Airflow Helm chart
What does it cost to upgrade the Airflow Helm chart for Airflow 3?
No price is published for this work, and the cost depends on the written scope. The main cost components are the people who review your current values, rewrite the webserver settings as apiServer settings, split out the DAG processor, adjust your GitOps setup, and test the release in staging. Ask for a quote that lists each of these as a separate line, states which items are one-time work, and identifies any recurring items such as retained support. The size of your environment matters as much as the chart itself: the number of Helm releases, the number of clusters, and whether you run a database migration job under a GitOps tool all change the effort. The chart upgrade is also connected to the Airflow 3 upgrade guide, which asks you to be on Airflow 2.7 or later before upgrading.
How long does an Airflow Helm chart upgrade take?
The duration depends on how far your deployment is from the Airflow 3 requirements and how many environments must move. Teams that are already on a recent 2.x release, use the official chart, and keep their values in version control usually have a shorter path than teams with custom images, hand-edited manifests, or DAG code that has not been checked. The main drivers are the number of releases, the staging cycles needed to prove the migration job runs, and the time to fix DAG code flagged by Ruff AIR301 and AIR302 checks. Chart 1.22.0 supports Airflow 2.11 and later and 3.0 and later, so confirm the chart version you will pin before scheduling. Ask any provider for a timeline tied to named milestones in the written scope rather than a single estimate.
Can our own platform team upgrade the Airflow Helm chart without a partner?
Yes, a team with Kubernetes, Helm and Airflow experience can plan and run this upgrade internally. The chart documentation and the Airflow 3 upgrade guide describe the required value changes, and the chart's own upgrade guide covers the rest. The areas that most often need outside help are GitOps tools such as Argo CD, Flux, Rancher or Terraform, where the chart requires useHelmHooks set to false and applyCustomEnv set to false on both createUserJob and migrateDatabaseJob, or migrations will not run. Other common gaps are a metadata database backup and rollback plan, and staging tests that match production. Decide early which of these your team will own and which you want reviewed by someone who has done the work before.
What does the upgrade require from our staff and environments?
Plan for staff who know your Kubernetes cluster, your Helm releases, your GitOps tool and your Airflow deployment, plus a database owner for the metadata database. Your environment needs a staging release that mirrors production closely enough to test the migration job, the API server and the separate DAG processor. The Airflow prerequisites list Python 3.10 to 3.14, PostgreSQL 14 to 18 or MySQL 8.0, 8.4 and Innovation releases, and Kubernetes 1.30 to 1.35 for Airflow 3.3.2, so confirm that your cluster and database versions fit before the work starts. MariaDB is not supported as a metadata database, and SQLite should not be used in production. Ask the provider to list every access, tool and environment they need from you in the written scope.
Can we keep running Airflow 2 while we plan the chart upgrade?
Yes, but the window is limited. Airflow 2 entered limited maintenance on October 22, 2025 and reached end of life on April 22, 2026, after which it gets no fixes. Limited maintenance covers security and critical bug fixes only. Keeping Airflow 2 running therefore means accepting a growing gap in patches, and that gap should be documented in your risk register. If you stay on 2.x for a while, the upgrade guide recommends moving to the latest 2.x first, so the chart you run today should not be left far behind. Ask your provider to show how an Airflow 2 deployment would be kept patched during the planning period and what the cutover date depends on.
How does the Helm chart upgrade relate to DAG and plugin changes?
They are separate workstreams that must meet at the staging release. The chart change moves webserver settings to apiServer, runs the DAG processor as its own component started with airflow dag-processor, and manages the migration job. DAG and plugin changes are code changes: the Airflow 3 upgrade guide says task code can no longer use database sessions or models directly, SubDAGs are removed in favor of TaskGroups, and Flask AppBuilder plugins need porting. Run Ruff AIR301 and AIR302 checks on your DAG repository early, and test the chart in staging with the same DAGs and plugins you will run in production. A chart that deploys cleanly can still leave DAGs failing at parse or run time, so ask for both sets of checks in the written scope.
What should we ask for in a written scope for this upgrade?
Ask for a scope that names each deliverable: the chart version to pin, the values file diff from webserver to apiServer, the DAG processor deployment, the GitOps configuration changes, the migration job settings, and the staging test plan with pass criteria. Also ask for the metadata database backup and rollback steps, the list of DAG and plugin items found by Ruff checks, and the handover documentation for your team. Confirm what is excluded, such as DAG rewrites, provider upgrades or Python changes, and whether ongoing support is offered as a separate retainer. Most DeployOpen engagements are fixed-scope, so the agreed scope and timeline are written down before work begins.
How DeployOpen scopes, delivers and supports the Airflow chart upgrade
DeployOpen plans and delivers the Helm chart move to Airflow 3 inside your own Kubernetes clusters, starting from a written scope and a staging release. The work covers chart values, the separate DAG processor, migration jobs under your GitOps tool, and handover documentation your platform team can run.
How we scope
- We inventory your current chart values, webserver settings and deployed Airflow version before quoting.
- The written scope names the clusters, namespaces, GitOps tool and chart versions in each environment.
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
How we build
- We move webserver settings to apiServer values and add a standalone DAG processor deployment.
- We set useHelmHooks and applyCustomEnv for the migration jobs to match your Argo CD, Flux, Rancher or Terraform setup.
- We run the full chart upgrade in a staging release first and record the migration job results.
How we hand over
- You receive the values files, GitOps manifests and a runbook covering the upgrade and the rollback path.
- We document each verification step so your team can repeat the staging test before the production change.
- Knowledge transfer sessions cover the chart version, the migration jobs and the checks that confirm success.
How we support
- Ongoing chart upgrades, patch updates and continuous testing are available as a retainer after handover.
- Support covers Airflow 3 patch releases and chart version changes that your team schedules.
- Ask us to confirm the support scope, response expectations and exclusions in the written agreement.
