Upgrading Apache Airflow to a newer Python version: when and how
Python is tied to the Airflow release you run, so the Python upgrade is part of the Airflow decision, not a separate task. If your workers, schedulers or images run on an older interpreter, or if a move to Airflow 3 forces a change, start by matching the Airflow release to a Python version from the supported range. The 3.3.2 prerequisites list Python 3.10 to 3.14, so any target in that range is supported for that release. Check the prerequisites page for the exact Airflow release you plan to install, because the list changes between releases. Airflow 2 reached end of life on April 22, 2026, so a Python change made only to keep Airflow 2 running now also carries the risk of running without fixes. Confirm your target Airflow version before you choose the Python version.
Two release changes shape the timing. Airflow 3.1 removed Python 3.9 support, so any environment still on 3.9 must move before it can take 3.1 or later. Airflow 3.2 added Python 3.14 support, which gives teams on 3.10 to 3.13 a newer interpreter option once they reach that release. Each Airflow release publishes a constraints file per Python version, and a plain pip install apache-airflow can produce an unusable installation, so install with the constraints file that matches both the Airflow version and the interpreter. Check the Airflow release notes for any other Python-related changes in the releases between your current and target versions.
The practical work is the dependency and image check. Ask your team to list every provider package, custom operator, and third-party library your DAGs import, then confirm each one supports the target interpreter. Packages that lag behind often hold the upgrade back, so a dependency that has no build for the new Python version can block the whole plan. Rebuild worker and scheduler images from scratch against the new constraints file, run your DAG import tests and a staging run of representative tasks, and keep the previous image available for rollback. Confirm with your own engineers which Python version your base image and CI runners use, because a mismatch there causes failures that look like Airflow bugs.
Which Python changes break which Airflow releases
This table compares each Python-related change in the researched Airflow releases with its effect on your environment and the action to take. Read it by release: find the version you are moving to, then check the impact column against your workers, schedulers, images and any custom code. The main takeaway is that Python 3.9 support ends in 3.1, and that Airflow 3.2 changes image builds and Python-side database code, not only the interpreter.
| Change | Impact | Action needed |
|---|---|---|
| Python 3.9 removed in Airflow 3.1.0 (September 25, 2025) | Environments still on Python 3.9 cannot run Airflow 3.1 or later | Move interpreters to 3.10 or later before upgrading past 3.0 |
| Python 3.13 added in Airflow 3.1.0 | Python 3.13 becomes a supported interpreter for Airflow 3.1 | Test your dependencies on 3.13 if you plan to use it |
| Structlog logging everywhere in Airflow 3.1.0 | Custom logging configuration written for the older logging setup may not apply as before | Review custom log config, handlers and log shipping, then test log output in staging |
| Python 3.14 added in Airflow 3.2.0 (April 7, 2026) | Python 3.14 becomes a supported interpreter for Airflow 3.2 | Confirm that each provider and custom dependency installs on 3.14 before switching |
| SQLAlchemy 2.0 upgrade in Airflow 3.2.0 | Python code that uses the database layer directly may need changes | Search custom operators, hooks and plugins for SQLAlchemy calls and test them against 2.0 |
| MySQL client dropped from images in Airflow 3.2 | Image builds that install or rely on the MySQL client may fail or lose connectivity | Add the client to your own image if you need it, and verify MySQL connections after the rebuild |
| Python 3.10 to 3.14 supported in Airflow 3.3.2 | Sets the full supported interpreter range for the current patch release | Pick a target interpreter from this range and match it to the constraints file for that Python version |
What to inventory and back up before changing the Python version
A Python upgrade touches the metadata database, the dependency set, the container images and the test environment at once. The cards below cover the six items to pin down before anyone changes a Python version number, so that a failed upgrade can be rolled back and diagnosed.
Metadata database backup
Take a consistent backup of the metadata database before any image or interpreter change. The production guide warns that the default SQLite backend is for testing only and can lose data in production, so confirm you are on PostgreSQL or MySQL and that restore has been rehearsed. Record the backup location, the Airflow version and the Python version it was running on so a rollback has a known starting point.
DAG code and custom operators
List every DAG file, custom operator, hook and plugin that imports a third-party package with native extensions or a pinned version. These are the components most likely to fail on a new interpreter. Mark each one with its owning team and whether it has automated tests, because untested code is where Python-version regressions surface late.
Current and target constraints files
Identify the constraints file that matches your current Airflow release and Python version, and the one for the target pair. Airflow publishes one per release and Python version because a plain pip install apache-airflow can produce an unusable installation. Diff the two files to see which pinned packages change, and keep both in version control so the build is reproducible.
Custom images and base layers
Record the base image tag, the Python interpreter it provides, and every RUN step that installs system packages or compiles dependencies in each custom image. Confirm the base image publishes the target Python version and that its OS packages still provide the libraries your dependencies link against. Changing a base layer silently changes the interpreter, so pin tags by digest where your registry allows it.
Provider packages and lagging dependencies
List each installed provider package and the Python versions it supports, then note any third-party client library that has not yet released a build for the target interpreter. Those gaps decide whether the upgrade can proceed on schedule. Ask each provider's maintainers or check their release notes before committing to a date, and keep the old image available until every gap is closed.
Staging that mirrors production
Build a staging environment with the same executor, worker image, scheduler count and database engine as production, and run a representative set of DAGs against it. Compare task logs, runtimes and resource use with production baselines. A staging environment that uses a different executor or fewer workers will not reveal the issues that matter most for this change.
How the Python move differs for workers, schedulers and images
The scheduler, DAG processor and API server all run the Airflow code and the Python interpreter that imports your DAG files, so they change first. In an Airflow 3 deployment these are separate processes, which means each one needs the new interpreter and the same installed package set. Running them on different Python versions is a common source of confusing import failures, so treat them as one release.
- Interpreter on every control-plane host: Confirm that each scheduler, DAG processor and API server host or pod runs a Python version listed in the Airflow 3.3.2 prerequisites, which currently covers 3.10 to 3.14.
- Installed package set: Reinstall Airflow and providers against the constraints file for the target Airflow version and Python version, not against floating versions.
- Database compatibility: Confirm that the PostgreSQL or MySQL metadata database versions in your environment appear in the prerequisites for the target release before you restart components.
- Parsing behavior: Parse every DAG file under the new interpreter in a staging environment and compare the import errors and DAG counts with the current production baseline.
- Rollout order: Confirm with your own change process whether the DAG processor, scheduler and API server can be restarted in a fixed order, and what the rollback path is if a component fails to start.
Before cutover, check that the control plane starts cleanly on the new interpreter and that parsing produces no new import errors.
Workers execute task code, so the Python version must match the environment that runs your operators, hooks and third-party libraries. With CeleryExecutor, each worker host carries its own virtual environment or image and must be moved as a set. With KubernetesExecutor, each task pod starts from the image named in the configuration, so the worker image and the task image must agree on Python version and packages. Mixed fleets during a rolling change can produce tasks that behave differently depending on which worker picks them up.
- Worker environments: Rebuild the Celery worker virtual environments or images from the constraints file for the target Python version, then drain and replace workers rather than patching them in place.
- Task pod images: Confirm which image the KubernetesExecutor uses for task pods and rebuild it alongside the scheduler image.
- Queue draining: Confirm with your platform team how long running tasks take to finish and whether workers can be drained without losing work.
- Native libraries: Test packages with compiled extensions, such as database drivers and data libraries, because wheels for a new Python version may not exist for every package you use.
- Executor mix: Confirm which executors your deployment runs, since the executor documentation notes that several executors can coexist since Airflow 2.10.0.
Before cutover, run a representative task on each worker type and each executor path under the new Python version and compare its output with the current result.
Container images bundle the interpreter, Airflow, providers and your DAG dependencies into one artifact. Upgrading Python usually means building a new base layer and rebuilding every image that inherits from it, including custom images with additional packages. The production guide recommends container images for Kubernetes deployments, so this is often the path that carries the most weight. Images should be tagged so that the scheduler, API server, workers and task pods can be traced back to one build.
- Base image selection: Confirm which official Airflow base image tags are available for the target Python version and pin the digest your build uses.
- Custom layers: Rebuild every team's custom image, since a change in the base layer does not rebuild derived images on its own.
- Build inputs: Build from the constraints file for the target release and Python version, and record the resulting package list in the build output.
- System packages: Check operating system packages and client libraries the image installs, because some change with the base image.
- Image scanning: Run your organization's vulnerability scanning and signing steps on the new images before they reach any cluster.
Before promotion, confirm that the image tags deployed to each component match the build record.
Installs from PyPI on virtual machines and installs in air-gapped networks follow the same logic but need more preparation. Airflow publishes a constraints file per release and Python version, and a plain pip install of apache-airflow can produce an unusable installation, so the constraints file is the baseline for any reproducible install. In an air-gapped network, the wheels, constraints file and any provider packages must be mirrored into the internal repository before work starts, and an unmirrored dependency will block the build.
- Constraints file: Use the constraints file for the target Airflow version and Python version as the single source of resolved versions.
- Internal mirror: Confirm that your internal package mirror holds every wheel, including transitive dependencies, for the new interpreter.
- Python runtime: Confirm how the new Python interpreter is delivered to restricted hosts and who approves it through your software intake process.
- Offline validation: Install into a build host with no outbound access to prove the mirror is complete before you attempt the production install.
- Provider compatibility: Confirm with each provider's documentation which Python versions it supports before you pin it.
Before cutover, record the exact versions installed on every host so that the rollback target is known.
The upgrade sequence: staging, constraints, image rebuild and cutover
This sequence moves Airflow workers, schedulers and images to a newer Python version in an order that keeps a known-good rollback point throughout, so each change can be tested before it reaches production.
Confirm the Airflow and Python pair
Check the target Airflow release against its prerequisites page, which lists the supported Python versions, metadata database versions and Kubernetes versions. For Airflow 3.3.2, the listed Python range is 3.10 to 3.14. Confirm that your current Airflow version can move to the target release, and note any Python version the release has dropped, since Airflow 3.1 removed Python 3.9. The output is a written pair (Airflow version and Python version) signed off by the platform owner.
Pull the matching constraints file
Download the constraints file for the exact Airflow release and Python minor version from the constraints path in the Airflow repository, rather than resolving dependencies from a plain install. Store the file in your build repository and pin its version alongside the Airflow pin. The output is a constraints file checked into source control, with the Airflow and Python versions recorded in its name and in the build configuration.
Rebuild the images on the new Python base
Update the base image and the build definition to the target Python version, install Airflow and your provider packages using the constraints file, and build a new image tag for each component that runs Airflow code, including schedulers, workers, the API server and the DAG processor. Confirm that the build fails on any conflicting dependency rather than silently overriding it. The output is a set of tagged images with a build log and a dependency list that can be compared against the previous image.
Test dependencies that lag behind
Install each custom package, private library and provider your DAGs import into the new image, and run their test suites under the target Python version. Compiled extensions, pinned libraries and vendor SDKs are the usual blockers, so check each one against its own release notes before deciding to upgrade or replace it. Record any package that has no compatible release for the target version, since that item will block cutover. The output is a dependency register with a pass or blocker status for every package.
Run DAG and integration tests in staging
Deploy the new images to a staging environment that uses a copy of the production metadata database or a representative restore, then parse every DAG, run representative tasks against non-production connections and compare task outcomes with production history. Include the connection types and operators your organization depends on most. The check is complete when DAG parsing shows no import errors and the test runs match the expected results for each critical pipeline.
Define the rollback point before cutover
Before any production change, record the image tag currently running in each component and take a consistent backup of the metadata database. Store both with a note of the Airflow version, Python version and constraints file in use. Confirm that the previous image can still be pulled and that the backup can be restored in staging. The output is a rollback record that a second engineer has reviewed and that the team can execute without rebuilding anything.
Roll out components in a planned order
Stop new DAG runs or pause the schedule for the affected pipelines, then update components in a fixed order defined in the runbook, normally starting with the components that do not hold state. Deploy the new image to schedulers and workers, confirm that each component reports healthy and that task instances start running, and only then move the next component. Operators should watch for parse errors and failed task starts during the rollout. The output is a completed rollout log with the time and image tag of each component.
Verify production and keep the rollback path open
Once all components run the new image, confirm that the scheduled pipelines complete on their next runs, that logs and metrics arrive as before, and that no task fails on the Python change. Keep the previous image and backup available until the agreed monitoring window closes, and record the decision to retire them. The output is a signed cutover record that names the final Airflow and Python versions in production.
Why dependencies that lag behind block a Python upgrade
The Python upgrade is only as fast as the slowest dependency. Airflow itself installs on each supported Python version, but the libraries beneath it do not always move at the same pace. Compiled extensions and packages that ship only source distributions need a prebuilt wheel or a working local build for the new interpreter. Pinned libraries in your DAG repositories, in internal shared packages and in the images your teams build can all lack a compatible release. Before changing the base image, confirm for each target Python version that every direct and transitive dependency installs cleanly from your package mirror, including any air-gapped mirror you run.
Provider packages are the dependency most specific to Airflow. Each provider must support both the Airflow release you run and the Python version you are moving to, so an upgrade can stall because a provider you depend on has not caught up. Check the minimum Airflow version and supported Python versions for every provider you install, not only for core. Where a provider lags, your options are to hold the Python version, pin an older provider that still works, or replace the operator or hook. Each option changes the test scope, so decide which one applies before the cutover date is set.
Find these blockers early by resolving against the constraints file for the target Airflow release and Python version, then running it in CI. A resolution run that fails or produces a surprising downgrade shows a conflict before any image is built. Run your DAG test suite, provider-specific operator tests and image build on each target Python version in parallel, not just once at the end. Ask your team to record which packages were added, removed or changed in the resolved set, and to confirm with the package maintainers where a wheel is missing. Those records become the blocker list that drives the rest of the plan.
What makes a Python upgrade take longer
This table compares the factors that change how much work a Python move needs for Airflow workers, schedulers and images. Read each row against your own estate: the more a driver applies, the more images, dependencies and environments you must rebuild and retest. The main takeaway is that the Python version itself is rarely the slow part; the dependency surface and the number of places Python is pinned usually are.
| Driver | Why it changes effort | How to keep it down |
|---|---|---|
| Custom images and workers | Each image built on the Airflow base must be rebuilt, scanned and redeployed on the new Python, and each worker pool needs a rollout plan. | Consolidate images onto one base and one build pipeline, and confirm which worker queues must move together. |
| Pinned or compiled dependencies | Packages with C extensions or pinned versions may lack wheels for the target Python, forcing source builds or version changes. | Inventory requirements files, check wheel availability for the target Python, and remove unused pins. |
| Custom operators and hooks | Plugin code and provider-dependent operators must import and run on the new interpreter and the matching Airflow and provider versions. | List custom code by owner, then test imports and operator execution in a staging environment. |
| Size of the DAG estate | More DAGs mean more parsing and more task runs to validate, and a wider set of libraries that DAG code imports. | Group DAGs by the libraries they use and run a representative subset first, with parse checks on all of them. |
| Number of environments and air-gapped mirrors | Each environment needs its own image and constraint set, and air-gapped networks need the new packages mirrored before deployment. | Confirm which environments move first and ask whether your package mirror already holds wheels for the target Python. |
| Combined Python and Airflow major version change | Moving Python together with an Airflow major upgrade doubles the variables, since the Airflow release also changes APIs, database migration and DAG code. | Ask for the two changes to be sequenced in separate stages, with the Python move tested on the current Airflow version first where the release supports it. |
Questions buyers ask before moving Airflow to a newer Python
What does a Python upgrade for Airflow cost, and what drives the quote?
A quote follows from a written scope, so the cost depends on what the engagement covers: how many schedulers, workers, API server and DAG processor images need rebuilding, how many DAG repositories and custom operators need dependency testing, and whether the work includes staging validation and a rollback plan. DeployOpen does not publish a price list. Ask for the scope items, the deliverables and the assumptions behind each line, and ask which costs are one-time (the migration and testing) and which recur (any retainer for ongoing support or upgrades). Ask your team how many environments and images are in scope, because each adds build and test work.
How long does a Python version upgrade take for Airflow?
Duration depends on the factors that set the work: the target Python version, the Airflow release you run, how many images and environments exist, how many dependencies lack wheels for the new Python, and how much DAG code has custom dependencies to test. Airflow 3.3.2 lists Python 3.10 to 3.14 as supported prerequisites, so a move within that range has a defined target, but the test effort is driven by your dependency list. Ask for a schedule that separates inventory, image rebuild, staging test and production cutover, with each phase tied to a written deliverable.
What does our team need to provide for a Python upgrade?
The client team typically provides access to the repositories that build the Airflow images, the constraint and requirements files in use, a staging environment that mirrors production, a list of DAG owners who can confirm expected behavior, and access to the metadata database for a backup before any change. You also need someone who can approve the change window and the rollback approach. Confirm these inputs during scoping, because missing access or unclear DAG ownership is a common source of delay.
Can our internal team upgrade Python for Airflow without a partner?
Yes, many teams do this in-house, provided they have people who can rebuild images, read dependency resolution output and test DAGs in staging. The work is mostly disciplined testing rather than a single command. The main risks are installing Airflow without the matching constraints file, which Airflow's installation documentation warns can produce an unusable installation, and skipping tests for dependencies that lag behind the new Python. Decide based on who owns the images, whether anyone has done a Python upgrade on a scheduler, and how much staging capacity exists. A partner is useful when the team is short of time or experience, and the written scope makes the split of work explicit.
Should we upgrade Python and Airflow in the same change?
Separating the two changes is usually safer because each one can fail in different ways. A Python change affects the interpreter, compiled dependencies and images, while an Airflow change affects the scheduler, API server, DAG processing and removed features. If both change together, it becomes harder to tell which change caused a failure. If your Airflow version is moving to 3.x at the same time, check the upgrade guide first: the upgrade guide requires being on Airflow 2.7 or later and recommends moving to the latest 2.x first. Pair the two only when the combined test plan covers both sets of risks.
What happens if a dependency has no wheel for the target Python version?
The package may need to build from source, which requires compilers, system libraries and sometimes a newer version of the package, or it may be blocked until its maintainer publishes support. Your options are to upgrade the dependency to a release that supports the target Python, replace it, vendor a patched build, or delay the Python upgrade until the dependency is ready. Find these cases early by resolving your requirements against the target constraints file and reviewing the build output. Each blocked package should have a named owner and a decision recorded before the image rebuild starts.
Which Python versions does Airflow support, and does a newer version drop older ones?
The Airflow 3.3.2 prerequisites list Python 3.10 to 3.14 as supported. Airflow 3.1.0, released on September 25, 2025, added Python 3.13 and removed Python 3.9 support, and Airflow 3.2.0, released on April 7, 2026, added Python 3.14 support. Confirm the exact support matrix for your Airflow release in the prerequisites page before you pick a target, because a version that works on one Airflow line may not be listed on another.
Where the authoritative version and constraint details live
Before you pick a target Python version, confirm the supported range in the official Airflow prerequisites page for the exact Airflow release you plan to run. The prerequisites list the Python versions, database engines and Kubernetes versions that release supports. Use the version you are moving to, not the one in your current environment, and record the date you checked. Supported ranges change between releases, so a value copied from an older internal runbook can be wrong for the Airflow version you are deploying.
Next, read the release notes for every Airflow version between your current release and your target. They state which Python versions were added or dropped, and they flag other changes that will affect the same upgrade window, such as changes to the database layer or the API server. Check the Airflow release notes for the entries that mention Python, SQLAlchemy, container images or the Task SDK, and note which of those changes your DAGs, images or base operating system depend on. Confirm with your platform team which Python interpreter is installed in your worker and scheduler images, since that is what the release notes and prerequisites apply to.
Last, use the installation documentation on installing from PyPI to find the constraints file for your exact Airflow version and Python version. Pin your installs to that file rather than resolving the latest package versions, because a plain install of the core package can produce an unusable environment. Re-check the constraints URL during the upgrade itself, rebuild it for each Python version in your matrix, and keep the file under version control with the image build so that a later rebuild resolves the same dependency set.
How DeployOpen scopes and delivers an Airflow Python upgrade
DeployOpen plans a Python move for Airflow as a fixed-scope piece of work, starting from the Python versions your workers, schedulers and images run today. We check every dependency and DAG import against the target interpreter, then build, test and hand over the change inside infrastructure you control.
How we scope
- Inventory every Airflow component, worker image and scheduler host with its current Python version and install method.
- Confirm the target Python version against the Airflow prerequisites for your Airflow release before work starts.
- Agree in writing which providers, custom operators and private packages are in and out of the Python move.
How we build
- Rebuild container images and virtual environments using the constraints file that matches the target Airflow and Python versions.
- Run dependency resolution and DAG import checks on the new interpreter, then fix or pin packages that lag behind.
- Test representative DAG runs on staging workers before any production image or environment is switched over.
How we hand over
- Document the target Python version, constraints file reference, image tags and rollback steps for each component.
- Record every dependency that was pinned, replaced or removed, with the reason it could not move forward.
- Walk your platform and data engineering teams through the rebuild steps so they can repeat it for future releases.
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after the fixed-scope work is complete.
- Track provider and dependency releases that affect the Python versions your Airflow line supports.
- Confirm the support terms and response expectations in the written agreement before the retainer begins.
