Apache Airflow 3.2 to 3.3 upgrade

Apache Airflow 3.2 to 3.3 upgrade: why teams move and the supported path

A team running Airflow 3.2 in its own infrastructure should move to 3.3 when it can test the new line in staging and schedule a production window. Move because the 3.3 line adds features your pipelines may need, not because of a calendar date. The current patch release is 3.3.2. The 3.3 line adds a task and asset state store, pluggable retry policies, Java and Go Task SDKs and histogram timer metrics. Start by listing which of these your roadmap needs. Then confirm, against the project's version life cycle table, how long your current 3.2 patch line will receive fixes, since that date sets your deadline.

Airflow 3 is the supported line. Airflow 2 reached end of life on April 22, 2026, so it gets no fixes, and teams still on 2.x carry unpatched security exposure. The project is released under the Apache License 2.0 and has no paid edition, so your costs are your own: infrastructure, the engineering time for testing and cutover, and any implementation partner you bring in. Staging environments and DAG regression testing are one-time costs for this move. Patch testing and ongoing operation recur with each release. When you request a quote, ask who owns the staging environment, what test evidence is delivered, and how the rollback plan is documented.

Treat the move as a planned path through the 3.3.x patch releases, not a single jump. The 3.3.0 release arrived on July 6, 2026, 3.3.1 on August 12, 2026, and 3.3.2 on September 17, 2026. Each patch changes behavior that affects production, so read the notes for every release you pass through and record which patch each environment runs. Deploy to staging first, then move production to the same patch you tested. Confirm the Python, database and Kubernetes prerequisites for your chosen patch before building images. The full changes are listed in the Apache Airflow release notes.

Which 3.3 changes affect your DAGs, workers and clients

This table lists each Airflow 3.3 change named in the release notes, with the part of your environment it touches and the action to take before or during the upgrade. Read the rows against your own DAG code, worker images and API clients, since most changes only matter if your estate uses the feature or depends on the behavior. The main takeaway is that the pandas 3 and bearer credential changes can break working pipelines and clients even when you change no DAG code.

Change	Impact	Action to take
Task and asset state store (3.3.0)	Adds a new store for task and asset state. Existing DAGs keep running, but teams that plan to use it need the store available to schedulers and workers.	Confirm the metadata database and migration steps for 3.3.0 in staging, then decide which DAGs will use the store.
Pluggable retry policies (3.3.0)	Retry behavior can now be supplied through pluggable policies instead of only fixed retry settings.	Review existing retry settings and test any custom policy in staging before enabling it in production.
Java and Go Task SDKs (3.3.0)	Adds language SDKs for writing tasks outside Python. Python DAGs are unaffected unless you add non-Python tasks.	Only plan SDK work if you run Java or Go tasks. Check that worker images include the required runtime.
DAG bundle versions on clear, rerun and backfill (3.3.0)	Clear, rerun and backfill can now target a specific DAG bundle version, which changes which code a rerun executes.	Document which bundle version your recovery runbooks expect, and test a rerun of a recent run after the upgrade.
OpenTelemetry timer metrics as histograms (3.3.0)	Timer metrics are emitted as histograms, so dashboards and alerts built on the old metric shape may change.	Inventory timer metric queries and alert rules, then update them to read histogram data before cutover.
pandas 3 DataFrame XCom storage (3.3.1)	pandas 3 changes how DataFrame XComs are stored. Mixed versions can fail, and downgrading strands DataFrame XComs written under pandas 3.	Upgrade pandas 3 on schedulers, API server and all workers together. Do not plan a rollback path that assumes pandas 3 data can be read on older pandas.
Explicit bearer credential precedence (3.3.2)	An explicit bearer token now takes precedence over the session cookie. Invalid explicit credentials return 401 or 403 even when a valid cookie is present.	Audit scripts and integrations that send both a token and a cookie, rotate or fix invalid tokens, and retest them against the 3.3.2 API.

Why pandas 3 must reach every component together

The 3.3.1 release notes state that pandas 3 changes how DataFrame XComs are stored, and that is the reason this section comes before any cutover plan. Schedulers, the API server, the DAG processor and every worker should move to pandas 3 in the same change window. A component still on pandas 2 may read or write DataFrame XComs in a format another component does not expect, so a mixed fleet can fail in ways that look like task errors rather than version mismatches. Include workers that run in separate images, virtualenvs, Kubernetes pods or edge environments, because they are the components most often left behind.

Downgrading is the step to plan for most carefully. The release notes warn that a rollback strands DataFrame XComs already written under pandas 3, so a reader who returns to the earlier pandas version may not be able to load values that completed runs stored. Before cutover, inventory every DAG that passes a DataFrame through XCom, including tasks that push a DataFrame return value implicitly and downstream tasks that pull it. For each one, decide whether the value is needed after the upgrade window, whether it can be recomputed, and whether the retention of the metadata database covers the period you may need to roll back to. Record the decision so that an on-call engineer does not have to rediscover it during an incident.

Before the cutover, confirm the pandas version inside each image, virtualenv and worker pool, not only in the build file that should produce it. Check the scheduler, API server, DAG processor and each worker type with the same command your team uses to report package versions, and compare the results against the constraint file used for the Airflow 3.3.2 install. Check the Apache Airflow release notes for the 3.3.1 entry before you finalize the change plan, because the wording of compatibility notes can change in later patches. Treat any component that reports a different pandas version as a blocker for the cutover.

How the upgrade differs for pip, container and Helm installs

On virtual machines, the upgrade is a package change on every host that runs an Airflow component. The constraint file for the 3.3.x release decides which provider and dependency versions install, so use it on every host to keep the environments identical.

Constraint file: Install Airflow 3.3.x with the constraint file that matches your Python version, so dependencies resolve the same way on each host.
Stop before you swap: Stop the scheduler, DAG processor, API server, triggerer and workers before changing packages, then run the database migration once the new version is installed.
Pandas 3 alignment: If DAGs pass DataFrames through XComs, install pandas 3 on every component, including workers, in the same change window. The release notes warn that downgrading strands DataFrame XComs written under pandas 3.
Python and database versions: Confirm each host runs Python 3.10 to 3.14 and that your metadata database is PostgreSQL 14 to 18 or MySQL 8.0, 8.4 or an Innovation release, as listed in the prerequisites.

Before you start, check that every host's Python and database versions match the prerequisites for the target 3.3.x release.

What to back up and inventory before touching the 3.2 deployment

Six preparation items sit between a running Airflow 3.2 deployment and a safe move to 3.3. Each one protects a different thing: the metadata that records run history, the code that defines pipelines, the extensions that may break, the clients that authenticate, the XCom data that depends on pandas, and the environment where the upgrade gets rehearsed first.

Consistent metadata database backup

Take a backup of the metadata database while the scheduler and workers are paused or drained, so task state, DAG runs and the new state store data are captured together. Airflow recommends PostgreSQL or MySQL for production, because SQLite is for testing only and running it in production can lead to data loss. Confirm that you can restore the backup into a scratch database and that the restored instance starts. Test the restore before the upgrade, not after.

Copy of DAG bundles and configuration

Record the exact DAG bundle source revisions, the deployed airflow.cfg or environment variables, and the Helm values or Compose files that define each component. Store them outside the running cluster so you can rebuild the 3.2 environment if the upgrade fails. Note which DAG bundle versions are in use, since 3.3.0 adds bundle versions on clear, rerun and backfill and you will want to know what history references.

Inventory of plugins, providers and operators

List every custom plugin, installed provider package and in-house operator, with its version and the Airflow version it was built against. Check each one against the 3.3 release notes and the provider documentation before the upgrade, and confirm that its dependencies install in your image. Plugins that depend on removed or changed internals are the most common source of failed scheduler or API server starts, so treat this list as the scope of your regression tests.

REST API clients using bearer or cookie credentials

Identify every script, CI job and integration that calls the REST API, and record whether it sends a bearer token, a session cookie, or both. In 3.3.2 an explicit bearer credential takes precedence over the session cookie, and an invalid explicit credential fails with 401 or 403 even when a valid cookie is present. Clients that send both, or that rely on a stale token while a browser session is active, need to be found and fixed before you move to 3.3.2.

Pandas-based DAGs that use XCom

Find DAGs that return or pass DataFrames through XCom, and record the pandas version used by the scheduler, API server, workers and any task images. Airflow 3.3.1 notes that pandas 3 changes how DataFrame XComs are stored, and that downgrading strands DataFrame XComs written under pandas 3. Plan one pandas version across all components, upgrade them together, and decide how in-flight DataFrame XComs will be drained or rewritten before you start.

Staging environment built from production-like data

Build a staging environment that matches production in Python version, database engine, executor, worker images and pandas version, and load a recent copy of the metadata database that has been anonymized where DAG run data carries personal or regulated information. Run representative DAGs, API clients and plugin loads against it before any production change. Ask your team to confirm that the staging copy is refreshed close to the cutover date, so the rehearsal reflects current state.

Upgrade sequence with checks and rollback points

This sequence moves a self-hosted Airflow 3.2 environment to 3.3 in an order that keeps a clean rollback path until the first DataFrame XCom is written under pandas 3. Each step closes with a check you can record as evidence before the next step starts.

Freeze DAG deploys and schedule changes
Pause DAG deployments, manual schedule edits, and variable or connection changes for the upgrade window. Confirm with your change owner that the freeze covers CI/CD pipelines, GitOps syncs, and any team that deploys DAG bundles directly. Record the Airflow version and DAG bundle version running on each component so the rollback target is known. Done when the deployment pipeline shows no pending syncs and the version inventory is saved to the change record. Rollback point: lifting the freeze is the only action needed here, because no environment has changed yet.
Back up the metadata database and DAG code
Take a consistent backup of the metadata database using a method your database team has tested, and archive the DAG bundles, Helm values, and deployment manifests in version control. Export the installed Python package list from the running workers, including the pandas version. Restore the backup into a scratch database and confirm that DAG run history and task states query correctly. Done when the restore test passes and the backup location is written into the change record. Rollback point: this backup is the only route back to the 3.2 state, so keep it until the upgrade is accepted.
Upgrade staging with production-like data
Build a staging environment from the same images, constraint files, and configuration used in production, and load a recent copy of the metadata database. Upgrade staging to the 3.3.2 patch release, the current patch in the version table, and install pandas at the major version you intend to run in production. Trigger representative DAGs, including tasks that return DataFrames through XCom and tasks that call the REST API. Done when those runs complete and the team has a signed-off list of any remaining failures. Rollback point: staging can be rebuilt from the backup taken in the previous step without touching production.
Run the database migration
Stop schedulers, DAG processors, API servers, and workers so that no component writes to the metadata database during the migration. With the 3.3 release installed, run airflow db migrate against the database and read the output for errors. Confirm that the database reports the target Airflow version and that a read query against DAG and task instance tables succeeds. Done when the migration completes without errors and the check passes in both staging and production. Rollback point: if the migration fails, restore the database from the backup taken in step two and keep all components on 3.2 until the cause is fixed.
Upgrade schedulers, DAG processors, API servers, and workers together
Roll out 3.3.2 to the scheduler, the standalone DAG processor, the API server, and every worker in one change window, with the same pandas version on each component. Upgrade every scheduler instance in the same window, since Airflow supports more than one running at a time. Check that each component reports 3.3.2 and the same pandas version, that scheduler heartbeats are current, and that API health checks pass. Done when all components run the target versions and no task fails on import. Rollback point: until the first DataFrame XCom is written under pandas 3, you can return to 3.2 and restore the database backup. After that point, downgrading strands DataFrame XComs written under pandas 3, and those XComs cannot be recovered by downgrading.
Validate run outcomes and DataFrame XComs
Compare outcomes for a sample of production DAGs across several scheduled cycles against the pre-upgrade baseline saved in step two. Read a set of DataFrame XComs in a downstream task and in an external consumer to confirm they deserialize with the expected columns and types. For DAGs that use the task and asset state store, check that state is written and read as expected, and review retries and logs for new errors. Done when outcomes match the baseline and the XCom sample reads correctly. Rollback point: a failed XCom read is a forward fix, since downgrading cannot recover DataFrame XComs already written under pandas 3. Confirm with your team which affected tasks can be rerun safely.
Move API clients to 3.3.2 credential handling
Review scripts, CI jobs, and integrations that call the REST API, especially any that send both a bearer token and a session cookie. In 3.3.2, an explicit bearer credential takes precedence over the session cookie, and an invalid explicit credential fails with 401 or 403 even when a valid cookie is present. Rotate or fix stale tokens before the change, then test each client against staging. Done when each client returns the expected responses and the list of clients and their owners is recorded. Rollback point: you can revert a client to its previous credential configuration, but invalid tokens will still fail even when a cookie is present.

What makes a 3.2 to 3.3 upgrade take longer

This table lists the factors that most often extend a move from Airflow 3.2 to 3.3, along with what each one changes and how to reduce it. Use it to see which drivers apply to your estate, then ask for a written scope that sets timing against those factors rather than a generic estimate.

Driver	Why it adds effort	How to keep it down
DAGs that write DataFrame XComs	Under pandas 3, DataFrame XCom storage changes, and downgrading strands data already written, so each affected DAG needs review and test runs.	Inventory DataFrame XCom use first, then upgrade pandas on all components together.
Custom plugins and operators	Plugins and custom operators must be checked against the 3.3 changes, including any that depend on retry handling or metrics names.	Keep plugins in a versioned repository with automated import and smoke tests.
Custom executors and install methods	Custom executors, images, and PyPI constraint setups each need their own rebuild and validation path.	Document the install method and pin constraint files before starting.
REST API clients and scripts	Scripts and CI jobs that call the API need to be confirmed against the 3.3.2 bearer token precedence change, which can turn valid cookie-based calls into 401 or 403 errors.	List every client, its credential type, and its owner, then test each one against a staging API server.
Metadata database size and cleanup	Large history tables lengthen migration and backup windows, and cleanup has to be finished before the migration runs.	Run cleanup and a verified backup in staging, then time the migration with a copy of production data.
Number of environments	Each environment (development, staging, production) is upgraded and tested in sequence, and differences between them multiply the checks.	Keep environments on the same versions and configuration until the final step.
Task and asset state store adoption	Teams that choose to use the new state store or pluggable retry policies add design, testing, and rollout work beyond the bare upgrade.	Decide early whether these features are in scope, and keep them out of the upgrade change if they are not.
Approval and change-control steps	Change boards, security reviews, and maintenance windows set the calendar, often more than the technical work itself.	Submit the change request early with the rollback plan and test evidence attached.

Questions teams ask before upgrading Airflow 3.2 to 3.3

What does an Airflow 3.2 to 3.3 upgrade cost?

The cost depends on the scope agreed in writing, not on the version number alone. Most of the fee covers assessing your DAG code, metadata database, executors, Helm or container build, and identity setup, then testing and cutover. Ask for a quote that separates one-time work (assessment, staging rebuild, cutover, rollback plan, documentation) from any recurring support or retainer. Costs rise with the number of environments, the size of the DAG estate, the use of DataFrame XComs, custom plugins, and any custom Task SDK code. DeployOpen does not publish a price list, so a quote follows from the written scope.

How long does the upgrade take?

Duration depends on scope: how many deployments, how many DAGs and providers you run, how complete your staging environment is, and how many change windows your organization allows. Teams that already run checks and tests in CI usually shorten the work, while undocumented plugins or hand-rolled scripts that read the metadata database extend it. Ask for a timeline in the written scope with explicit milestones for staging, the pandas 3 rollout, and production cutover. No fixed duration is stated here because none is supplied for this release.

What do we need to provide during the upgrade?

The client team usually provides access to non-production and production environments, the current deployment definitions, a copy of the DAG repository, a list of connections and secrets backends, and a named owner who can approve change windows. You also need someone who knows which DAGs are business critical and who can validate outputs after cutover. Ask the partner to list these inputs in the scope so that delays are visible early.

Can our own team do this upgrade without a partner?

Yes, if your team has Airflow operators, a staging environment that mirrors production, and time to test. The upgrade is a version change with specific behavior changes to check, not a rewrite. Teams without recent Airflow 3 experience often underestimate the pandas 3 coordination and the scripts that rely on tokens. A partner is useful when capacity or evidence requirements are tight. Either way, confirm the target release and check the official release notes before you start.

Is it safe to downgrade after pandas 3 has written DataFrame XComs?

No, you should not plan on a simple downgrade. Airflow 3.3.1 notes that pandas 3 changes how DataFrame XComs are stored, that pandas 3 should reach all components, including workers, together, and that downgrading strands DataFrame XComs written under pandas 3. Plan the pandas 3 change as a one-way step for those XComs, decide in advance how you will handle in-flight runs, and rehearse the rollback path in staging. Before cutover, identify DAGs that pass DataFrames through XComs and confirm the pandas version on every worker.

How does the bearer token change in 3.3.2 affect our scripts?

Scripts and integrations that send an explicit bearer credential will now use that credential even when a browser session cookie is also present. Airflow 3.3.2 states that invalid explicit credentials fail with 401 or 403 even when a valid cookie exists. Scripts that depended on the cookie masking a stale token will start failing, which is usually a good signal to fix them. Inventory every CI job, cron script, and client that calls the REST API, rotate or refresh tokens, and test each one against staging before production.

What does a fixed-scope engagement include?

A fixed-scope engagement is agreed in writing before work starts and sets out the scope, the deliverables, and the timeline. For an Airflow upgrade, expect the written scope to name the environments in scope, the components to upgrade, the checks to run, the cutover approach, the rollback plan, and the documentation handed over. Ask what is excluded, such as DAG rewrites, new providers, or unrelated plugin work, and how changes to scope are priced. Ongoing support and upgrades after cutover are offered as separate retainers.

What your team provides and what DeployOpen delivers

Your team provides the inputs that decide whether a 3.2 to 3.3 upgrade can be planned accurately. The first is access: staging and production environments with the Airflow version, the metadata database, the DAG bundle source and the Helm release or container images in use, plus read access to the scheduler, DAG processor, API server and worker logs. The second is a DAG inventory that lists each DAG repository, its owners, the operators and custom code it calls, and whether any task still uses the metadata database directly. You also need to name who owns database backups and restore testing, and who can authorize the cutover window. Confirm early whether your workers run pandas, because the 3.3.1 DataFrame XCom change depends on which components receive pandas 3 together. Confirm whether any client still sends explicit bearer tokens, since 3.3.2 changes how they are evaluated.

In return, DeployOpen delivers a written upgrade plan covering scope, component order across the DAG processor, scheduler, API server and workers, pre-checks, the rollback path and the decisions that remain open, such as whether to adopt the new state store, retry policies or the Java and Go Task SDKs in this window. The plan is accompanied by a tested runbook with the exact commands, configuration changes and validation steps for each environment, and a staging rehearsal recorded against the same steps your production change will use. Validation evidence includes the rehearsal results, DAG parse and run outcomes, authentication checks for API clients and the post-cutover state of metrics. Scope, deliverables and timelines are agreed in writing before work starts. Your team keeps ownership of production changes and sign-off, and DeployOpen does not issue audit reports or certifications for the upgrade.

How DeployOpen scopes, delivers and supports the upgrade

DeployOpen plans the move from Apache Airflow 3.2 to 3.3 as a written scope with named checks for your environment, then builds and tests it in infrastructure you control. Engagements are fixed-scope, so the work, deliverables and timeline are agreed before any change starts.

How we scope

Inventory your current 3.2 version, executors, metadata database and Helm or container deployment before agreeing the scope.
Identify which DAGs use retry logic, DataFrame XComs or the Java and Go Task SDKs so the affected work is listed up front.
Record in the written scope which 3.3.x patch level you will target, including the 3.3.1 and 3.3.2 changes.

How we build

Rehearse the upgrade in a staging environment that matches production executors, database and bearer token clients.
Coordinate pandas 3 rollout so the scheduler, API server, DAG processor and all workers change together.
Test bearer token clients against the 3.3.2 credential precedence rule before cutover, and keep a tested rollback plan.

How we hand over

Deliver a runbook covering the upgrade sequence, the validation steps run in staging and the production cutover checklist.
Document any changes to retry policies, timer metrics dashboards and state store usage in your environment.
Confirm with your team the knowledge transfer sessions and the handover acceptance criteria written into the scope.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after the handover.
Retainers can cover later Airflow 3.3.x patch reviews, provider compatibility checks and upgrade planning for the next release line.
Ask for the support terms, response expectations and any SLA in writing with your quote, since these are set per engagement.

Apache Airflow 3.2 to 3.3 upgrade