Moving from Astronomer Astro to self-hosted Apache Airflow

Moving DAGs from Astro to community Airflow is mostly a packaging and operations change: the DAG logic usually carries over, while image builds, Astronomer-specific packages, secrets, logging and the scheduler stack move to your team. Effort depends on how much Astro-specific code and platform tooling your DAGs depend on.

On this page

Moving from Astronomer Astro to self-hosted Apache Airflow starts with your Runtime version

Most of the DAG logic you run on Astro will carry over unchanged to community Apache Airflow. The work concentrates in three places: packaging, dependencies and operations. The first decision is therefore not about DAG code at all. It is which Airflow release sits under the Astro Runtime line your deployments use, because that release sets the target version, the Python and database prerequisites and the upgrade path you will follow on your own infrastructure.

Astro Runtime is a Docker image that Astronomer builds and publishes on top of Apache Airflow. Each Runtime line bundles a specific Airflow release, so the mapping is the first thing to write down. According to Astronomer's release notes, the Runtime 3.0 to 3.3 lines bundle Airflow 3.0.x to 3.3.x, for example Runtime 3.3-9 with Airflow 3.3.2. Pull the Runtime tag from each Astro deployment, look up its Airflow version in the release notes, and record the result beside the deployment name. Any Runtime 2.x line maps to Airflow 2.x, which needs a separate plan.

Airflow 3 is the current major line. Airflow 2 entered limited maintenance on October 22, 2025 and reached end of life on April 22, 2026, after which it receives no fixes. A team leaving Astro on an Airflow 2 Runtime is therefore also making a major upgrade, and the Airflow 3 upgrade guide requires starting from Airflow 2.7 or later. Confirm your current Runtime and Airflow versions before you size the move. The effort drivers section explains how the version gap, dependency count and operational scope change the work.

What maps cleanly from Astro to community Airflow and what does not

This table compares each part of an Astro setup with its community Airflow equivalent and notes what needs rework. Read it as a checklist: rows marked to confirm depend on Astronomer-specific behavior that you should verify in Astronomer's documentation and your own deployment before assuming a mapping.

Astro capabilityCommunity Airflow equivalentMigration notes
DAG files and task logicDAG files run under Airflow 3 with the Task SDKTask code cannot use the metadata database directly; move any session or model access to the REST API or Task SDK
Python dependenciesPinned requirements installed into the official Docker imageRecreate the Astro requirements set and confirm each package matches the Airflow 3.x constraints for your release
Astro Runtime imageOfficial Apache Airflow Docker image on a matching 3.x releaseRuntime 3.3-9 bundles Airflow 3.3.2; confirm the mapping for your Runtime line before choosing the base tag
Astronomer-specific packages and operatorsProvider packages from the Airflow ecosystemConfirm each Astronomer package in your DAGs; replace it with a community provider or task code, since no mapping is assumed
Airflow connections and variablesConnections and variables in the metadata database or a secrets backendExport from Astro and re-enter or load them; confirm which secrets backend your team will use
Environment variablesContainer environment variables and Helm chart valuesMove Astro deployment settings into your image build and chart values; confirm any Astro-only variables
DAG deliveryDAG bundles or images delivered by your CI pipelineAstro deploy workflows do not carry over; define how DAG code reaches the DAG processor and workers
Executor choiceExecutors such as CeleryExecutor or KubernetesExecutorConfirm the Astro executor and worker sizing, then select one you can operate; hybrid executors are not supported in Airflow 3
Logging and monitoringTask logs and metrics from your own stackConfirm Astro log retention and alerting, then configure log shipping and metrics collection on your side
Deployment management and alertingYour platform team's tooling, such as Helm releases and alert rulesNo direct equivalent is assumed; the platform team must own upgrades, scaling, backups and alert routing

Mapping Astro Runtime versions to Airflow releases before writing any code

Start by fixing the target Airflow version, because every later choice (base image, Python version, database, Kubernetes cluster) depends on it. Open the Astro Runtime release notes and find the Runtime line your DAGs were last tested on. The documented range is Runtime 3.0 to 3.3, which bundles Airflow 3.0.x to 3.3.x, so a Runtime 3.3 build such as 3.3-9 corresponds to Airflow 3.3.2. Record the bundled Airflow version for each Astro deployment in your inventory, and check the Runtime build deployed in each environment, since production and development may not match. Astronomer does not provide an export tool that performs this mapping for you, so the team must build the table by hand from the release notes and the deployment settings. Match the Airflow release to the Runtime line the DAGs were tested on, then move to the current patch release on the official line, which is 3.3.2 for Airflow 3 at the time of writing. Before choosing a base image, read the Airflow 3.3.2 prerequisites and confirm that your Python version falls in the supported range (3.10 to 3.14), that your metadata database is PostgreSQL 14 to 18 or a supported MySQL release, and that your Kubernetes version is within 1.30 to 1.35. MariaDB is not supported as a metadata database, and SQLite should not be used in production. If any of these fail against your current platform, resolve that first, because it changes the infrastructure you must build. DAGs still running on Airflow 2 need a separate path. The Airflow 3 upgrade guide requires being on Airflow 2.7 or later before upgrading, and recommends moving to the latest 2.x release first. Run the Ruff AIR301 and AIR302 rules over the DAG code to flag constructs removed or changed in Airflow 3, and treat the findings as a backlog to fix before the cutover. Confirm with your own testing that the DAGs behave the same after each step, since the linter does not cover runtime behavior.

Documented paths for moving DAG code, images and checks onto Airflow

The Apache Airflow project does not document a vendor-to-Airflow import tool, so the migration work is assembled from upgrade guidance, static checks, the Task SDK, the official image and chart, and the production guide. Each card below covers one of these paths and what your platform team should verify before rebuilding DAGs from Astro Runtime.

Airflow 3 upgrade guide and checklist

The upgrade guide is the project's record of what changed between Airflow 2 and Airflow 3, including the requirement to be on Airflow 2.7 or later before upgrading. For a team leaving Astro, it is the baseline for deciding which Runtime-specific behavior must be reproduced and which Airflow 2 patterns need rework. Read the checklist against your current Runtime version, then list every DAG and plugin it touches. Confirm the Airflow release your Runtime maps to before you start reading.

Ruff AIR301 and AIR302 rules

The upgrade guide recommends checking DAG code with the Ruff AIR301 and AIR302 rule sets, which flag code that uses removed or changed Airflow features. Running them across the whole DAG repository gives your team a list of findings to triage, rather than relying on memory of each Runtime upgrade. Expect the findings to include imports from Astronomer-specific packages, which need a replacement decision rather than a simple rename. Record each finding and its owner in the migration tracker.

Task SDK and REST API

In Airflow 3, task code can no longer use Airflow database sessions or models directly. Tasks must use the REST API or the Task SDK instead, so any DAG that queried the metadata database through Astro-era helpers needs a redesign before it runs on community Airflow. Review each task for direct database access, then decide whether the data should come from the API, a provider hook or an external system. This change affects the code, not only the deployment.

Official Docker image as the rebuild base

The community Docker image is the documented base for a rebuilt image that replaces Astro Runtime. Your Dockerfile should pin the Airflow version, install only the provider packages your DAGs need, and add your own internal certificates or tooling on top. Astro Runtime images carry extra components that you will need to identify and replace yourself. Confirm the Python version the image ships against the prerequisites before you build.

Official Helm chart for Kubernetes

The community Helm chart uses the official Docker image and is the project's packaged route for running Airflow on Kubernetes. It replaces the deployment that Astronomer manages for you, so your team owns values files, secrets references, executor settings and upgrades. Review the chart values against the components you will run, including the scheduler, DAG processor and API server that Airflow 3 requires as separate pieces. Confirm which chart version matches the Airflow release you chose.

Production deployment guide

The production deployment guide sets out the choices your platform team now owns: a PostgreSQL or MySQL metadata database instead of SQLite, container images on Kubernetes, and secrets handling. It is the checklist to compare against your current Astro configuration, which the platform previously supplied. Ask Astronomer whether it offers an export of DAG bundles or environment configuration, and what that export contains, including connections, variables and environment variables. Treat any export as input to review, not a finished configuration.

The sequence for a parallel run and cutover from Astro

This sequence moves DAGs off Astro one deployment at a time, with an inventory, a rebuilt image, a pilot, a guarded parallel run and a rollback path, so each stage has a recorded exit check before the next one starts. The output is a retired Astro deployment with evidence that the self-hosted Airflow produced the same results.

  1. Inventory every deployment and its dependencies

    The platform team lists each Astro deployment, its Runtime version, DAG repository and deploy method, installed packages and requirements, Airflow connections, variables, environment variables and the secrets each one reads. Include who owns each deployment and which downstream systems the DAGs write to. Owners of each DAG group confirm the list for their pipelines. Exit check: a single inventory sheet where every DAG, package, connection, variable and secret has a named owner and a target location in the new environment, signed off by the data platform lead and each team owner.

  2. Decide the target Airflow version and base image

    Map each Astro Runtime line in use to its bundled Airflow release using the Astronomer release notes, then choose one target Airflow version for the self-hosted estate. Check the chosen release against the Airflow prerequisites for Python, metadata database and Kubernetes versions, and confirm DAG code against the Airflow 3 upgrade rules if any DAGs came from Airflow 2. Exit check: a written version decision that names the base image tag, Python version and metadata database, approved by the platform architect and the security owner.

  3. Rebuild the image on the official base with pinned constraints

    Create a new image from the official Apache Airflow image rather than the Astro Runtime image. Replace Astronomer-specific packages with community providers or their own code, and install remaining dependencies against a pinned constraints file for the chosen Airflow version. Store the Dockerfile and constraints in version control with a build pipeline that tags every image. Exit check: the image builds from a clean checkout, a dependency report shows no Astronomer-only packages, and the engineer who built it and the platform owner have both reviewed the diff.

  4. Stand up a pilot with the metadata database and secrets backend

    Deploy a pilot environment in infrastructure you control, with a PostgreSQL or MySQL metadata database, separate scheduler, DAG processor and API server components, and the secrets backend your organization has chosen. Load the connections and variables from the inventory through a controlled process, not by copying values into chat or tickets. Exit check: the pilot starts cleanly, health checks pass, a test DAG runs end to end, and the security owner confirms that no secret values are visible in the UI or logs.

  5. Move low-risk DAGs and compare run outcomes

    Start with DAGs that have no external side effects or that write to a sandbox target, and run them on both Astro and the pilot for the same logical dates. Compare task states, row counts, output files, retry behavior and alert delivery. Record every difference and classify it as expected or defect. Exit check: a comparison log for each migrated DAG where every difference is explained and fixed or accepted by the data owner, signed off by the team that owns the pipeline.

  6. Run both systems in parallel with writes guarded

    Run the production DAGs on both platforms while only one is allowed to write to production targets. Guard writes with a feature flag, a variable that disables side-effecting tasks on the non-authoritative system, or a separate target schema. Confirm the scheduler is running as intended on the self-hosted side and that the scheduling rules match the Astro schedules. Exit check: a daily report showing no duplicate writes to production targets across the agreed parallel period, reviewed by the data platform lead and the owners of the downstream systems.

  7. Cut over one deployment at a time with a rollback path

    Move each Astro deployment separately. Before switching, pause its DAGs on Astro, enable the matching DAGs on the self-hosted environment, and confirm the first scheduled run completes. Keep the Astro deployment paused rather than deleted so that the rollback is to re-enable it and disable the self-hosted DAGs. Document the rollback steps and the person authorized to trigger them. Exit check: the first production run after cutover succeeds, downstream owners confirm their data, and the change record is approved by the service owner.

  8. Decommission Astro after the agreed retention period

    After the retention period agreed with the business and compliance owners, remove the Astro deployments, revoke their API tokens and deploy keys, and delete any stored secrets copied from Astro. Archive the inventory, comparison logs and cutover records as evidence. Confirm the retention period in writing with your compliance team, since it depends on your own obligations. Exit check: an Astro account summary showing no active deployments, a revocation record, and an archive signed off by the platform owner and compliance lead.

What the platform team must now run itself on Airflow

On Astro, Astronomer ran the Airflow components on your behalf and your team mostly worked with DAGs. On self-hosted Airflow 3, each process is yours to deploy, configure, restart and monitor. The split changed in Airflow 3, so the component list is different from the one you may know from Airflow 2.

  • API server: started with airflow api-server, it replaces the webserver from Airflow 2 and serves the UI and API.
  • DAG processor: it must run as a separate component in Airflow 3, so it needs its own deployment, health checks and restart policy.
  • Scheduler: you can run more than one scheduler at the same time, and they coordinate through database row-level locks, so the database must be sized for that load.
  • Workers and executor: choose among LocalExecutor, CeleryExecutor and KubernetesExecutor. The hybrid CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported as of Airflow 3.0.0, so any DAG or deployment design that assumed them needs a new executor choice.

Confirm which executor your team will operate, who owns each process, and how each one is restarted and scaled before the design is frozen.

What drives the cost and duration of the move

This table lists the factors that change how much work the move takes, what each one means for an Astro to self-hosted Airflow project, and what to ask for in a written quote. Read it as a checklist: the more of these drivers that apply to your estate, the more the scope should be broken into named phases. One-time build and cutover work is separate from the recurring patching, upgrades and on-call the platform team takes on afterward.

DriverWhy it mattersWhat to ask for in a quote
Number and complexity of DAGsEach DAG needs review, rebuild on the official image and a test run. Dynamic DAG generation and cross-DAG dependencies add more analysis per DAG.A DAG inventory with counts by complexity and a list of dynamically generated DAGs called out separately
Astronomer-specific code and packagesCode that imports Astronomer-only packages or relies on Astro features must be replaced before the image builds. This is often the largest unknown.A dependency scan listing each Astronomer-specific import, package and feature, with the planned replacement for each
Runtime version gap to Airflow 3Runtime 3.x lines bundle Airflow 3.0.x to 3.3.x. A DAG still written for Airflow 2 needs upgrade work even before the move. The Airflow 3 upgrade guide requires Airflow 2.7 or later before upgrading.The current Runtime and Airflow versions in use, and the list of DAGs that still need Airflow 3 changes
Airflow 2 features that changedSubDAGs were removed in favor of TaskGroups. Task code can no longer use database sessions or models directly. Plugins and the auth setup also change.A list of affected features, the Ruff AIR301 and AIR302 findings, and the rework each one needs
Number of deployments and environmentsEach Astro deployment becomes a separate Airflow installation with its own database, secrets, connections and release process to build.The count of deployments and environments, and whether they share one installation or stay separate
Executor and scale targetsChoosing LocalExecutor, CeleryExecutor or KubernetesExecutor, and sizing workers, changes the infrastructure and the tuning effort. Multiple schedulers add database locking considerations.The target executor, peak concurrent task counts, and the sizing assumptions behind the worker design
Secrets and SSO designAstro-managed environment variables and connections must be moved to a secrets backend you run. Identity integration is planned separately for the Airflow 3 auth manager.Where secrets will live, how connections are migrated, and which identity provider and auth manager the scope covers
Kubernetes or VM target and existing platformAn existing cluster, database and image registry shorten the build. A new Kubernetes control plane adds etcd, upgrade and hardening work. Supported Kubernetes versions are listed in the Airflow prerequisites.A statement of what already exists (cluster, PostgreSQL, registry, CI) and what must be built or supplied
Logging and monitoring integrationAstro-hosted logs and metrics must be replaced with your own log store, alerting and dashboards before on-call can rely on the new platform.The target log and metrics stack, alert routes, and the dashboards and runbooks included in the scope
Length of the parallel runRunning Astro and self-hosted Airflow side by side to compare results lengthens the project and doubles some infrastructure cost while it lasts.The parallel-run period, the DAGs included in it, and the comparison criteria that close it
Retained support after cutoverPatching, upgrades, on-call and retainers recur after the move. Community Airflow 2 is at end of life, so upgrade planning is ongoing work.Which support tiers are included, who pays for the recurring work, and the upgrade cadence agreed in writing

What usually goes wrong in an Astro exit and how to prevent it

Most failed Astro exits do not fail on the first run. They fail in the gaps between Astronomer-managed behavior and the self-hosted setup the platform team now owns. The cards below cover the risks that appear most often, each with a check to run before cutover.

Double runs during parallel operation

While the Astro deployment and the self-hosted Airflow both run the same DAGs, every schedule fires twice unless one side is paused. Airflow supports more than one scheduler at once using database row-level locks, but that protects the scheduler's own work, not your business logic. Keep each DAG active in only one environment at a time, and record the pause and unpause order for every DAG in the cutover runbook.

Unsupported metadata database in production

Teams often carry over a local or test database when they stand up the first self-hosted environment. The Airflow production guide warns that SQLite is for testing only and can lose data in production, and the prerequisites say MariaDB is not supported as a metadata database. Provision PostgreSQL or MySQL from the start, and confirm the version against the prerequisites list before building the environment.

Missing Astro-managed connections and secrets

Astro stores connections, variables and environment values in its own control plane, so DAGs may depend on values nobody wrote down. These gaps usually surface only when a task fails at 2 a.m. Build a full inventory of connections, variables and environment values from the Astro environment, then load each one into your secrets backend and confirm that every DAG can resolve its dependencies in staging.

Image drift from the tested Runtime behavior

Astro Runtime is an image that extends Apache Airflow, so a DAG tested on a given Runtime line may rely on bundled packages or defaults that an official base image does not include. Map each Runtime version to its Airflow release, such as Runtime 3.3-9 with Airflow 3.3.2, and rebuild on the official base with pinned provider and Python packages. Run the same test suite against both images before cutover.

Airflow 2 code that fails on Airflow 3

Airflow 3 blocks task code from using database sessions or models directly and removes SubDAGs in favor of TaskGroups. DAGs that passed on Airflow 2 can fail or behave differently after the move. Run Ruff with the AIR301 and AIR302 rules over the DAG repository, replace direct metadata access with the REST API or Task SDK, and convert every SubDAG before the first Airflow 3 run.

Under-resourced scheduling and parsing

On Astro, the platform handles scheduler and parser capacity. Once your team owns it, a DAG bundle that grows over time can starve the DAG processor and delay every schedule. Airflow 3 requires the DAG processor to run as its own component, so size it separately from the scheduler and API server. Measure parse time and scheduling lag during parallel operation, and set a capacity target before you retire the Astro deployment.

Questions buyers ask before moving DAGs off Astro

How much does moving DAGs off Astro cost?

The cost is set by the written scope, not by a list price, because DeployOpen does not publish one. The main cost components are the discovery and inventory of DAGs, connections and variables, the rebuild of images on the official base, the work to replace Astronomer-specific packages, the new platform build (database, executors, secrets, DAG delivery) and testing and cutover. Some costs are one-time, such as the migration itself, while the platform costs recur through infrastructure and the internal team's ongoing operation. Ask for a quote that separates these items and states which DAGs and integrations are in scope, and which Airflow version and executor model are assumed.

How long does a move from Astro to self-hosted Airflow take?

The timeline depends on the number of DAGs, how many use Astronomer-specific packages, how many connections and secrets must be re-created, the executor and high availability design you choose, and how many environments need a parallel run before cutover. No fixed duration applies to every estate, so treat any generic figure with caution. Your Runtime version matters too: Astro Runtime 3.0 to 3.3 lines bundle Airflow 3.0.x to 3.3.x, so mapping your current Runtime to an Airflow release is an early step that shapes the work. Ask for a milestone plan in the written scope.

What does the client team need to provide during the migration?

Expect to provide access to the current Astro deployments and their DAG repositories, a list of connections, variables and secrets (without exposing values in tickets), the owners of each DAG and its downstream consumers, and the target infrastructure accounts or clusters. Your team also needs to decide who approves cutover and who can answer questions about business logic in each pipeline. Early involvement from the data platform owners and the security team usually prevents rework on identity, secrets and network paths.

Can a capable internal team migrate off Astro without a partner?

Yes, many teams can, especially when they already run Kubernetes or VMs, know Python and have a clear DAG inventory. The difficult parts are usually not the DAG code but the platform: a production metadata database (SQLite is for testing only), the Airflow 3 components that run separately, secrets handling, DAG delivery, logging and recovery. Be honest about capacity. A partner can take a fixed-scope piece, such as the image rebuild or the platform build, while your team keeps ownership of the DAGs.

Do DAGs have to be rewritten when leaving Astro?

Often not entirely, but the code needs review. DAGs built on Astro Runtime generally run on the community Airflow release that the Runtime bundles, so the core logic usually carries over. Changes come from Astronomer-specific packages that must be replaced, from Airflow 3 rules such as the removal of direct database session access from task code, and from SubDAGs, which are replaced by TaskGroups. Ruff rules AIR301 and AIR302 can flag many of these items before you run anything.

Can Astro run alongside self-hosted Airflow during the transition?

Yes, a parallel run is a common way to reduce cutover risk, but it must be designed deliberately. Each DAG should have one owner and one active schedule at a time, so the two environments do not both trigger the same work. Confirm which source systems, file locations and downstream tables each environment writes to, and plan a cutover order by DAG group. Ask the vendor and partner to confirm how the two environments will be kept from double-running pipelines.

What does the platform team take on after cutover?

After cutover, your platform team owns what Astronomer previously operated: upgrades to new Airflow patch and minor releases, scheduler and worker capacity, database backups and cleanup, secrets and identity integration, image builds, monitoring and on-call response. Kubernetes or VM hosts also need their own patching and support cycles, which you should track against the Kubernetes release and support lifecycle. Ask for a written handover covering runbooks, ownership of each component and the support option you want after go-live.

How DeployOpen works on this

DeployOpen plans, implements, customizes, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, so the Airflow you move to runs in your cloud accounts, data centers or air-gapped network. We start with a written scope that maps your Astro Runtime versions, DAGs, connections and secrets to the community Airflow you will run.

How we scope

  • Inventory every Astro Runtime version, deployment, DAG folder and Astronomer-specific package in use today.
  • Map each Runtime line to its bundled Airflow release, then agree the target Airflow version and Python and database prerequisites in writing.
  • Agree in writing which DAGs move, which connections and secrets are re-created, and how a parallel run and cutover will be verified.

How we build

  • Replace Astronomer-specific packages and rebuild images on the official Airflow base image, pinned to a tested version.
  • Stand up the scheduler, DAG processor, API server and workers on the executor and database your team selects, with a PostgreSQL or MySQL metadata database.
  • Test DAG runs side by side with Astro, check Ruff AIR301 and AIR302 findings, and document each test result for your team.

How we hand over

  • Document the image build, deployment manifests, secrets handling, backup and restore steps and the cutover runbook.
  • Walk your platform and data engineering teams through the operations they now own, including upgrades, logging and alerting.
  • Deliver the written scope outcomes, configuration records and test evidence so your team can run and audit the platform without us.

How we support

  • Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
  • Ongoing support, upgrades and continuous testing are available as a retainer after handover.
  • Ask any partner, including DeployOpen, to confirm in writing who patches the Airflow image, the base OS and the Kubernetes cluster after go-live.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.