Apache Airflow vs Dagster vs Prefect: which to choose

Apache Airflow vs Dagster vs Prefect: which fits your team

Start with your situation. If you run Airflow 2, its support has already ended: Airflow 2 reached end of life on April 22, 2026, so it receives no fixes. If you are choosing an orchestrator for a new platform, the decision turns on how your team models work. Airflow 3 suits teams that want Python DAGs of tasks, a mature executor model (Celery, Kubernetes, local and multi-executor setups) and a scheduler that can run in multiple replicas. Dagster suits data platform teams that want to model pipelines as software-defined assets, meaning tables or models declared as Python functions. Prefect suits teams that want Python-native flows with lightweight deployment and an optional managed control plane in Prefect Cloud.

All three projects are licensed under Apache-2.0, so the open-source core carries no license fee. The paid layer is where the choice gets more expensive. In Dagster, role-based access control, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs sit in Dagster+, not in the open-source edition. In Prefect, the server documentation says RBAC, audit logs and SSO are provided by Prefect Cloud, and Prefect's pricing lists SSO from the Team plan and object-level RBAC on Enterprise. Airflow has no paid edition from the project itself, so access control and audit requirements must be met through your own identity and logging setup. Confirm which of these controls your compliance and access policies require before you commit.

Which orchestrator fits which team situation

This table maps five common team situations to the orchestrator that fits best and the condition under which that fit holds. Read each row against your own estate, skills and network rules, since the fit changes if the condition does not apply. The main takeaway is that the open-source cores of all three tools are Apache-2.0, so the deciding factors are usually existing Airflow 2 code, how your team models data, and which paid features you need.

Team situation	Best fit	Why and the condition that holds it
Legacy Airflow 2 estate	Apache Airflow 3	Airflow 2 reached end of life on April 22, 2026, so it gets no fixes. Airflow 3 keeps the DAG model your team already knows. The fit holds if most DAGs can be migrated with limited rewriting; confirm the upgrade path in the Airflow 3 upgrade guide before choosing.
Data platform team building around assets and lineage	Dagster	Dagster organizes work around assets, which are tables or models declared as Python functions. The fit holds if lineage and asset-level observability drive design. Column lineage and catalog search are listed as Dagster+ features, so confirm whether you need them in the paid tier.
Python engineering team that wants flows without a DAG model	Prefect	Prefect builds pipelines as ordinary Python flows with retries, caching and event-based automations. The fit holds if the team writes Python daily and does not need an asset model. It requires Python 3.11 or later, so check your runtime versions first.
Team needing SSO and audit logs without a paid tier	Apache Airflow 3 or Dagster (open source) with a custom identity layer	Dagster+ lists SAML SSO, RBAC and audit logs as paid features, and Prefect's server docs place RBAC, audit logs and SSO in Prefect Cloud. Airflow is Apache-2.0 with no paid edition from the project itself, so the fit holds only if your team can add authentication through its auth manager and logging. Confirm this with your identity team before committing.
Team that must run everything inside its own network	Apache Airflow 3 or self-hosted Dagster or Prefect server	All three have self-hosted paths. Dagster open source runs as a webserver, a daemon and code locations, and Prefect server bundles an API, UI and database. The fit holds if you accept operating the database, scheduler and upgrades yourself; avoid the SQLite defaults in production and plan for PostgreSQL.

What is free and what sits behind Dagster+ or Prefect Cloud

Apache Airflow is an Apache Software Foundation project released under the Apache License 2.0, and the project itself offers no paid edition. Every capability in the open-source distribution is available to you without a license fee. Costs come from the infrastructure you run, the people who operate it and any commercial support or managed service you buy from a third party.

No vendor edition gate: The project publishes no tier that unlocks features, so the same code runs in your environment with no license key.
Support is external: Commercial help comes from separate companies or from your own staff, not from the Apache project.
Operating components are yours: The scheduler, metadata database, executors and upgrades are your responsibility when you self-host.
Version life cycle matters more than license: Airflow 2 reached end of life on April 22, 2026, so it gets no fixes, and staying on it adds risk regardless of cost.

Before you commit, confirm with your security team which access controls, single sign-on and audit configurations your Airflow version supports in your environment, and whether any external support contract is needed to meet your internal requirements.

What drives the running cost of each orchestrator

Each orchestrator's running cost comes from a different set of components: the database and scheduler layout, the daemon and code locations, the server and its supporting services, and the people who upgrade and respond to failures. The cards below set out what drives each cost and what to ask for in a quote, so the comparison reflects your own environment rather than list prices.

Airflow database, scheduler and executor

Self-hosted Airflow runs a scheduler, a metadata database and an executor, and you own all three. The Airflow docs say PostgreSQL or MySQL is recommended for production, that SQLite is for testing only, and that more than one scheduler can run at once using database row-level locks. Each extra scheduler, the executor choice (for example CeleryExecutor workers or KubernetesExecutor pods) and the database tier add infrastructure and monitoring work. Ask the quote to name the database service, the scheduler replica count and the executor, and who maintains each.

Dagster daemon and code locations

Open-source Dagster runs a webserver that can have multiple replicas, a single daemon for schedules, sensors and run queuing, and code location servers limited to one replica each. Run storage defaults to SQLite and can move to PostgreSQL. Cost grows with the number of code locations, each of which is a separate server to run, patch and monitor. Ask who owns the daemon's availability, how code locations are split across teams, and whether the quote covers the PostgreSQL instance for run and event log storage.

Dagster+ when RBAC or lineage is required

Role-based access control, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs are listed on Dagster's pricing page as Dagster+ features, not part of open-source Dagster. If your controls require any of them, the subscription becomes a recurring line item that you pay the vendor directly, on top of hosting. Confirm which of these features your policies need before you size the platform, and ask Dagster or a reseller for a written quote covering the plan, user count and any hybrid agent hosting.

Prefect server database, Redis and load balancer

Self-hosted Prefect server uses SQLite by default and PostgreSQL for production. Multi-server deployments need PostgreSQL 14.9 or later, Redis for event messaging and a load balancer in front of the servers. Each of these is a service you provision, secure, back up and upgrade. Ask the quote to list every component in the multi-server design, including who runs Redis, how database backups are tested and whether the load balancer is in scope.

Prefect Cloud seats for RBAC and SSO

Prefect's server documentation says RBAC, audit logs and SSO are provided by Prefect Cloud and not by self-hosted Prefect server. Prefect's pricing page lists a Team plan at $100 per user per month and an Enterprise plan at custom pricing, with SSO from Team and object-level RBAC on Enterprise. Cost therefore scales with the number of users who need those controls. Ask for a quote that states the seat count, which plan includes each control you need and whether usage above the included allowance is billed separately.

Upgrade and on-call effort

Upgrades are a recurring cost for every option. Airflow 2 reached end of life on April 22, 2026 and gets no fixes, so Airflow 2 estates carry a migration cost now. Dagster and Prefect release often, and Prefect notes that new clients can be incompatible with old servers. Budget for staged upgrades, test environments and an on-call rota that covers the scheduler, daemon or server. Ask the vendor or partner to state who performs upgrades, how many environments are included and what support window applies.

What moving off Airflow 2 actually involves

Airflow 2 entered limited maintenance on October 22, 2025 and reached end of life on April 22, 2026, so it no longer receives fixes of any kind, including security patches. Any team still running Airflow 2 is carrying an unpatched scheduler and webserver, and the first decision is how quickly to leave. The three paths are an upgrade to Airflow 3, a rewrite of DAGs as Dagster assets, or a rewrite as Prefect flows. Each path begins with the same inventory: every DAG, custom operator, plugin, connection, variable and executor setting, plus the external systems each DAG touches. Check executor configuration first. The hybrid CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported as of Airflow 3.0.0, so any DAG estate relying on them needs a different executor design before an upgrade can succeed.

Upgrading to Airflow 3 keeps the DAG model, so the work concentrates on removed or changed imports, operators, configuration and the metadata database upgrade, documented in the Airflow 3 upgrade guide. Rewriting to Dagster or Prefect replaces the authoring model, so each DAG becomes assets with declared dependencies or a Python flow with its own retries and deployments. That rewrite touches schedules, secrets, alerting and the run history your auditors may rely on. Whichever path you choose, plan a parallel run in which the old and new schedulers process the same inputs, then compare outputs, timing and failure behavior for a period you agree on before cutting over. Ask any implementation partner for a written scope that lists the DAG count by category, the parallel-run window, rollback steps and who signs off each cutover. Effort estimates without that inventory are not reliable.

How a team typically runs an orchestrator selection

The sequence below moves from understanding the current Airflow estate to a tested choice between Dagster and Prefect, so the decision rests on evidence from the team's own pipelines rather than feature lists alone.

Inventory existing DAGs and dependencies
Start by listing every Airflow DAG, its schedule, its upstream and downstream dependencies, the operators and providers it uses, and any custom plugins or connections it relies on. Name an owner for each pipeline and mark which ones are business-critical. Airflow 2 is past end of life, so any DAG that runs on it is already outside vendor fixes, and this inventory also shows which code will need rewriting in any migration. The step is done when the team can say how many pipelines exist, which ones depend on each other, and which ones would need to move first.
Decide managed or self-hosted control plane
Decide early whether the team will run the orchestrator's control plane itself or pay for a managed service. Dagster+ and Prefect Cloud both gate features such as SSO, RBAC and audit logs behind paid tiers, so the answer changes which capabilities are available at all. Self-hosting shifts scheduler, database, upgrade and backup work onto the platform team, while a managed control plane shifts it to the vendor under contract. Confirm current paid feature lists directly with each vendor, since plan contents change. The output is a written position on control plane ownership that the pilot must respect.
Pick the metadata database and deployment target
Choose the metadata or state database and the execution target before evaluating code. Airflow documents PostgreSQL or MySQL for production and advises against SQLite. Dagster's run and event log storage defaults to SQLite and can use PostgreSQL, and Prefect server uses SQLite by default with PostgreSQL recommended for production. Decide whether tasks run on long-lived workers, on a Kubernetes pod per task, or on another target, and confirm that the chosen database and runtime are supported by the candidate's current documentation. The check is a short design note that names each database and runtime per candidate.
Map the current pipelines onto each candidate
Rewrite a representative slice of pipelines in each candidate's model. Airflow's task-centric DAGs translate differently into Dagster software-defined assets than into Prefect's Python flows, and that difference usually drives most of the migration effort. Engineers should note where operators, custom sensors or cross-DAG dependencies need redesign and where they map cleanly. The output is a mapping sheet per candidate listing what converts directly, what needs redesign and what has no equivalent, which the team can review with the data engineers who own those pipelines.
Run a pilot pipeline in two candidates
Choose one pipeline with real data, real schedules and at least one failure mode, and run it in two shortlisted candidates in a non-production environment. Measure how each handles retries, backfills, alerting, secrets, access control and upgrades, and have the security team test SSO and audit logging against the tier the team plans to buy. Record the operator effort to build, test and monitor each version. The pilot is complete when the team has a comparison table built from its own runs, reviewed by both platform and data engineers.
Plan cutover and retire the Airflow estate
Plan the migration in waves, starting with lower-risk pipelines, and run old and new pipelines in parallel until outputs reconcile. Decide what happens to Airflow metadata, run history and connections, and agree a date after which Airflow 2 DAGs are no longer maintained or triggered. Document the runbook, rollback path and ownership for each wave. Ask the chosen vendor for a written statement on upgrade support windows and version compatibility before committing. The step closes when every migrated pipeline has a reconciled output, a named owner and a documented rollback.

Questions buyers ask before choosing an orchestrator

How much does it cost to self-host Airflow, Dagster or Prefect?

Each project is Apache-2.0 licensed, so the software itself has no license fee. The recurring cost is the infrastructure and the engineering time to run it: a scheduler or daemon, a web server, a metadata database, and workers or executors sized to your workload. Airflow adds the executor choice (Local, Celery, Kubernetes or others) and a PostgreSQL or MySQL metadata database. Dagster needs a webserver, a daemon and code location servers, with PostgreSQL recommended in place of the default SQLite for run storage. Prefect self-hosts an API, a UI and a database, and multi-server setups add PostgreSQL, Redis and a load balancer. Costs rise with the number of DAGs or runs, the size of workers, high availability requirements and the number of environments. Ask any vendor or partner for a quote that itemizes compute, database, storage, backup and the engineering hours for upgrades and on-call.

Can our team migrate existing Airflow 2 DAGs without outside help?

A team with Python and Airflow experience can often migrate in-house, but the answer depends on how much of the estate uses Airflow 2 features that changed in Airflow 3 and on how many DAGs depend on the old hybrid executors. Airflow 2 reached end of life on April 22, 2026, so the work is now a required upgrade, not an option. The Airflow upgrade guide lists the breaking changes to review, and the CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported as of Airflow 3.0.0. A team that moves to Dagster or Prefect rewrites DAGs as assets or flows, which is a larger change than upgrading Airflow in place. An in-house pilot on a few representative DAGs will show whether your team can finish the work, and a fixed-scope engagement from DeployOpen can cover the same ground if capacity is short.

Does Airflow 3 include SSO and role-based access without a paid tier?

Airflow is open source under Apache-2.0 and has no paid edition from the project itself, so access control is configured through its auth manager rather than bought as a license. Confirm the exact auth manager options for your Airflow 3.3.2 deployment in the auth manager documentation, and check whether your identity provider integration needs extra development or a maintained plugin. Dagster and Prefect are different: their pricing pages place role-based access control, SAML SSO and audit logs in Dagster+ or Prefect Cloud (Prefect lists SSO from its Team plan). The cost question is therefore less about the license and more about the build and support effort for your own identity integration, plus any compliance evidence you need for access reviews.

Which features need Dagster+ or Prefect Cloud rather than open source?

For Dagster, the pricing page lists role-based access control, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs as Dagster+ features rather than part of the free open-source Dagster. For Prefect, the server documentation says Prefect Cloud provides RBAC, audit logs and SSO that self-hosted Prefect server does not include, and the Prefect pricing page places object-level RBAC and multiple workspaces on Enterprise. Confirm current plan contents on the pricing pages before you decide, because editions change. If your audit or access requirements depend on these features, plan either a paid subscription or your own implementation of the equivalent controls, and price both options.

How long does an orchestrator migration take?

No duration can be stated from the researched facts, so confirm it against your own scope. The main drivers are the number of DAGs and their dependencies, custom operators and hooks that need replacing, the number of data sources and connections, how much test coverage exists to prove output parity, how many environments must move, and whether you are also changing the metadata database, executor or identity setup. Moving from Airflow 2 to Airflow 3 is a version upgrade with a defined set of breaking changes, while moving to Dagster or Prefect rewrites how work is modeled. Ask for a written plan with phases, cutover criteria and a rollback path, and agree on it before work starts.

What does running these orchestrators require from our engineers?

Expect to need Python engineers who can own DAGs, assets or flows, and platform engineers who can run the infrastructure. For Airflow, that means the scheduler (which supports multiple instances using database row-level locks), a PostgreSQL 14 to 18 or MySQL 8.0 database, the executor and its workers, and Python 3.10 to 3.14 for Airflow 3.3.2. Dagster needs the webserver, daemon and code locations, with run storage moved off SQLite for anything serious. Prefect needs Python 3.11 or later and, for production, PostgreSQL. Your team also owns upgrades, backups, monitoring and security patching. Check whether you have on-call coverage for scheduler outages and whether the people who run the platform are different from the people who write pipelines.

Does Prefect's client and server version compatibility affect upgrades?

Yes, it affects the order of upgrades. Prefect's server documentation says old clients work with new servers, but new clients can be incompatible with old servers because new fields are added to the REST API. The practical rule is to upgrade the server first, then move workers and flow environments to the new client version, and to test that sequence in a staging environment. Pin client versions where possible, and track the Prefect release notes, since 3.x and 2.x lines were both receiving releases in 2026. Confirm your own rollout plan with your platform team before upgrading production.

Which team should pick each platform, and what would change the answer

Pick Apache Airflow when your estate is already built around task DAGs, your team runs Python and SQL pipelines on a scheduler it trusts, and you need the widest choice of execution backends. Airflow 3.3.2 is the current release, and it offers LocalExecutor, CeleryExecutor, KubernetesExecutor, EdgeExecutor and AWS Batch and ECS executors, with several executors able to run in one deployment since Airflow 2.10.0. That range matters most to platform teams with mixed workloads, such as short tasks on Kubernetes beside long jobs on shared workers. Airflow also fits organizations whose governance depends on a mature, Apache-2.0 project with no paid edition from the project itself. The catch is that if you are still on Airflow 2, which reached end of life on April 22, 2026, the upgrade is now mandatory work, and that is the moment to ask whether the migration should land on Airflow 3 or on a different engine.

Pick Dagster when your team thinks in tables, models and data products rather than in tasks, and when lineage and asset health are the main reason for the tool. Dagster organizes work around software-defined assets, so a team can express what should be fresh and how it depends on upstream data, which is the strongest asset model of the three. Choose Prefect when your engineers want the lightest path from a Python function to a scheduled, retried and observable flow, and when event-driven automations matter more than a DAG-wide view of the estate. Both Dagster and Prefect carry a governance question. Dagster+ holds role-based access control, SAML SSO, column lineage, audit logs and alerts, and Prefect Cloud holds RBAC, audit logs and SSO, so a regulated team that needs these controls must either buy the hosted tier or build equivalents. Confirm the exact controls you need against the Dagster pricing page and the Prefect pricing page before you shortlist either option.

Several conditions would reverse this verdict. If your auditors require SSO, RBAC and audit logs on the orchestrator itself, and you will not buy a paid tier, Airflow 3 with your own identity layer may be the more defensible route, but only if you have confirmed how its auth manager fits your IdP. If most of your pipelines are long-running jobs that need shared Celery workers, Airflow remains the strongest fit. If your data platform is already built on asset-based dbt and lineage catalogs, Dagster usually moves faster. If your team builds mostly ad hoc, event-triggered Python with little scheduling, Prefect is simpler to operate. Before deciding, run one representative pipeline on each candidate, record the self-hosted components it needs (scheduler, database, workers and webserver) and ask each vendor for a written quote that separates open-source operation from the paid governance features your controls require.

How DeployOpen helps choose and implement an orchestrator

DeployOpen is a Chainscore Labs company that plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For Apache Airflow vs Dagster vs Prefect, we start by mapping your existing DAGs, assets and flows, then help you decide which orchestrator fits your team and your Airflow 2 migration.

How we scope

Inventory existing Airflow 2 DAGs, operators, connections and the team that maintains them before choosing a target.
Define which features you need from open-source Dagster or Prefect and which would require Dagster+ or Prefect Cloud.
Agree scope, deliverables and timelines in writing before work starts, since engagements are fixed-scope.

How we build

Deploy the chosen orchestrator with its metadata database, scheduler or daemon, and workers in your environment.
Port a representative set of pipelines and test them locally and in the target environment before cutover.
Configure authentication, secrets handling and access controls using the options your chosen edition actually provides.

How we hand over

Document architecture, configuration, runbooks and the upgrade path for your chosen version and editions.
Walk your platform and data engineers through deployment, monitoring, backups and pipeline authoring conventions.
Provide evidence of configuration and controls that your internal or external assessors can review.

How we support

Offer ongoing support, upgrades and continuous testing as a retainer after handover, scoped in writing.
Plan version upgrades around release windows and end-of-life dates published by each project.
Confirm with each vendor which features, licenses and support terms apply to your edition before committing.

Apache Airflow vs Dagster vs Prefect