Apache Airflow vs Kestra: when each one fits
Choose Apache Airflow 3 if your data engineers already write Python, your estate is built on DAGs, and your main job is scheduled batch pipelines that need a mature scheduler and a wide base of existing operators and knowledge. Airflow is Apache-2.0 licensed with no paid edition from the project itself, so the license and the feature set are the same for every deployment. Choose Kestra 2.0 if pipelines should be written as declarative YAML flows that analysts, platform staff and non-Python contributors can read and change, and if event-driven triggers are central to how work starts. Kestra's open-source edition is also Apache-2.0 licensed. The two options differ mainly in authoring model and in what security controls you get without paying.
The decision turns on three questions. First, who authors pipelines: if the answer is mostly Python developers, Airflow's DAG model is the lower-friction path, and if it includes operators who prefer a YAML editor or a no-code editor, Kestra's authoring model fits better. Second, whether event-driven triggers are central: Kestra lists scheduled and event-driven triggers among its core features, while Airflow's model is built around schedules and sensors. Third, and most often decisive for regulated teams, whether SSO, RBAC and audit logs must be in the open-source edition. Kestra's edition comparison says the open-source edition has basic authentication only, manual secret and access management and no audit logging, with SSO, SCIM, RBAC and audit logs listed under Enterprise Edition. Confirm current Enterprise terms with Kestra before you rely on them.
Before you commit, check what the upgrade path costs in each direction. Airflow 2 reached end of life on April 22, 2026, so staying put means moving to Airflow 3, and that move still requires reviewing DAGs against the Airflow 3 upgrade guidance. Moving to Kestra means rewriting DAGs as flows, and Kestra 2.0 has its own breaking changes and migration guide. Ask both vendors and your own engineers to estimate effort on a representative sample of your DAGs, and record which auth, audit and secret requirements must be met on day one. That single list will usually settle the choice.
Which platform fits which team scenario
This table maps common team situations to the platform that usually fits better and the condition that would reverse the choice. Read each row against your own skills, existing DAG estate and compliance obligations, since the stronger option depends on them more than on feature lists.
| Team scenario | Better fit and reason | Condition where the other platform is stronger |
|---|---|---|
| Python-heavy data engineering team with a large existing DAG estate | Apache Airflow. Existing DAGs, operators and team habits carry over to Airflow 3, and the Apache-2.0 project has no paid edition to buy into. | Kestra becomes stronger if the estate is being rewritten anyway and the team wants declarative YAML flows that non-Python authors can maintain. |
| Operations or platform team wanting declarative flows managed in Git | Kestra. Flows are defined in YAML, Git version control integration is listed in the README, and the YAML editor, no-code editor and AI Copilot stay in sync. | Airflow fits better if the team already reviews Python code in Git and needs the Python ecosystem more than YAML authoring. |
| Team needing event-driven triggers with AI or infrastructure tasks | Kestra. Scheduled and event-driven triggers, AI agent and RAG task types, and a built-in MCP server are documented in the README. | Airflow fits if triggers are mostly time-based and the team can build sensor logic in Python that it already tests and maintains. |
| Regulated team that needs SSO, RBAC and audit logs from day one | Neither open-source edition provides these out of the box. Kestra's edition comparison places SSO, SCIM, RBAC and audit logs in Enterprise Edition, so a Kestra Enterprise license must be priced into the plan. | Airflow may suit if the team can use its auth manager and a supported identity layer, which must be confirmed against the Airflow auth manager documentation for the controls required. |
| Team that must run many small workers and mixed executors | Airflow, which documents several executors, including KubernetesExecutor, and lets several run in one deployment since Airflow 2.10.0. | Kestra fits if the team wants worker groups and task runners managed through its Enterprise high availability design. |
| Team with a high-availability requirement on the open-source edition | Airflow, which documents multiple schedulers using database row-level locks, with PostgreSQL or MySQL as the metadata database. | Kestra's high availability architecture is listed under Enterprise Edition, so it fits teams already buying that edition. |
| Team migrating from Airflow 2 after its end of life | Depends on the estate. Airflow 2 reached end of life on April 22, 2026, so the move to Airflow 3 is required either way, and Kestra adds a rewrite on top of it. | Kestra fits if the team plans a full rewrite into YAML flows and accepts the Kestra 2.0 migration work as the cost of that rewrite. |
How Airflow 3 and Kestra are built and deployed
Airflow 3 is a set of cooperating services you assemble and operate yourself, with a metadata database at the center. Kestra is a single application with a declarative model, and its open-source edition defaults to one server. The cards below set out what each design asks of your platform team after go-live.
Airflow scheduler and multiple schedulers
The scheduler parses triggers, decides which tasks run and hands them to executors. Airflow 3 supports running more than one scheduler at the same time for performance and resiliency, coordinating through database row-level locks. Running several schedulers is a design decision to test against your database and workload, not a default you get for free. Check how your team will monitor scheduler health and which database capacity you have reserved for lock contention.
DAG processor, API server and their boundaries
Airflow 3 separates DAG parsing into its own DAG processor and serves the UI and API from a dedicated API server. Your team must run, scale and restart each process, and confirm how they reach the metadata database and each other in your network. Ask whether your platform already has a pattern for supervising several long-running Python services, since this separation is what changes the day-two runbook compared with Airflow 2.
Metadata database choice
Airflow stores DAG runs, task state and connections in its metadata database. The prerequisites list PostgreSQL 14 to 18 and MySQL 8.0, 8.4 and Innovation releases, and do not support MariaDB as the metadata database. SQLite is for testing only, and the production guide warns that running it in production can lead to data loss. Plan backup, restore and version upgrades for this database as a core service, because every scheduler and worker depends on it.
Executor selection
The executor decides where tasks run, such as LocalExecutor, CeleryExecutor, KubernetesExecutor or the AWS Batch and ECS options. Since Airflow 2.10.0, several executors can run in one deployment, but the hybrid CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported as of Airflow 3.0.0. Pick the executor before building pipelines, since moving between them changes worker operations, scaling and failure behavior.
Kestra's declarative YAML model
In Kestra, a flow is a declarative YAML definition that can be edited in a YAML editor, a no-code editor or an AI Copilot, and these views stay in sync. Scheduled and event-driven triggers sit in the same definition, and Git version control integration is listed among its features. For your team, the review and deployment process centers on YAML files in version control rather than Python code executed by a scheduler, so confirm your code review and promotion practice fits that model.
Kestra single-server default and what Enterprise adds
The open-source Kestra edition has a single-server default deployment, basic authentication only and manual secret and access management. High availability with Kafka plus Elasticsearch, Redis, AMQP or GCP Pub/Sub queue backends, worker groups and task runners is part of Kestra Enterprise Edition. Before choosing an open-source deployment for production, confirm with Kestra which recovery targets and availability design your environment needs, and what a single server means for your maintenance windows.
Feature comparison: Apache Airflow 3 and Kestra 2.0
This table compares the open-source editions of Apache Airflow 3 and Kestra 2.0 on licensing, authoring, triggers, identity, audit, secrets and high availability. Read the Kestra rows that name Enterprise features as paid items, and confirm Airflow access control details against the auth manager documentation before you decide.
| Capability | Apache Airflow 3.3.2 | Kestra 2.0 (v2.0.6) |
|---|---|---|
| License of the open-source edition | Apache-2.0, from the Apache Software Foundation with no paid edition from the project | Apache-2.0 for the open-source edition, with a separate Enterprise Edition |
| Current version | 3.3.2 is the current patch release; Airflow 2 reached end of life on April 22, 2026 | v2.0.6 published October 13, 2026; the 1.3 line still receives patches |
| Authoring model | Python DAGs, with tasks defined in code | Declarative YAML flows, with a YAML editor, a no-code editor and an AI Copilot kept in sync |
| Scheduled triggers | Supported through DAG schedules; confirm options in the Airflow docs | Scheduled and event-driven triggers listed in the Kestra README |
| Event-driven triggers | Confirm which event-driven options your Airflow 3 setup supports before you rely on them | Event-driven triggers are a core part of the platform |
| Plugins and integrations | Provider packages and custom operators written in Python | Hundreds of built-in plugins for databases, cloud storage and APIs; scripts in any language |
| AI features | No built-in AI agent task type in the Airflow core; confirm with your team's plans | AI agent and RAG workflows as task types, plus a built-in MCP server that exposes flows to AI agents |
| Authentication and RBAC in the open-source edition | Depends on the auth manager you configure; confirm capabilities in the auth manager documentation | Basic authentication only; SSO, SCIM and RBAC are Enterprise Edition features |
| Audit logs | Confirm audit logging against your chosen auth manager and logging setup | No audit logging in the open-source edition; audit logs are an Enterprise Edition feature |
| Secrets management | Use the secrets backends your team configures; confirm supported backends in the docs | Manual secret management in open source; external secrets managers such as HashiCorp Vault are Enterprise Edition features |
| High availability | Multiple schedulers run at once using database row-level locks, backed by PostgreSQL or MySQL | Single-server default deployment in open source; high availability architecture is an Enterprise Edition feature |
What the Kestra open-source edition lacks compared with Enterprise
The most useful way to read this comparison is that the two products draw their paid lines in different places. Apache Airflow is an Apache Software Foundation project under the Apache License 2.0, and the project itself offers no paid edition. Access control and audit therefore depend on what your team configures and which auth manager you run, not on a vendor tier. Kestra is different: its open-source edition is also Apache-2.0 licensed, but Kestra's own edition comparison places several controls in the Enterprise Edition. Before you shortlist either product, map each control you need to the edition that provides it.
Per Kestra's edition comparison, the open-source edition is fully functional for production but has basic authentication only, a single-server default deployment, manual secret and access management, and no audit logging. Enterprise Edition adds SSO, SCIM and RBAC, service accounts and API tokens, external secrets managers such as HashiCorp Vault and Azure Key Vault, audit logs, multi-tenancy and worker isolation. It also adds a high availability architecture using Kafka with Elasticsearch, Redis, AMQP or GCP Pub/Sub queue backends, plus worker groups and task runners for Kubernetes and cloud batch services. If an auditor or security team will ask who changed a flow and when, the audit log gap alone may decide the edition.
Ask Kestra for the current license terms and pricing in writing before you cost either option. Request a list of which Enterprise features apply to your deployment model, the support terms for the 2.0 line and the 1.3 line, and whether any high availability backend is required for your target scale. Then compare those answers with what Airflow needs from your own team: an identity provider integration, a secrets backend, a metadata database that is not SQLite, and the audit and logging pipeline you build around them. Both paths carry effort, but only one of them has a vendor invoice attached.
What drives the running cost of each platform
Running cost for Apache Airflow and Kestra is set by four components: the infrastructure each one needs, the people who author and operate pipelines, the upgrades you will take on, and the support you buy. Read each card against your own team's skills and your availability and recovery requirements, then ask every vendor and partner for the same written scope before comparing totals.
Infrastructure for Airflow
Airflow runs a scheduler, workers and a metadata database that your team operates. The Airflow production guide says the default SQLite backend is for testing only and PostgreSQL or MySQL is recommended, and the prerequisites recommend at least 4 GB of RAM. Multiple schedulers can run at once against the same database, which adds resilience but also database load. Ask who sizes and backs up the metadata database and how workers scale with your task volume.
Infrastructure for Kestra
The open-source edition ships with a single-server default deployment, which keeps the footprint small. High availability is a paid Enterprise feature that adds Kafka plus Elasticsearch, Redis, AMQP or GCP Pub/Sub queue backends, and worker groups. Each backend you add is one more service to size, patch and monitor. Ask whether your availability target requires the Enterprise architecture, and which queue and storage services you already run.
Skilled staff for DAG and flow authoring
Airflow pipelines are Python, so your engineers test DAGs with the same unit test and code review practices they use for application code. Kestra flows are declarative YAML with scripts in any language, which lowers the bar for analysts but moves complexity into plugin versions and task configuration. Both need ongoing ownership of plugins, connections and secrets. Ask who will maintain the platform after handover and what share of their time that represents.
Upgrade work and breaking changes
Airflow 2 reached end of life on April 22, 2026, so teams still on it must move to Airflow 3, and the upgrade guide lists the changes to plan for. Kestra 2.0 is a major release with breaking changes, including replacing ForEach and ForEachItem with a single Loop task. Upgrades recur for both platforms, but the Airflow move is a one-time migration of existing DAGs. Ask for an inventory of DAGs or flows and a list of breaking changes that touch them before quoting.
Support and subscription costs
Airflow has no paid edition from the project itself, so any support cost is a managed service, a vendor contract or an implementation partner. Kestra's Enterprise Edition is a paid subscription that adds SSO, RBAC, audit logs and HA. Implementation support from DeployOpen is usually agreed as a fixed-scope engagement, with ongoing support and upgrades available as a retainer. Ask each provider for the recurring fee, what it covers and what a one-time migration would add.
The cost of moving existing DAGs to Kestra or upgrading to Airflow 3
This sequence shows what it takes to either move an existing Airflow estate to Airflow 3 or rebuild its DAG logic as Kestra flows, and how to cut over without losing a run. Both paths start from the same inventory and end with a tested rollback.
Inventory DAGs, dependencies and runtime context
List every DAG, its schedule, upstream and downstream dependencies, the operators and hooks it imports, the connections and variables it reads, and the systems it writes to. Include DAGs that only run on manual trigger or on a sensor. Your data engineers own the list, and a platform owner signs off on it. The step is done when each DAG has a named owner, a business purpose and a decision recorded as either keep, upgrade or rebuild. Ask the implementation partner to state in writing how they will scope this inventory before any code changes.
Decide the path: upgrade to Airflow 3 or rebuild
Airflow 2 reached end of life on April 22, 2026 and gets no fixes, so staying on Airflow means moving to Airflow 3 (current patch 3.3.2). Moving to Kestra means rebuilding logic as YAML flows and accepting Kestra 2.0 breaking changes. Check the Airflow 3 prerequisites against your Python, PostgreSQL or MySQL, and Kubernetes versions, and confirm that your metadata database is not MariaDB or SQLite. The decision is recorded with the reasons, the people accountable and the platform version you will target.
Map connections, secrets and access to the target
On the Airflow side, connections and variables are stored in the metadata database or a configured secrets backend. On the Kestra side, the open-source edition relies on manual secret and access management and basic authentication, while the Enterprise Edition adds SSO, SCIM, RBAC, service accounts, API tokens and external secrets managers such as HashiCorp Vault and Azure Key Vault. Build a table that maps each Airflow connection to its new home and each access policy to its new role. If you need SSO, RBAC or audit logs in Kestra, confirm the Enterprise licensing terms with Kestra before writing flows, because these features do not exist in the open-source edition.
Separate what transfers from what is rebuilt
Business logic, schedules as concepts, data contracts and Git history transfer with the work, although schedules must be re-expressed in each tool's syntax. Operators, Python-specific code, sensors, custom retry behavior, alert callbacks and access policies are rebuilt. For Kestra, the Airflow for-loops and branching need to be reworked into Kestra tasks, and the Kestra 2.0 migration guide lists the breaking changes such as replacing ForEach and ForEachItem with a single Loop task. The output is a rebuild register that lists each DAG, its target flows and the test that proves equivalence.
Build and test migrated pipelines side by side
Implement the upgraded DAGs or the new flows in a non-production environment, then compare outputs on the same input data. Test failure paths as deliberately as success paths: retries, timeouts, partial loads and alert delivery. For Airflow 3 upgrades, verify the executor choice, since the hybrid CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported from Airflow 3.0.0. Done means each pipeline has a recorded output comparison and a passed failure-path test, signed by the data owner. Ask the partner to name who runs these tests and how results are archived.
Run old and new pipelines in parallel
Keep the existing Airflow 2 or Airflow 3 schedules running while the migrated pipelines write to a separate target, or to a shadow schema, for a period agreed in the written scope. Compare row counts, freshness, lineage and alert volume each day. The parallel period is the most expensive phase because two platforms run and two teams watch them, so ask for that cost to be named in the quote. The step is complete when discrepancies are either resolved or accepted in writing by the data owner.
Cut over and freeze the old schedules
Switch the production schedules to the new platform in a planned window, disable the old DAGs or flows so nothing runs twice, and update downstream consumers, dashboards and alert routes. Confirm that secrets are read from the intended store and that access policies match the register. The step closes when the first scheduled runs on the new platform succeed and the consumers confirm fresh data.
Keep a tested rollback and retire the old estate
Keep the old DAGs, metadata database snapshot and configuration in read-only form for a retention period set in your change policy, and rehearse the rollback before cutover. Retire the old estate only after the rollback path has been exercised and the change record is complete. Document the final mapping, test evidence and owner for each pipeline so that auditors and future maintainers can trace each decision.
Where implementation projects usually run long
Implementation projects on Airflow usually run long at the metadata database and the upgrade path. Airflow 3.3.2 expects PostgreSQL 14 to 18 or MySQL 8.0, 8.4 or an Innovation release, and the production guide is clear that the default SQLite backend is for testing only. The first question to settle is who owns that database: backups, failover, connection pooling and version upgrades need a named team before scoping. Choose the executor early too, because CeleryKubernetesExecutor and LocalKubernetesExecutor no longer exist in Airflow 3.0.0 and any Airflow 2 DAG that relied on them needs a new design. Teams moving from Airflow 2, which reached end of life on April 22, 2026, should inventory every DAG for Python dependencies that are unpinned, pinned to Airflow 2 internals, or bundled with custom operators, since these are where migration estimates most often slip. Confirm with Apache Airflow's upgrade guide for Airflow 3 which changes apply to your DAGs.
Kestra projects run long somewhere else: in flow authoring conventions, plugin selection and identity. The open-source edition is fully functional for production but offers basic authentication only, manual secret and access management and no audit logging. If your security team expects SSO, SCIM, RBAC, service accounts, Vault or Azure Key Vault integration or audit trails, those sit in Kestra Enterprise Edition, so the identity requirement should be raised before the first flow is written, not after. Kestra 2.0 also brings breaking changes, including replacing ForEach and ForEachItem with a single Loop task, so existing flows written against the 1.3 line need a migration pass. Plugin selection needs a governance decision as well, since hundreds of built-in plugins are available and each one is a connection your security team must review. For high availability, the Enterprise edition adds Kafka with Elasticsearch, Redis, AMQP or GCP Pub/Sub queue backends. Ask Kestra which backend it expects you to run and who operates it.
Before scoping either platform, confirm four things in writing: which metadata database and backup regime you will run, which Airflow or Kestra features your security baseline requires and whether they are in the edition you intend to use, which identity provider and group model you must integrate, and how many existing DAGs or flows will be migrated versus rebuilt. Test each assumption against your own environment, including network paths to the database, secret stores and identity provider. Treat these as fixed-scope questions: a written scope with named deliverables makes the migration and integration work visible before the budget is set.
Apache Airflow vs Kestra: questions buyers ask before hiring help
Can we keep our Airflow 2 DAGs or do we have to rewrite them?
Airflow 3 is the current major line, and Airflow 2 reached end of life on April 22, 2026, so existing DAGs will need to move to a supported version either way. Airflow's own upgrade guide is the first read, and it covers the changes that affect DAG code. Kestra takes a different approach: its flows are declarative YAML, so Python DAGs are not carried over line by line. Kestra's 2.0 migration guide also documents breaking changes, including replacing ForEach and ForEachItem with a single Loop task. Ask your partner to inventory every DAG, operator, custom plugin and connection, then sort them into those that move to Airflow 3 with minor edits, those that need rewriting as Kestra flows, and those that should be retired.
What drives the time a migration takes?
The timeline depends on the size and shape of the existing estate rather than on a single number. The main drivers are the count of DAGs, how many custom operators and hooks they depend on, the number of external connections and secrets, whether dependencies are pinned to Airflow 2 providers, and how many teams must test and sign off on their pipelines. Event-driven flows you want to build in Kestra add design work that a like-for-like port does not need. No migration duration is stated here because none is supported by a researched source. Ask for a written scope that lists DAG inventory, environments, test criteria and cutover steps, then compare the estimate against that list.
What does a Kestra Enterprise purchase add over the open-source edition?
Kestra's edition comparison says the open-source edition is fully functional for production but has basic authentication only, a single-server default deployment, manual secret and access management and no audit logging. Enterprise Edition adds:
- SSO, SCIM and RBAC, plus service accounts and API tokens
- External secrets managers such as HashiCorp Vault and Azure Key Vault
- Audit logs, multi-tenancy and worker isolation
- A high availability architecture with Kafka plus Elasticsearch, Redis, AMQP or GCP Pub/Sub queue backends, worker groups and task runners for Kubernetes and cloud batch services
Confirm current pricing and terms directly with Kestra, since the figures change and are not supplied here.
Can we self-host Airflow or Kestra without outside help?
Yes, both are open source under the Apache License 2.0, so your team can install and run either one in its own infrastructure. The harder question is whether your staff can operate it to production standard. Airflow's production guidance says SQLite is for testing only and recommends PostgreSQL or MySQL, and it lists supported Python, database and Kubernetes versions that you must track. Kestra's open-source edition defaults to a single server with manual secret and access management, so you would build the controls yourself if you need them. Outside help is most useful for the hardening, upgrade and evidence work that a small team has no time for.
What skills does our team need to run Airflow 3 or Kestra?
Airflow needs Python skills for authoring DAGs, plus database operations for the metadata store, executor configuration and Kubernetes or worker management when you scale out. Kestra needs YAML and Git practice, because flows are declarative and the README describes Git version control integration, along with the operations work for its queue, storage and worker setup. Both need someone who can read logs, manage secrets and plan upgrades. Ask yourself which group will write and review pipelines: if data engineers already write Python, Airflow fits that skill set, and if analysts and platform staff prefer configuration files, Kestra may suit them better.
How does a fixed-scope engagement differ from a retainer?
A fixed-scope engagement agrees scope, deliverables and timelines in writing before work starts, which suits a defined migration or a single platform build. A retainer covers ongoing support, upgrades and continuous work over a period, which suits teams that will keep changing pipelines after go-live. For a migration, a fixed scope keeps the deliverables clear, while a retainer can cover the upgrade cycle that follows. Ask for the deliverables, acceptance criteria, what happens when the inventory reveals more DAGs than expected, and how change requests are priced before you sign.
What should we ask for in a quote for this work?
A useful quote separates one-time work from recurring work. One-time items usually include the DAG and flow inventory, platform build, migration, testing, documentation and cutover. Recurring items usually include upgrades, monitoring, backups and support. Ask the provider to state which Airflow and Kestra versions are in scope, which environments are covered, how secrets and identity integration are handled, and who owns the code after handover. DeployOpen does not publish a price list, and a quote follows from the written scope.
How DeployOpen helps choose, implement or migrate
DeployOpen is a Chainscore Labs company that plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure. For Apache Airflow vs Kestra, we start with the decision your team has to make, then scope the work in writing before anything is built.
How we scope
- We confirm which pipelines, schedulers and teams are in play before agreeing any work.
- Most engagements are fixed-scope, with deliverables and timelines agreed in writing first.
- We identify which Kestra edition features your authentication and audit requirements need.
- Quotes follow from the written scope, so the scope document should list every DAG or flow to move.
How we build
- We stand up Airflow 3 or Kestra in your cloud account, data center or air-gapped network.
- We set up the metadata database, executors, secrets handling and identity integration you choose.
- We test migrated DAGs against Airflow 3 behavior and Kestra flows against their YAML definitions.
- We harden the deployment and document configuration, access and backup procedures as it is built.
How we hand over
- You receive documentation covering architecture, configuration keys, runbooks and upgrade steps.
- Your team runs a handover session on operating the scheduler, workers and database.
- Migration records list which DAGs were ported, rewritten or retired, with the reasons for each.
- Evidence for compliance assessments is organized so your own auditors can review it directly.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers after handover.
- A retainer can cover Airflow or Kestra version upgrades, including breaking changes in each release.
- Security testing of pipelines and their APIs can be scheduled alongside platform support.
- Independent auditors and certification bodies, not DeployOpen, issue any audit reports or certifications.
