Apache Airflow high availability guide

What Apache Airflow high availability means for a platform team

Apache Airflow high availability is not a single setting. It is a set of separate redundancy decisions across the scheduler, API server, DAG processor, triggerers, workers and metadata database, and each component fails in a different way. Airflow is released under the Apache License 2.0 (Apache-2.0) and has no paid edition from the project, so the resilience design and its running cost sit with your team or the partner you hire. Once scheduled pipelines feed reporting or operations, the practical question is which component can stop without stopping scheduled work, and how long each one can be unavailable before someone notices.

The hardest part is the metadata database. The scheduler and the other components depend on it for pipeline state, so it is the most likely single point of failure. The Airflow production guide warns that the default SQLite backend is for testing only and can lead to data loss in production, and it recommends PostgreSQL or MySQL instead. The current prerequisites list PostgreSQL 14 to 18 and MySQL 8.0, 8.4 and Innovation releases, and state that MariaDB is not supported as a metadata database. Decide early whether you will run a clustered database with automatic failover, a managed service in your own cloud account, or a single instance with a tested restore procedure, and confirm that the choice is supported for your Airflow version.

Airflow 3 changes the shape of the design. The webserver is replaced by an API server started with the airflow api-server command, and the DAG processor must run as a separate component. A redundancy plan written for Airflow 2 therefore has to be redrawn, because the API server and DAG processor need their own instances and their own failure handling. Airflow supports running more than one scheduler at the same time, using database row-level locks, which is the main scheduler-level resilience option. Confirm how each component behaves on the exact patch you will deploy, since the upgrade guide and the version life cycle define what is supported.

Before choosing a topology, ask for a written component-by-component map. It should state how many instances of each component you will run, what happens to running tasks when a worker or triggerer stops, how the metadata database fails over, and who restores it and how often that restore is tested. Check the version life cycle as well: Airflow 3 was first released on April 22, 2025, Airflow 2 reached end of life on April 22, 2026, and end-of-life versions receive no fixes. Build the design for the supported release you will run after go-live, and treat any failover that has not been tested as unproven.

Which Airflow components need redundancy and why

Airflow's components fail in different ways and recover in different ways. Start with the metadata database, because every other component depends on it, then size each remaining component against what stops when it fails.

Scheduler

The scheduler decides which task instances are due and queues them for execution. Airflow supports running more than one scheduler at the same time, coordinating through database row-level locks, so a second scheduler can keep dispatching work if one stops. With a single scheduler, a failure halts new task scheduling until a replacement starts. Confirm how already-running tasks behave with your executor during a scheduler outage, and test failover against your own DAG volume.

API server

In Airflow 3, the webserver is replaced by the API server, started with airflow api-server. It serves the UI and the REST API that task code uses to reach Airflow state, so an outage affects operators and automation even when schedulers keep running. Confirm in the Airflow 3 documentation for your chosen version whether several instances behind a load balancer are supported. Decide how TLS and sessions are handled at the load balancer, and test a rolling restart.

DAG processor

Airflow 3 requires the DAG processor to run as its own component, separate from the scheduler. It parses DAG files, so a stopped DAG processor stops new or changed DAGs from being picked up. Parsing load grows with DAG count and file complexity, so size it separately from the scheduler. Check the upgrade guide and your version's documentation for whether it can run as several instances and how DAG files are shared between them.

Triggerer

The triggerer runs deferred tasks, which wait on external events without holding a worker slot. If it stops, deferred tasks do not resume until it runs again. Whether you can run multiple triggerers, and how they share deferred work, depends on the Airflow version you deploy. Check the triggerer documentation for your chosen version before designing high availability. If the documentation does not confirm multiple instances, plan a restart procedure and alerting for the single instance.

Workers by executor

Worker redundancy depends on the executor you choose. LocalExecutor runs tasks on the scheduler host, so there is no separate worker tier to replicate. CeleryExecutor runs tasks on worker processes that can be added on several hosts, and losing one reduces capacity. KubernetesExecutor starts a pod per task, so cluster scheduling and node redundancy matter as much as Airflow settings. Pick the executor first, then map the failure modes it creates.

Metadata database

The metadata database holds DAG runs, task state, connections and the state that schedulers coordinate through. Every other component depends on it, so it is the first redundancy decision. The Airflow 3.3.2 prerequisites list PostgreSQL 14 to 18 and MySQL 8.0, 8.4 and Innovation releases as supported, and MariaDB is not supported. SQLite should not be used in production. Ask your database team which replication and failover model they operate, and confirm it works with Airflow's row-level locking.

Which database, executor and failover model to choose

This table compares the choices that determine which Airflow components can keep running when another one fails: the metadata database engine, how it is hosted, the executor, and where the API server and DAG processor run. Read each row for when the option fits and what it adds to daily operations. The main takeaway is that the database and executor decide where redundancy exists, so settle them before sizing anything else.

Decision	Option	When it fits	Operational cost
Metadata database engine	PostgreSQL 14 to 18 or MySQL 8.0, 8.4 or Innovation releases	Production deployments, including more than one scheduler using database row-level locks	Engine licensing and patching are yours or your provider's, and version support must match the Airflow prerequisites
Unsupported or test-only databases	MariaDB (unsupported) or SQLite (testing only)	Local experiments only, never a production metadata store	Using them in production risks data loss and blocks a supported upgrade path. Confirm with the prerequisites page before any build
Database hosting model	Self-managed PostgreSQL or MySQL with replication and failover you build	Teams with database administration skills and a tested failover runbook	Your team owns replication, backups, failover testing and version upgrades, and failover time depends on your design
Database hosting model	Managed database service with provider-run failover	Teams that want failover and patching handled by a provider under their own cloud account	Recurring service fees, and you must confirm the provider's failover behavior and supported engine versions in writing
Executor	LocalExecutor	Small deployments where tasks can run on the scheduler host	Little infrastructure to run, but tasks share the scheduler host, so there is no separate worker redundancy
Executor	CeleryExecutor	Long-lived worker pools on virtual machines with predictable capacity	Adds worker nodes and a message broker you operate, and you must plan redundancy for both
Executor	KubernetesExecutor	Teams already running Kubernetes that want each task isolated in its own pod	Requires a supported Kubernetes cluster (1.30 to 1.35 for Airflow 3.3.2), cluster capacity planning and image management
Executor	EdgeExecutor, AWS Batch or ECS executor	Edge sites, or workloads already run on AWS Batch or ECS	Depends on the network path or AWS services you use. Confirm feature fit and failure behavior with the executor documentation
Placement of API server and DAG processor	Separate processes, each run as a separate component on its own host or pod	Any production deployment, since Airflow 3 replaces the webserver with an API server and requires a separate DAG processor	Each process needs its own redundancy plan, monitoring and sizing, and both must reach the metadata database

Building Apache Airflow high availability in the right order

This sequence builds Airflow high availability in dependency order: pin the version, place the metadata database, separate the components, add redundancy, then confirm that every node runs identical DAG code. Each step ends with a check, so you know what is verified before you move on. New builds should start on Airflow 3.

Pin the Airflow 3 patch release
Choose one Airflow 3 patch release from the version life cycle table and one Python version from the supported range of 3.10 to 3.14, then record both in your build standard. The current patch release is 3.3.2. New builds should start on Airflow 3 because Airflow 2 reached end of life on April 22, 2026 and gets no fixes. Existing Airflow 2 estates must be on Airflow 2.7 or later before upgrading, so move to the latest 2.x release first. Done when a clean environment reports the pinned version and every node is built from the same image or package set.
Provision the metadata database for failover
Provision PostgreSQL 14 to 18, or MySQL 8.0, 8.4 or an Innovation release, as the metadata database using the failover model your team has chosen, such as a managed service or a replicated cluster. MariaDB is not supported, and SQLite is for testing only and can lose data in production. Ask the database owner how failover changes the connection endpoint and whether a failover interrupts in-flight scheduler transactions. Done when every Airflow host connects to the writer endpoint and a forced failover drill in staging lets Airflow reconnect without manual edits.
Install each component on its own host
Place the scheduler, API server, DAG processor, triggerers and workers on separate hosts or pods so that losing one host does not take out a whole function. The production guidance recommends at least 4 GB of RAM per host. If you run Kubernetes, confirm the cluster version is within the supported range of 1.30 to 1.35. Keep configuration in one shared source so every component reads the same settings and the same database connection string. Done when each component starts, reports healthy in its logs, and points at the same metadata database.
Start two or more schedulers
Start two or more schedulers against the same metadata database. Airflow uses database row-level locks so that schedulers share scheduling work between them. Confirm that both appear as live in the database or your monitoring stack. Then run a test DAG with many small tasks and check the scheduler logs to see that both instances create task instances. Done when stopping either scheduler leaves scheduling running on the other, and no task instance stays queued after the test completes.
Deploy API servers and the DAG processor
Run two or more API server instances started with the airflow api-server command, since Airflow 3 replaces the webserver with the API server. Put them behind a load balancer, and run the DAG processor as its own component, which Airflow 3 requires. Confirm with the documentation for your exact version whether and how more than one DAG processor can run. Configure load balancer health checks against the health endpoint your version provides. Done when the UI and API respond correctly through the load balancer while any single API server instance is stopped.
Add triggerers and workers
Add triggerers to run deferrable operators, and add workers for your chosen executor. Airflow allows several executors in one deployment from version 2.10.0, but the hybrid CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported from Airflow 3.0.0, so choose executor families deliberately for each worker pool. LocalExecutor runs tasks on the scheduler host, so it adds no worker redundancy. Done when a deferred task resumes after you restart one triggerer and tasks are picked up by more than one worker.
Confirm DAG delivery matches on every node
Confirm that schedulers, the DAG processor, API servers, triggerers and workers all read the same DAG code, whether it arrives through git-sync, a shared volume or a container image. After each deployment, compare the DAG list and file checksums on every node, and run one test DAG on each worker. Check the production deployment guide for your version to confirm the recommended container approach. Done when every node reports the same DAG versions and a DAG change appears on all nodes within the same deployment cycle.

How to test that scheduler and database failover really work

The point of a failover test is to prove, before go-live, that each failure you designed for produces the behavior you wrote down. Airflow high availability depends on several moving parts: more than one scheduler coordinating through database row-level locks, one or more API server nodes behind a load balancer, workers that pick up tasks from the executor, and a metadata database that can switch to a standby. Each part should have a named test with a pass condition, so the team is checking a specification rather than watching a system and hoping it behaves.

Start with the scheduler. While a DAG with several scheduled tasks is running, stop one scheduler process or node. The acceptance criterion is that the remaining schedulers keep creating and queuing task instances, and that no DAG run or task instance is created twice. Check the DAG run history and task instance list for duplicates after the test. Then stop an API server node while someone loads the UI and calls the REST API through the load balancer. The criterion is that requests continue to succeed or fail over within the time your load balancer health checks allow. Record the exact behavior you observe, because the timing depends on your load balancer settings, not on Airflow alone.

Next, kill a worker in the middle of a task. The result depends on your executor. With CeleryExecutor, a worker process killed mid-task may leave the task in a state that the scheduler detects and handles through its own timeout and retry logic. With KubernetesExecutor, the pod termination path differs. Write down the retry count you expect, whether the task is marked failed, up for retry, or left running, and whether the retry lands on a different worker. Confirm these expectations against the Airflow executor documentation for your executor and version, and against your own task retry and timeout settings, rather than assuming Airflow behaves the same way across executors.

Last, perform a database switchover to the standby and watch reconnection. Confirm that schedulers, API servers and workers reconnect without manual restarts, or document the restart step if one is required. Check the state of tasks that were running during the switchover, including whether any were lost, reported as failed, or resumed. Test with a realistic workload, not an idle cluster. Make the test repeatable: store the acceptance criteria as a runbook, run it before go-live, and run it again after every Airflow upgrade and every change to the database or load balancer. Your database vendor's failover documentation and your platform team's own switchover procedure define the steps, so treat those as inputs to the test plan.

How the high availability design changes by deployment model

This setup runs every Airflow component on one host, usually with the default SQLite metadata database. It suits evaluating DAGs and the interface. It is not high availability, and it should not carry production workloads.

Component placement: The scheduler, DAG processor, API server, triggerer and executor share one machine, so a single host failure stops the whole deployment.
Database hosting: SQLite on local disk. The production deployment guide warns that the default SQLite backend is for testing only and that running it in production can lead to data loss.
DAG delivery: DAG files sit in one local folder with no second copy to fail over to.
Worker scaling: Limited to the resources of the single host.
Extra work: Little during the test, but the environment must be rebuilt on PostgreSQL or MySQL and redesigned for multiple hosts before any production use.

Before reusing this host for anything beyond testing, confirm that the metadata database has been moved off SQLite and that backup and restore have been tested on the target database.

What drives Apache Airflow high availability effort and cost

This table lists the factors that change how much design, build and verification work a highly available Airflow deployment needs, and which way each one pushes cost. Read each row for its direction and the practical step that keeps it manageable. No fees or durations are given here, because a fixed-scope quote follows from the written scope.

Driver	Why it changes effort or cost	How to keep it down
Number of DAGs and task volume	More DAGs and tasks raise scheduler load, metadata database writes and worker capacity needed to keep schedules on time. Failover testing also covers more running work.	Confirm the current DAG count and peak concurrent tasks before design. Size schedulers and workers from measured load rather than estimates, and add a second scheduler only when load and recovery goals justify it.
Metadata database hosting choice	A self-managed PostgreSQL or MySQL cluster needs replication, failover tooling, backups and patching owned by your team. A managed service shifts much of that work to the provider but adds service terms to review.	Reuse database HA tooling and runbooks your organization already operates. Confirm supported PostgreSQL 14 to 18 or MySQL 8.0, 8.4 or Innovation releases with the Airflow prerequisites before choosing. MariaDB and SQLite are not supported for production metadata.
Executor and worker capacity	Celery needs a broker and worker pools that can survive node loss. Kubernetes executors depend on cluster capacity and scheduling policy. Running several executors in one deployment adds configuration and test paths.	Pick one executor for the workload where possible. Standardize worker images and resource profiles. Test worker loss during a representative run before go-live.
Number of environments (dev, test, production)	Each environment that must mirror production HA multiplies infrastructure, configuration and test effort. Low-stakes environments can usually run a smaller topology.	Scope which environments need full HA. Keep development and test on a reduced topology with the same Airflow version and configuration pattern, so behavior carries over to production.
Integrations and secrets backend	Each external system, connection and secrets store must stay reachable during failover, and each needs its own credential rotation and access testing. A central secrets backend adds a dependency that must also be highly available.	Inventory connections and secrets before design. Use one secrets backend with a documented recovery path. Limit integrations in the first phase to those the highly available path must support.
Required failover recovery time	Shorter recovery time targets need more standby capacity, automated detection, and more frequent failover testing. Longer targets allow manual steps and smaller standby footprints. The target must be set by your business owners.	Ask the client's service owner to document the acceptable recovery time for each component. Design only to that agreed number, and separate scheduler, API server, triggerer and database targets where they differ.
Compliance evidence needs	Assessors may ask for proof of failover tests, backup restores, access records and change history. Producing that evidence adds recurring effort. DeployOpen prepares the systems, controls and evidence, but independent auditors issue any assessment result.	Agree the evidence list with your compliance lead before build. Automate log export and test records where possible. Confirm which control frameworks apply with your auditor.
Existing team skills	Teams already experienced with PostgreSQL clustering, Kubernetes operations and Airflow 3 upgrades need less onboarding. Gaps in these areas usually add training, documentation and longer handover work.	Map the skills available in your team against the components in scope. Ask for runbooks, handover sessions and knowledge transfer to be included in the written scope.

What to monitor, back up and upgrade after go-live

Once a highly available Airflow deployment is live, most of the risk sits in the operating routine. Monitoring should cover each component separately, because a healthy API server can hide a scheduler replica that has stopped scheduling. Track scheduler heartbeats and the health of every scheduler replica, not only whether one of them is running. Watch API server availability behind the load balancer, and check triggerer and worker health, since deferred tasks and queued work stop silently when those processes fail. On the database, track replication lag on the standby and the number of open connections from schedulers, API servers, triggerers and workers, because connection exhaustion is a common cause of partial outages.

Back up the metadata database and prove the restore works. The metadata database holds DAG runs, task state, connections and variables, so a backup that has never been restored is an assumption, not a control. Schedule regular backups with point-in-time recovery if your database platform offers it, and run restore tests into a separate environment on a fixed cadence. Record the measured restore time and the data loss window, then compare both with the recovery targets your organization has agreed for this platform. Confirm with your database team which backup tooling and retention rules apply to the environment.

The backup plan must also include the Fernet key and connection secrets. Airflow uses the Fernet key to encrypt connection passwords and other sensitive values stored in the metadata database. A restored database without the matching key cannot decrypt its connections, so the key needs its own storage, access control and rotation procedure, kept separate from the database backups. If you use an external secrets backend, confirm that its own recovery path is documented and tested as well. Ask your security team who holds the key and how a key rotation would be rolled out across every replica.

Track the version life cycle from the start. The Airflow support policy gives limited-maintenance releases security and critical bug fixes only, and end-of-life releases get no fixes. Airflow 2 entered limited maintenance on October 22, 2025 and reached end of life on April 22, 2026, so any deployment still on Airflow 2 is now running without fixes. Set a calendar review against the version support table at least once per quarter, and record which patch release each component runs. The upgrades topic covers minor upgrades, provider updates and what a support agreement includes.

Finally, document the topology, failover runbooks and owners as part of go-live, not afterward. The document should list every scheduler, API server, triggerer, worker pool and database node, the load balancer and DNS entries, and the steps for failing over the database and restarting a stuck scheduler. Name an owner for each component and an escalation path. Run the failover runbook at least once in a non-production environment and update it with what you learned. Illustrative example: a runbook that has not been rehearsed often fails at the step that assumes a credential is still valid.

Securing redundant Airflow components and their shared state

Redundant Airflow components share one metadata database, one set of secrets and one codebase. The controls below keep those shared pieces consistent and reachable only by the components that need them. Each card names one control and what to check, and the Apache Airflow security hardening topic covers the full treatment.

Limit metadata database access

Schedulers, workers, triggerers, the API server and the DAG processor all connect to the metadata database, so it should accept connections only from those components' subnets or security groups. Workers often run in less trusted zones than the scheduler, so they need the tightest rule. Check the allowlist from each component's actual network position, and confirm that no component reaches the database through a public address. Verify the component list against your deployment's architecture before you lock the rules down.

Terminate TLS at the load balancer

In Airflow 3 the API server starts with the airflow api-server command and usually sits behind a load balancer that handles TLS and health checks. Encrypt client traffic to the load balancer, and decide whether traffic between the load balancer and each API server replica is also encrypted. Pair this with your chosen auth manager, since only one auth manager can be active at a time. Confirm that every replica enforces the same auth configuration, and that health checks reach replicas without exposing the UI to unauthenticated callers.

Keep the Fernet key identical

Airflow uses a Fernet key to encrypt connection passwords and variables stored in the metadata database. Every scheduler, worker, triggerer, API server and DAG processor must hold the same key, or one component will fail to read values written by another. Confirm that the key comes from one managed source and that a new replica receives it at startup. Before anyone changes the key, check the Airflow documentation on how existing encrypted values are handled.

Align the secrets backend across nodes

If you use an external secrets backend, every component must resolve connections and variables against the same backend, with the same paths or prefixes and the same access policies. Otherwise a DAG can succeed on one replica and fail on another after a restart or scale-out event. Test lookups from each component type, not only from the scheduler. Confirm which identity each component uses to reach the backend and that those identities are granted only the secrets they need.

Run one patched image everywhere

Schedulers, workers, triggerers, API servers and DAG processors should run from the same tagged image, because mismatched Airflow or provider versions across replicas cause failures that are hard to trace. Pin the image by digest in the deployment and confirm that a rollout replaces every component together. Track the version life cycle as well, since limited-maintenance releases receive only security and critical fixes and end-of-life releases receive none.

Questions teams ask before building Airflow high availability

How much effort and cost does high availability add to an Airflow deployment?

High availability adds effort through duplicated components, shared state and failover testing, and the size of that addition depends on your scope. Apache Airflow is licensed under Apache-2.0 with no paid edition, so the cost sits in infrastructure, services and people.

Main drivers:

The number of schedulers and whether they share one metadata database, since scheduler redundancy relies on database row-level locks
Metadata database high availability, whether a managed service or a self-run PostgreSQL or MySQL cluster
Redundant API servers, a separate DAG processor and triggerers
Executor choice and worker capacity
DAG delivery, log storage, monitoring and alerting

Recurring costs include compute, the database service, storage and the staff who patch and upgrade. Ask for a quote that separates one-time build work from recurring infrastructure and support.

How long does building Airflow high availability take?

The timeline depends on how quickly your environments, database and identity dependencies are ready, and on how many components must be proven to fail over. No duration is fixed here, so ask for a timeline tied to the written scope.

Schedule risk usually comes from waiting on access to database instances, network paths, DNS and certificates. Delays also come from agreeing on the executor and DAG delivery model, and from running failover tests in an environment that matches production. Moving DAGs from Airflow 2 adds code checks and rewrites, which should be scoped separately. Confirm which milestones the quote includes, who signs off each one, and how a client-side delay is handled.

What must our team provide before high availability work starts?

Your team must provide access, decisions and named people before the build begins. Expect to supply:

Database access: an existing PostgreSQL or MySQL instance with high availability, or permission to create one, plus backup and restore paths
Network: routes and firewall rules between schedulers, API servers, workers, the database and the message broker if CeleryExecutor is chosen
Identity: an identity provider, group structure for the auth manager and service accounts
Environments: development, test and production environments with comparable sizing
Named owners: one each for infrastructure, database, security, identity, and the business owner who accepts failover test results

Missing owners cause the most delay, because decisions on executor, secrets and DAG delivery stall without them. Confirm each item against your own environment before the quote is signed.

Can our internal team build Airflow high availability without an outside partner?

Yes, an internal team can build it if it already runs Airflow 3 or a comparable scheduler, administers PostgreSQL or MySQL, and can test failover in a non-production environment. The harder part is usually not installation but the ongoing work: upgrades, provider updates and failover retesting after each change.

Check these before deciding: whether anyone has run more than one scheduler against one metadata database, who owns the database failover runbook, whether the team can handle the Airflow 3 split between the API server and the DAG processor, and how many people can cover on-call duty. A partner fits best when the timeline is short, skills are thin, or you want a fixed-scope deliverable with documented tests. Ask any partner to show how it will test scheduler and database failure in your environment.

Does Airflow high availability require Kubernetes?

No. Schedulers, API servers, triggerers and workers can run on virtual machines or in containers, and Kubernetes is one option among several. The official Helm chart and the production guidance both favor container images when you do run on Kubernetes.

The choice mainly affects the executor. CeleryExecutor needs a message broker and worker hosts, KubernetesExecutor needs a cluster, and LocalExecutor runs tasks on the scheduler host, which limits how far it can scale out. If you use Kubernetes, the Airflow 3.3.2 prerequisites list Kubernetes 1.30 to 1.35. Confirm with your platform team that your cluster falls in that range and can reach the database and broker endpoints you need.

Which Airflow version should we start on for high availability?

Start on Airflow 3, which was first released on April 22, 2025, and use the current patch release, 3.3.2, unless your own testing points to a different patch. Airflow 2 reached end of life on April 22, 2026, so it receives no fixes and is a poor base for a new build.

Airflow 3 changes the components you deploy. The webserver is replaced by an API server started with airflow api-server, and the DAG processor must run as its own component. Task code can no longer use database sessions directly and must use the REST API or Task SDK. The Simple auth manager is the default, and only one auth manager can be active at a time. Before building, confirm the Python and database versions in the prerequisites for your chosen patch, and confirm that existing DAGs have been checked against the upgrade rules.

Which PostgreSQL and MySQL versions does Airflow 3 support for metadata?

For Airflow 3.3.2, the prerequisites list PostgreSQL 14 to 18 and MySQL 8.0, 8.4 and the Innovation releases. MariaDB is not supported as the metadata database, and SQLite should not be used in production.

The production deployment guide recommends PostgreSQL or MySQL. Its default SQLite backend is for testing only, and running it in production can lead to data loss. High availability depends on the database as much as on the schedulers, so choose a version your database platform still supports and that your failover method handles, whether that is a managed service, replicas or clustering. Confirm with your database team which failover features apply to that version, and verify the choice against the exact Airflow patch you plan to run, since prerequisites change between releases.

What should a fixed-scope quote for Airflow high availability include?

A fixed-scope quote should state what is built, what is tested, what the client provides and what is excluded, all agreed in writing before work starts. Ask for these elements:

Components and counts: schedulers, API servers, DAG processor, triggerers, workers and the chosen executor
Database: the metadata database, its version, and whether failover is configured or only prepared
Environments and deliverables: which environments are built, with runbooks and configuration documentation
Failover test cases: scheduler, API server, worker, triggerer and database failure, with pass criteria agreed in advance
Client dependencies: named owners and the milestones they affect
Exclusions: DAG rewrites, migrations, provider work and support after go-live
Fees and payment milestones: plus the assumptions behind them

Ask how changes to scope are priced. DeployOpen plans and implements on fixed-scope terms, with scope, deliverables and timelines agreed in writing before work starts, and it does not publish a price list, so the quote follows from the written scope.

How DeployOpen scopes, builds and hands over Airflow high availability

DeployOpen plans, implements, documents and supports self-hosted Apache Airflow inside your own cloud accounts, data centers or air-gapped networks. For high availability, we start from your workload and failure requirements, then build and verify redundancy for each Airflow component before handing the system over.

How we scope

Map which components need redundancy: schedulers, API servers, DAG processors, triggerers and workers
Confirm the metadata database choice and whether the target PostgreSQL or MySQL version is on the supported list
Agree in writing which failover tests, documentation and handover items are in the fixed-scope deliverables

How we build

Configure two or more schedulers against a shared metadata database that supports row-level locking
Deploy redundant API servers and a separate DAG processor, with a load balancer or ingress you control
Set up database backup and failover, then test scheduler, worker and triggerer failure with documented results

How we hand over

Deliver architecture diagrams, runbooks for failover and recovery, and the configuration used in each environment
Walk your platform and on-call engineers through restarting components and checking scheduler health
Record known limits, such as database failover behavior and any component left single-instance by design

How we support

Offer ongoing support, upgrades and testing as a retainer after go-live, scoped separately from the build
Review Airflow release and end-of-life dates, including Airflow 2 reaching end of life on April 22, 2026, against your plan
Re-run failover tests after Airflow, database or Kubernetes upgrades to confirm redundancy still holds

Apache Airflow high availability