Apache Airflow security hardening

Airflow encrypts sensitive values before it writes them to the metadata database. Connection passwords, connection extras and variables marked as sensitive are stored as ciphertext, and the Fernet key is the secret that encrypts and decrypts them. The key is set through the fernet_key option in the [core] section of the Airflow configuration, or through the matching AIRFLOW__CORE__FERNET_KEY environment variable. Treat it as a root credential for everything Airflow stores: anyone who holds it and can read the metadata database can recover every stored secret in plaintext.

The risk runs in both directions. If the key is lost, the encrypted values in the database cannot be decrypted, so connections and variables must be re-entered by hand after recovery. If the key leaks, an attacker with a database backup or read access to the metadata database can decrypt those values offline. Protecting the key therefore matters as much as protecting the database itself. Create it with the cryptography library's Fernet generator, which the Airflow documentation describes, and generate it in the environment where it will be used rather than copying one from a shared document or example.

Keep the key out of the container image, the DAG repository and any file committed to version control. Inject it at runtime from the secrets store your platform already uses, such as a Kubernetes Secret populated by an external secrets operator, a cloud key management service, or the vault your organization runs. Confirm with your security team who can read that store, whether access is logged, and how the key is backed up. A backup of the key should live in a separate control domain from the database backup, so one compromise does not expose both.

Every component that reads encrypted values must hold the same key. In a deployment with separate scheduler, API server, DAG processor, triggerer and worker processes, a component started with a different key will fail to decrypt connections it needs and the failure may only appear at task runtime. Verify this by listing your components and their environment sources, then running a test task that uses a stored connection after each configuration change. Also confirm how your chosen version handles multiple keys, because rotation depends on it.

Rotation needs a written plan before the first key change. Airflow's documentation describes accepting a list of Fernet keys, where the first key encrypts new values and the others remain available for decryption, which allows re-encryption of stored values before an old key is retired. The exact steps, the command used to re-encrypt, and any restart order are version-specific, so confirm them against the Airflow release you run in its documentation. Rehearse the full procedure on a copy of the metadata database and a non-production environment first, record the time it takes, and keep the previous key available until every stored value has been verified.

The clearest way to test Airflow hardening is to turn each control into a check that someone can rerun, then run those checks against the deployed environment rather than relying on configuration files alone. Start with the metadata database. Confirm that the connection string points to PostgreSQL or MySQL on a supported version, and that no scheduler, API server, worker or DAG processor falls back to SQLite. The Airflow prerequisites state that SQLite should not be used in production and that MariaDB is not supported as a metadata database, so any component that reports either one has failed the control. Query each running component directly, because a value set in one image or environment file can differ from the one another component reads.

Next, test secrets at rest. Create a connection with a recognizable dummy password through the interface or API, then read the metadata database table with a privileged database account. The stored value should be ciphertext and should not be readable as plain text. Restore a copy of the database into a separate environment where the Fernet key is absent, and confirm the values remain unreadable. Also check that the Fernet key is injected from your secret store at runtime and does not appear in image layers, container logs or the values file in your deployment repository. Record the key version in use so that rotation tests can be compared against it later.

Role tests need one test user for each role in the auth manager you run in production. Airflow 3 uses the Simple auth manager by default, and only one auth manager can be active at a time, so confirm which one is set before writing any test. For each user, attempt the actions that role should not have, such as triggering a DAG, editing a connection, reading a variable or calling the REST API with that user's token. Each attempt should return a denial and produce an audit entry. For workers, run a network probe from a worker pod or host to a destination outside the allowlist and confirm the connection fails. Then probe an allowed destination, so you know the test is not simply blocking all traffic.

Test fail-closed behavior in staging by pointing the secrets backend at an address that does not respond, triggering a task that needs a connection, and confirming that the task fails. It should not fall back to a value in an environment variable, a cached copy or a default. Then scan the image for baked-in credentials. Search the filesystem, environment metadata and build history for private keys, tokens and passwords, and confirm that nothing from the build context was copied in unintentionally. Use a secret scanning tool, and record its name and version so the scan can be repeated with the same settings.

Finally, prove that the tests can fail. In a staging copy, remove one control on purpose, such as the worker egress allowlist, then rerun the matching probe and confirm that it now reaches the blocked host and the test reports a failure. Repeat the exercise with a second control, such as the metadata database setting. A suite that has never failed against a broken configuration gives little assurance. Record every result as evidence: the command or query, the environment, Airflow and image versions, the date, the tester, the output and the reviewer's approval. Store the record with the change ticket so that your team or an independent auditor can rerun the same checks after each upgrade.

Hardening is not finished at go-live. The recurring work falls into three areas: reviewing logs, keeping images and packages current, and keeping documentation accurate enough that someone other than the builder can prove each control still works. Start by setting a review schedule for audit and access logs and naming who reads them. Airflow records user actions in its audit log, and access to the API server and the metadata database produces its own records in your logging stack. Decide in advance who reviews each source, how often, what counts as an exception, and who approves a change to a role or connection. A log that nobody is assigned to read provides no control, even if it is retained correctly.

Patching has two tracks. The first is the Airflow core and its runtime image. Each new patch release should trigger a rebuild of your container image from the official Airflow image, a re-run of your tests, and a redeploy on a set cadence. The second is provider packages, which are versioned separately from core and often carry the connection and hook code that touches your secrets. Keep a dependency manifest that pins each provider version, and review the release notes for each update. Confirm the support status of your installed minor line against the version life cycle table before each planning cycle. Airflow 2 entered limited maintenance on October 22, 2025 and reached end of life on April 22, 2026. Limited maintenance means only security and critical bug fixes. End of life means no fixes at all, so a deployment still on 2.x after that date carries known exposure with no upstream remedy.

The documentation set should cover five items, each of which doubles as audit evidence. First, the Fernet key custody process: where the key is stored, who can read it, how it is backed up, and how a key rotation is performed without losing access to encrypted connection values. Second, the role matrix, listing each Airflow role, the people or groups mapped to it, and the last review date. Third, the network rules for the scheduler, workers, API server and metadata database, including which ports are open between them. Fourth, the image build pipeline, covering source of base images, scanning steps, signing, and promotion to production. Fifth, the rotation runbook for database credentials, connection secrets and API tokens, with the steps, owners and rollback plan.

For each control, record the evidence it produces and where that evidence lives. A role review produces a signed list of memberships. A rebuild produces an image digest, a scan report and a deployment record. A rotation produces a change ticket and a before and after check that old credentials fail. Ask your auditor or assessor which evidence formats they accept before you design the collection process, since that decides whether you need exports, screenshots or pipeline artifacts. Confirm each item against your own environment and your vendor documentation rather than relying on a template. Keeping this record current is the work that most often lapses after go-live, so assign it an owner and review it on the same cycle as the patch plan.

Finally, review the whole set at least once per Airflow minor release and after any change to executors, auth managers or the metadata database. Airflow 3 changed several security-relevant components, including the API server, the DAG processor and the default Simple auth manager, so a control written for Airflow 2 may no longer describe what runs. Treat each upgrade as a trigger to re-check the role matrix, the network rules and the rotation runbook, and to record that the re-check happened. This keeps the documentation tied to the running system instead of to the day the hardening project closed.