How Airflow masking works and when you need to plan for its gaps
If your connections and variables hold passwords and tokens, Airflow masks them in task logs, the Variables UI and rendered fields, but only for values it reads through Connections or Variables and only when the name matches its keyword list. The masking is name-based: Airflow looks for sensitive-looking names such as password, secret, token, api_key and private_key, and extra keys with those names are masked too. The sensitive_var_conn_names setting extends that list with your own names, which is the first thing to review if your teams use names like dsn, auth_header or client_cert. Apache Airflow is licensed under Apache-2.0, so you can read the masking code in the repository and test its behavior in your own version before you rely on it.
Masking is a display and log control. It does not encrypt anything at rest and it does not restrict who can read a value. Anyone with permission to open a connection or variable in the UI, or to run code that reads it, can still see the plaintext. Passwords stored in the metadata database need the Fernet key and your secrets backend decisions, which are covered in other sections. Masking also depends on timing: mask_secret must run before any output is produced, so custom code that prints a value it fetched earlier leaks it before masking can apply.
Plan for the gaps now. Values passed through XCom, and values passed as environment variables, for example to KubernetesPodOperator, are not masked by this mechanism. A pipeline that pulls a token from a Connection and then pushes it into XCom or into a pod spec has moved it outside the masked path. Confirm with your own tests which paths in your DAGs carry secrets, then decide whether to use a secrets backend read by the worker, Kubernetes Secrets mounted as files, or redesigned tasks that never pass credentials between steps. Add a check to code review that flags print, logging and XCom calls that touch secret-named values, and test a sample run to confirm what appears in the logs.
Which values Airflow masks by default
Airflow's default masking works in four places: task logs, the Variables UI, rendered fields and sensitive connection extra keys. It decides what to hide by matching names, not by inspecting values, so the names your team chooses determine what stays protected.
Task logs
Airflow replaces matching secret values with masked text when they appear in task log output. This covers values read through Connections and Variables that the masking logic recognizes. Check that a test DAG prints a known sensitive value and confirm the log shows it masked before you rely on this in production. Do not assume that masking covers output your own code writes through other paths.
Variables UI
Variables whose keys match a sensitive pattern show their values masked in the Variables screen of the web interface. Anyone who can view the Variables list still sees the key and any unmasked fields, so the masking is a display control, not an access control. Review who holds read access to Variables in your RBAC model.
Rendered fields
Template fields rendered for a task instance in the UI are masked when the field name or the underlying value is recognized as sensitive. This is where a password passed as an operator argument can surface after templating. Open the rendered view for a task that uses a credential and confirm that it does not show the plain value.
Sensitive connection extra keys
Connection extra fields with names that match a sensitive pattern are masked, in addition to the connection password. Provider-specific extra keys can carry tokens or private keys under names you may not expect. List the extra keys each of your connection types uses and check each one against the default patterns.
Matching on names
Matching uses the name of a key or field, not the content of the value. The default patterns include password, secret, token, api_key and private_key. A value stored under a neutral name such as credential_blob or cfg is not masked, even if it holds a live token. Set a naming convention for sensitive items before the first DAG reads them.
Extending the keyword list
The sensitive_var_conn_names setting adds your own names to the default list, so internal naming such as a vendor-specific key name can be masked. Changes apply to new log output and UI views, and they do not rewrite existing logs. Confirm the exact configuration key and its format in the masking documentation for your Airflow 3 release.
Masking configuration options and when each fits
This table compares the four ways to control masking in Airflow: the built-in name list, sensitive_var_conn_names, mask_secret in task code, and renaming keys to match existing patterns. Read each row for scope, what appears in logs and the UI, and the case where it fits. The main takeaway is that none of these options covers values passed through XCom or environment variables, so those need separate controls.
| Option | Scope and effect on logs and UI | Typical case where it fits |
|---|---|---|
| Default sensitive name list | Masks connection passwords, sensitive variables and sensitive connection extra keys in task logs, the Variables UI and rendered fields when names match patterns such as password, secret, token, api_key and private_key | Teams whose credential names already contain these words and who read secrets only through Connections or Variables |
| sensitive_var_conn_names | Adds your own key names to the masking list, so matching Variables and Connection keys are hidden in logs and the UI; it does not change how values are stored | Internal naming conventions, such as a vendor-specific key name, that the default list does not catch |
| mask_secret in task code | Registers a value at runtime so it is redacted from task output; it must run before the code prints or returns anything, otherwise the value may already be in logs | Custom operators or scripts that fetch a token from an external API or a file and print or log intermediate values |
| Renaming variables or connection keys | Brings secrets under an existing masking pattern without changing code paths; the value itself is unaffected, and existing references must be updated | Legacy pipelines with inconsistent names where a rename is cheaper than extending the list |
| Values in XCom | Not covered by any masking option above; XCom values are stored in the metadata database and may appear in UI views | Confirm with your team that no credential is pushed to XCom, and use a secrets backend or a short-lived token pattern instead |
| Values in environment variables | Not masked when passed to tasks, for example to KubernetesPodOperator; the name list does not apply to them | Confirm the container or pod spec, restrict who can read it, and keep sensitive values out of task definitions where possible |
| Tasks that print secrets directly | mask_secret cannot redact output that was produced before it was called, so ordering in task code determines the result | Confirm with a test run in a non-production environment that the expected values do not appear in task logs |
Extending the keyword list with sensitive_var_conn_names
Airflow's built-in name matching covers common secret words such as password, secret, token, api_key and private_key, but your organization may store credentials under names that do not contain those words, such as db_dsn, vendor_signing_material or an internal naming convention. The sensitive_var_conn_names setting extends the list of names Airflow treats as sensitive, so values read through Connections and Variables that match your additions are masked in task logs, the Variables UI and rendered fields. Before you rely on it, confirm the exact configuration key, the expected format (a list of names, and whether matching is by substring or exact name), and the section of the configuration file or environment variable where it is set in your deployment. Check the masking documentation for the current form, and verify the setting is applied to the scheduler, workers and API server, since a value may appear in a log written by any of them.
Renaming a Variable or Connection late is risky. Masking only applies to values that Airflow reads through those objects, so a secret that was already written to logs before a name change stays in those logs, and existing DAGs that print the value through a different path are not covered by the new name. Plan the change as a deliberate step: inventory which Connections and Variables need the extra names, rename them in a staging environment first, and record which DAGs and tasks read them. Note that old log files and any copies shipped to a log store or SIEM will still contain the unmasked value, so the rotation of credentials that may have been exposed belongs in the same change plan.
Before production use, test that each new name is masked. Create a throwaway Variable and Connection in staging with the new names, run a small test DAG that logs the value, and confirm the task log and Variables UI show a redacted value rather than the plaintext. Repeat the test after every configuration change and after every upgrade, since masking behavior is part of the release you are deploying. Remember that this test covers only the Connection and Variable path. XCom and environment variables need their own controls, which the sections on those gaps cover.
Setting up masking and verifying it in logs and the UI
This sequence turns Airflow's built-in masking into a deliberate control by naming secrets so they match, extending the sensitive name list where needed, and proving with a test value that the masking holds in task logs and the UI.
Inventory secret-bearing connections, variables and extras
The platform team and the owners of each pipeline list every connection, variable and connection extra that holds a password, token, key or other credential. Include values used by operators, hooks and custom code, and note which ones are read through Connections or Variables. The inventory is done when every credential has an owner, a name and a source of truth, and when values that would reach Airflow only through XCom or environment variables are flagged separately for later controls.
Rename secrets to match masked name patterns
Review each name in the inventory against the built-in patterns, which cover names such as password, secret, token, api_key and private_key. Rename connections, variables or extra keys that hold credentials but use a name outside those patterns, and record the change in your naming convention. The check is a written list in which every credential name contains a pattern the default masking recognizes, or is marked for the extended list in the next step.
Extend the list with sensitive_var_conn_names
For credentials whose names cannot reasonably change, add them to the sensitive_var_conn_names option in the Airflow configuration, which extends the default masking list. Set the option through the environment or configuration management used for your deployment, not by editing images by hand, and confirm the exact key format against the masking documentation for your Airflow version. The check is that the scheduler and workers started after the change and that the configuration shows the new value.
Call mask_secret in custom code before any output
For custom operators, Python callables or scripts that fetch a secret from somewhere other than a Connection or Variable, call mask_secret with the value before the code prints, logs or returns it to the UI. Reviewers should check that the call happens first in the code path, because masking applies only to output produced after the value is registered. The check is a code review note for each such task confirming the call precedes any logging and that the function is covered by a test.
Run a test DAG with a known test value
Create a non-production DAG that stores a clearly fake test value in a sensitive variable and in a connection password, then prints both from a task using the normal Airflow access path. Add a second task that prints the same test value through a plain string to show the difference. Run the DAG in a staging instance, since test values should never reach production logs. The check is that the test run completes and the output is the planned result for each task.
Check task logs for the masked value
Open the task log for each test task in the Airflow UI and search for the fake test value. The value read through a Connection or Variable should appear as masked text, while the value printed through a plain string will appear unmasked, which confirms the gap that other controls must cover. Record the result, the Airflow version and the configuration used. The check is a saved log extract showing masked output for the sensitive path and unmasked output for the deliberate control case.
Verify the Variables UI and rendered fields
Open the Variables page and the rendered fields view for a task that uses the test secrets, and confirm that sensitive values are hidden in both places. Repeat the check for any extended names added in the earlier step, because those are the entries most likely to be missed. The sequence is complete when the test value is absent from the logs, the Variables UI and rendered fields, and the results are filed with the naming inventory so that new secrets are checked against the same procedure.
Using mask_secret in custom task code before output
Airflow only masks values it knows about. It learns those values when they are read through a Connection or a Variable, so any secret your own code obtains some other way, such as from a file, an HTTP call to a vault, or a value passed in a DAG parameter, is invisible to the masking filter until you register it. In custom operators and Python tasks, call mask_secret on that value as soon as it is loaded, and do it before any print, logger call, or exception message can contain it. The masking docs state that mask_secret must run before any output is produced. Once a secret has been written to a task log, the masking filter cannot retract it, and the value stays in the stored log until retention removes it. Treat the call as the first line after the secret is fetched, not a cleanup step at the end of the task.
Masking matches the exact string you register, so derived forms of a secret can slip through. A base64 encoding, a URL-encoded copy, a hash, a value split across two fields, or a concatenation of a secret with a prefix will not match the registered original. Register each form your code actually emits, or better, change the code so the secret never leaves the function that needs it. Also check exception handling: an HTTP client error or a stack trace often echoes request headers or URLs that carry the token, and those strings are produced outside your own print statements. Confirm with your Airflow version's documentation how the mask applies to library logging before relying on it for those paths.
Test masking with the output your code really produces, not with a clean string in a unit test. Run the task in a staging environment with the same operator, log level, and logging handler configuration used in production, then search the task log and the rendered fields in the UI for the secret and for its encoded variants. Include a failure path in the test, because error messages are where leaks usually appear. Record the check in your DAG review checklist so that a change to the operator or to a library version is retested. If a task must handle a value that cannot be masked reliably, such as one passed to a pod environment, move that value to a controlled backend instead of relying on log filtering, as covered in the sections on XCom and environment variables.
Gaps through XCom and environment variables
Airflow's masking covers values read through Connections and Variables in logs, the Variables UI and rendered fields. Four paths sit outside that coverage, so each needs a control of its own. The cards below cover what each path exposes and what to put in place before pipelines handle live credentials.
XCom values carry secrets unmasked
XCom values are stored in the metadata database and passed between tasks, and Airflow does not mask them. A task that pushes a token or password into XCom places it where anyone with read access to the metadata database or the XCom views can see it. Keep credentials out of XCom entirely, and have the consuming task fetch the secret from a secrets backend at run time. Review DAG code for xcom_push calls and return values that include credential fields.
Environment variables bypass masking
Values passed as environment variables, such as those given to KubernetesPodOperator, are not masked when Airflow records or displays them. The pod spec, the Kubernetes API and any tool that dumps the container environment can expose them. Prefer a secrets backend that the container fetches from at start, or mount a short-lived file from a Kubernetes Secret with restricted permissions. Confirm whether your pod specs or rendered template fields store the value.
Tools echo values inside tasks
A shell command, a CLI client or a debug flag inside a task can print a credential that Airflow never looked up as a masked value. The task log then holds the plain text, even though Airflow masked the original connection. Call mask_secret on any value your code obtains from other sources before the first print, and run tools in quiet mode where they support it. Test each operator's output with a dummy secret before production use.
Secrets written to files or external systems
A task that writes a token to a file, uploads a key to object storage or sends a password to an external API moves the secret outside Airflow's masking entirely. Those copies persist after the task ends and may be readable by other users. Limit writes to temporary paths with restrictive permissions, delete them in a finally block, and rotate any credential a task had to write out. Ask each data team to list every destination their DAGs write to.
Restrict who reads task logs
Masking reduces what appears in logs but does not stop a reader from seeing values that bypass it. Log access is therefore a control in its own right. Limit the Airflow roles that can open task logs, rendered fields and the Variables UI to the teams that need them. Ship logs to a store with its own access policy, and confirm retention against your log policy.
Questions teams ask before masking secrets in Airflow
Is Airflow's built-in masking enough to meet compliance requirements?
No. Masking is one control within a larger set, and it does not by itself satisfy any compliance framework. Airflow masks connection passwords, sensitive variables and sensitive connection extra keys in task logs, the Variables UI and rendered fields, based on name matching. Values passed through XCom or as environment variables are not masked. Auditors will look for the surrounding evidence: where credentials are stored, who can read them, how access is reviewed and how logs are retained. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors issue any assessment outcome. Confirm the exact requirements with your assessor before you rely on masking as evidence.
How much effort does auditing existing connections and variables take?
Effort scales with how many connections, variables and DAG files you have, and with how many of them hold values that are not named like secrets. The audit has three parts: listing every connection and variable, checking each name against the default masking keywords and your sensitive_var_conn_names extension, and reviewing DAG code for values that bypass masking, such as XCom payloads and environment variables passed to KubernetesPodOperator. Teams with a small, well-named inventory finish quickly. Teams with years of ad hoc variables, or DAGs that generate secrets at runtime, need more time. Ask for an inventory count and a sample review before agreeing on a timeline.
Do we need a consultant to configure masking, or can our team do it?
Your team can configure the masking keyword list and the sensitive_var_conn_names setting on its own, and the change is small. The harder part is judgment: deciding which values leak through XCom, logs from custom operators or environment variables, and designing the controls that replace masking for those paths, such as a secrets backend or a worker isolation model. A partner is useful when you lack capacity or want an independent review. Whether you engage DeployOpen or someone else, ask for the review scope and deliverables in writing before work starts, since most engagements are fixed-scope.
How long does the review take with many DAGs and connections?
Review time depends mainly on the number of connections and variables, the number of DAGs and custom operators that write output, and whether those DAGs use XCom or environment variables for sensitive values. Each custom operator needs to be read to see whether it calls mask_secret before any output is produced. Log shipping also adds work, because masked logs in the UI can still be exposed if a downstream system stores raw output. No duration can be stated without seeing your inventory. Ask any provider for a timeline based on your counts, and confirm which DAG repositories and environments are in scope.
Which custom code creates the most masking gaps?
Three patterns cause most of the gaps. The first is custom operators and Python callables that print or return secret values without calling mask_secret first, since masking must run before any output is produced. The second is XCom, which stores values outside the masking path. The third is environment variables, including those passed to KubernetesPodOperator, which are not masked. Each pattern needs a different fix: a code change for custom operators, a redesign that keeps secrets out of XCom, and a secrets backend or pod-level secret reference for environment variables. Review these patterns first in any audit.
What does log shipping change about masking?
Log shipping moves task output to a central system, and masking in Airflow does not follow it there. If a task writes an unmasked value, the value reaches the shipped log even when the Airflow UI hides it through name matching. Teams should confirm whether their log pipeline applies its own redaction, who can search the central store and how long logs are retained. Decide this before the review, because fixing a value in Airflow does not remove copies already shipped. Cost here comes from the logging platform's storage and access controls, which your team or the platform owner should size.
What must our team provide before an audit starts?
Prepare a few inputs. Provide read access to the Airflow deployment's connection and variable listings (not the values), the DAG repositories and the custom operator code, the Airflow version in use, and a list of the executors and log destinations in production. Name a contact who can explain how each DAG author team works. Confirm the environments in scope, including staging. Also state whether any compliance assessment is planned and which control expectations apply, so the review checks evidence that matters. Ask any provider to confirm this list in the written scope.
Can masking be extended to cover our own secret names?
Yes. Airflow matches names such as password, secret, token, api_key and private_key by default, and the sensitive_var_conn_names setting extends that list with names your organization uses. Add names that match how your teams label credentials, then test them on a non-production variable first. Keyword lists catch names, not values, so a value stored under a neutral name will still show. Use the naming review as one input to the audit, and confirm the exact configuration keys for your Airflow version in the masking documentation.
How DeployOpen scopes masking and related controls
DeployOpen plans and implements Apache Airflow inside your own infrastructure, whether that is your cloud accounts, your data centers or an air-gapped network, so the masking configuration, the secret stores and the logs it protects stay under your control. Scoping starts with an inventory of where credentials actually flow: which connections and variables hold passwords and tokens, which DAG files read secrets from environment variables or XCom, and which operators print values into task logs. That inventory decides how much of the work is configuration and how much is code review, and it is the part most worth doing before anyone edits the sensitive-name list.
Every engagement is delivered under a written, fixed scope that names the Airflow version, the executors and secret sources in use, the DAG repositories included in the review and the deliverables, such as a masking configuration, a list of exceptions with their owners and a test that proves a seeded secret does not appear in a task log or the Variables UI. Ask for the scope to state which DAG authors and teams are covered, and who signs off on exceptions for values that cannot be masked in Airflow. DeployOpen prepares configuration, evidence and documentation for your compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make those determinations. Ongoing work such as re-reviewing DAGs after new providers are added, or re-running masking tests after an upgrade, is available as a retainer rather than being assumed in the initial scope.
