Apache Airflow for financial services

SEC guidance names four IT general control areas that management may need to evaluate for a SOX Section 404 assessment: program development, program changes, computer operations, and access to programs and data. For an Airflow estate, those areas map onto DAG code promotion, scheduled job records, run logs, and who can change the scheduler and its configuration. The guidance (Release 33-8810) also says management only needs to evaluate the ITGCs necessary for the proper and consistent operation of controls that address financial reporting risks, so the first task is to identify which DAGs feed financial statement balances. A nightly ledger reconciliation DAG is in scope. A marketing attribution job usually is not. Agree that list with your auditors and your finance controllers before you design anything else.

Because no SEC rule or PCAOB standard publishes a fixed IT general control catalog, the control set is yours to define and your external auditor will test against it. A practical design treats each in-scope DAG as a change item. Code lives in a version-controlled repository, changes go through review, and only tagged versions reach the production Airflow environment. Airflow 3 separates the DAG processor from the API server and scheduler, so the promotion pipeline should deploy DAG bundles from an approved source rather than letting anyone copy files onto a running component. Confirm with your team how DAG delivery is configured in your chosen deployment model before you write the control narrative.

On the operations side, evidence comes from three places: the metadata database that records DAG runs and task instances, the task logs your retention policy keeps, and the audit trail of who changed connections, variables and user roles. Restrict write access to the scheduler, the metadata database and the secrets backend to a small operations group, and separate that group from DAG authors. Confirm how your chosen auth manager maps roles, and whether your retention settings keep run history long enough for the period an auditor samples. Record the answers in the control narrative, because the auditor will ask for them.

DORA has applied since January 17, 2025 to 20 types of financial entity, from credit and payment institutions to insurers, and it applies directly without national transposition. For an Airflow deployment, the duties that matter most sit in Chapter V on ICT third-party risk. Article 28(3) requires a register of information covering every contractual arrangement for ICT services, so the platform's open-source components, hosting providers, managed database services and any partner that installs or supports Airflow each need a recorded entry. Article 28(8) requires documented and tested exit strategies and transition plans for ICT services that support critical or important functions. A payment or month-end reconciliation DAG will usually fall into that category, so the exit plan should name the Airflow metadata database, DAG repository, secrets references and the scheduling logic that would have to move if the platform changed.

Article 30(2) requires every ICT services contract to state where services are provided and data processed, to give advance notice of changes, and to cover data return on insolvency or termination. When Airflow runs in your own accounts, the location question is largely answered by your own infrastructure, but a partner's support access, remote logging or a hosted container registry still needs to be written into the contract. Backups of the metadata database (PostgreSQL or MySQL, not SQLite), DAG bundles and configuration should be stored in locations named in the agreement, and restore tests should be scheduled and recorded. The restore test is the evidence that turns a backup into a recovery capability. Confirm with your compliance team which of these items your competent authority expects in the register and in the exit plan, since the regulation itself does not prescribe a format.

DORA sets no certification scheme. Nothing in the Regulation lets a vendor or a partner certify that an Airflow deployment is DORA compliant, and the only authority-issued attestation the Regulation provides for is that a threat-led penetration test met its requirements under Article 26(7). Treat any claim of DORA certification for a platform or a service as a warning sign. What your team can produce is evidence: the register entries, the tested exit plan, backup and restore records, and change records for DAG and platform upgrades. Supervisors assess that evidence against the Regulation, so the reader should ask a delivery partner to show how each artifact is produced, who owns it after handover, and how it is kept current as Airflow releases move on.

DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. For an Airflow deployment in a bank, insurer or payment company, that means DeployOpen can configure the scheduler, metadata database, DAG delivery path, secrets backend and access model so they can be examined, and can organize the records an assessor will ask for. The opinion itself comes from the assessor the organization engages, whether an external auditor, a Qualified Security Assessor or the internal audit function. Ask early which assessor will test each control and what format they expect the evidence in, because that decides how the deployment should log and retain change and access records.

PCI DSS scope is the first question to settle when cardholder data passes through a pipeline. The standard applies to entities that store, process or transmit cardholder data or could affect the security of that environment, so a DAG that moves card transactions into a reporting warehouse, or a worker that can reach the cardholder data environment, can bring the Airflow hosts, their database and their secrets into scope. The payment brands decide who must validate and the assessment type, a Report on Compliance by a Qualified Security Assessor or a Self-Assessment Questionnaire, and each produces an Attestation of Compliance. Confirm the assessment type with your acquirer or the relevant brand before designing network segmentation, because the answer changes how much of the Airflow estate must be tested. The PCI Security Standards Council FAQ on enforcement explains why the Council itself does not make this call.

Swift users face a separate attestation. Under the Swift Customer Security Controls Framework, the controls must be independently assessed and the organization attests compliance each year in the KYC-SA application, and Airflow hosts that initiate or handle Swift messages fall within that framework. Confirm with your Swift security officer which architecture components are in scope, then decide whether Airflow workers that touch payment files run in a segregated zone with their own credentials and logs. DeployOpen can prepare the evidence for that zone, including access reviews and job change records, but the attestation is the institution's own statement, submitted through Swift and supported by the independent assessor.