Data consolidation after a merger

An ICFR scoping memo records which acquired business, data flows and systems management excludes from its internal control assessment, and why. The SEC staff allow exclusion of a newly acquired business, but not beyond one year from the acquisition date and not from more than one annual report, and the exclusion and its significance must be disclosed. The memo should list each in-scope financial data flow, its source and target systems, and the owner of each control over it. DeployOpen can prepare the system inventory, data flow diagrams and control mapping that feed the memo, but management owns the scoping decision and the disclosure. The SEC management ICFR FAQ is the primary reference to confirm the conditions with your auditors.

Auditors look for evidence that changes to pipelines and access to data are authorized, tested and reviewed. The SEC guidance names program development, program changes, computer operations and access to programs and data as example IT general control areas, and management only needs to evaluate the controls that address financial reporting risks. For a self-hosted pipeline estate, the usual evidence is change tickets linked to merged pull requests for DAG and flow code, approval records for production deployments, role assignments in Airflow or Dagster, and access review exports. The change tickets are the system of record; Git history and the CI pipeline logs supplement them. DeployOpen can configure these trails and prepare them for review, but the control design and the testing conclusion belong to your team and your auditors.

The orchestrator run history produces the execution record, and the reconciliation job produces the comparison result. In Airflow, task instance history records each run, its start and end times, retries and failures, and the reconciliation task can write row counts, control totals and mismatched keys to a table or file that is retained with the run. Dagster records asset materializations and check results for the same purpose. Exception records should capture each unmatched customer or product key, the rule that flagged it, the reviewer and the resolution date. Keep the reconciliation outputs in the organization's own storage with a retention rule that matches your record policy. Confirm the retention period with your finance controllers and auditors before the first close cycle.

OpenMetadata lineage and catalog exports can show where a figure in a financial report came from, but they are evidence of the data flow design, not proof that every run behaved as documented. OpenMetadata supports table, column, dashboard, pipeline and metric lineage with OpenLineage event integration, along with ownership, glossaries, classifications and data quality tests. Export the lineage graph, owner assignments and test results at each reporting period and store them with the period's close package. Lineage captured from scheduled runs should be compared with the run history so the reader can see both the design and the observed execution. Confirm with your auditors which lineage level, such as column level, they expect for the reports in scope.

A DORA register must list all ICT service arrangements, and for an acquired business it should include every data flow and third-party service that touches the merged estate. DORA Article 28(3) requires financial entities to keep this register at entity, sub-consolidated and consolidated levels. Article 28(8) also requires tested exit strategies and transition plans for services that support critical or important functions, which matters when interim pipelines between two warehouses are retired. For each arrangement, record the provider, the data categories, the locations of processing, the functions supported and the exit plan. The register can be partly generated from the catalog and pipeline inventory, but the entity remains responsible for its completeness. DeployOpen can help build the inventory and the exit plan documentation, while the competent authority and the entity's governance determine what the register must contain.

Records of processing must be updated when the merged estate changes what personal data is processed, where it is stored and who can access it from outside the EEA. GDPR Article 30 requires these records to describe the purposes, categories, recipients, transfers and, where possible, the Article 32 security measures. When a non-EEA acquirer's staff can access an EU target's systems, remote access counts as a transfer under EDPB Guidelines 05/2021, which requires a Chapter V transfer tool. Each pipeline's source, destination, transformation step and access group should map to an entry in the record. Catalog classifications and lineage exports help keep these entries current, but the controller must approve the updated record. Confirm the processing instructions for any processor with your legal team, since GDPR Article 28(3)(a) requires processors to act on documented instructions.

DeployOpen prepares the systems, controls and evidence that an assessment will examine, but independent auditors and certification bodies issue the opinions, attestations and certifications. In practice, DeployOpen configures Airflow run history, Dagster asset materializations, OpenMetadata lineage exports and change ticket integrations so they can be retrieved for review, and documents how each piece of evidence is produced. The auditor decides whether the evidence is sufficient for the controls in scope, and the registrant or regulated entity makes the disclosure or regulatory filing. Ask any implementation partner to name which evidence each deliverable produces and which party reviews it, and confirm the evidence list with your external auditor before the engagement is scoped in writing.

Retention periods come from your legal, regulatory and audit obligations, not from the orchestration tool. The platform can keep run history, logs and exports for as long as the storage budget and configuration allow, but the retention rule should be written by the data owner, legal team and auditors. Airflow's metadata database and log storage, Dagster's event storage and exported catalog snapshots each need an explicit retention setting and a backup plan. For records that may later support a regulatory or audit request, store exports in write-once or access-controlled storage in your own infrastructure. Confirm with your compliance team which records must be retained, for how long, and where they must be stored before the first run of the merged pipelines.

The cost is driven mainly by how many source systems and warehouses the two companies run, how many master-data conflicts exist between customer and product records, how deep the financial reconciliation must go, and how much of the estate falls in regulated scope. Each added system adds connector work, mapping rules and testing. Conflicting master records need business decisions on which attribute wins, which takes time from people who are not engineers. Reconciliation depth depends on how many ledgers, subledgers and reports must tie out for the close. Regulated scope adds evidence work for controls, change approval and access reviews. Ask the partner to show how each of these drivers maps to the line items in the quote.

Discovery, source mapping, master-data matching, interim pipeline builds, reconciliation design and the retirement plan for duplicate systems are one-time work. Platform run costs recur: compute and storage for the orchestrator, warehouse and catalog, backups, upgrades, security patching and on-call coverage for failed jobs. Catalog upkeep also recurs, because lineage, ownership and glossary entries go stale when teams change. Open-source components carry no license fee for Apache Airflow, which is licensed under Apache-2.0 with no paid edition from the project itself, but the hosting and staffing costs still apply. Ask the quote to separate one-time delivery from annual run costs, with the run costs stated per component.

Some organizations can, if they already run the orchestrator, warehouse and catalog in production and have engineers who know both source estates. The work is hard to do alone during a merger because the same people usually run the day-to-day close, handle integration decisions and support the acquired systems. Gaps typically appear in master-data decisions that need finance and business sign-off, in reconciliation evidence that auditors will ask for, and in on-call coverage for interim pipelines that must run until the old systems are retired. A useful self-assessment is to list the systems, name an owner for each, and check whether that owner has time for the next six months.

Finance needs to name owners for the reconciliation rules, approve how conflicting customer and product records are resolved, and confirm which reports must tie out for the close and for external disclosure. IT needs to provide read access to source systems, network paths between the two estates, service accounts with least privilege, and a change process for the new pipelines and data models. Both teams need to agree on a single decision log, because master-data choices affect billing, reporting and access rights. Decisions that wait for weeks are often the main cause of schedule slip, not the technical build.

The duration depends on scope: the number of systems in each estate, how many master-data conflicts need business decisions, how deep the reconciliation must go, and whether regulated reporting depends on the consolidated data. No duration is fixed in advance. For financial reporting, the SEC allows management to exclude a newly acquired business from its assessment of internal control over financial reporting, but not beyond one year from the acquisition date and not from more than one annual report, and the exclusion and its significance must be disclosed. Use that window to plan the control work, and ask the partner to propose a timeline tied to written milestones.

Regulated scope adds evidence and control work on top of the technical build. For SEC registrants, IT general controls over program changes, computer operations and access to programs and data matter only where they support controls over financial reporting, so scoping that link early reduces effort. DORA requires financial entities to keep a register of ICT service arrangements and tested exit strategies for services supporting critical functions, which the consolidation plan may affect. Where personal data moves across borders, a non-EEA acquirer's staff accessing EU systems can count as a transfer that needs a transfer tool. Each of these adds review, legal input and documentation.

A written quote should list the systems in scope for each estate, the deliverables for each phase, and the assumptions about master-data decisions, reconciliation depth and regulated controls. It should state who does each task, what the client team must provide and by when, and which items are fixed scope versus optional. Ask for the one-time delivery price separated from the recurring platform run, support and catalog upkeep, with the hosting assumptions stated. Require a retirement plan for duplicate systems as a deliverable, and ask how changes in scope are priced and approved. DeployOpen agrees scope, deliverables and timelines in writing before work starts and quotes from that written scope.

Check that the estimate starts from an inventory of both estates, not from a generic template. Ask how many source systems and warehouses are counted, how master-data conflicts are sized, and which reconciliation reports are covered. Ask which open-source orchestrator and metadata catalog the partner proposes and why, and whether the chosen versions are still supported. For Airflow, Airflow 2 entered end of life on April 22, 2026, so an estimate that builds on it needs a migration line. Confirm that the partner plans for backups, upgrades and on-call handover before the project closes.