Why public sector data pipelines need a self-hosted design first
An agency or public body that must move case, benefits or registry data between systems and publish open data usually needs self-hosted Apache Airflow for scheduling and Apache NiFi for flow-based ingestion and controlled transfer. These two tools fit most agency estates because they run inside the agency's own accounts or data centers, and the hosting boundary and records duties decide the design before any connector is chosen. Start with three questions: where the data may physically sit, which systems are allowed to hold copies, and how long each copy must be kept. Those answers decide where Airflow schedulers, NiFi nodes, staging stores and logs can run, and which authorization boundary they fall inside.
Commercial pipeline services often assume a shared multi-tenant cloud and a contract that covers processing abroad, which many agency data-handling rules do not allow for criminal justice, benefits or tax records. A self-hosted design lets the agency keep credentials, intermediate files and provenance records under its own controls, and it lets the team show an assessor exactly where each data set went. Confirm with your security and records officers which federal, state or national frameworks apply, since the answer determines whether a cloud region, a government cloud or an on-premises site is acceptable. Ask any implementation partner to map each flow to those boundaries in writing.
Version choice matters from the first day. Start new work only on NiFi 2 and Airflow 3. NiFi 1.x cannot be maintained further because it depends on Jetty 9.4, Spring Framework 5.3 and AngularJS 1.8, and Airflow 2 reached end of life on April 22, 2026, after which it receives no fixes. Confirm the current patch levels on the Apache NiFi download page and the Airflow version life cycle page before you fix a baseline, and plan upgrade windows into the project schedule rather than treating them as later work.
Which rules shape the pipeline build
This table maps each rule that often governs public sector data movement to the build decision it forces on a self-hosted pipeline. Read each row as a question to answer for your agency, because which rules apply depends on your data, your hosting model and whether a federal or state authority sets the requirement. The main takeaway is that hosting location, key length, log retention and portal accessibility are set by the rule, not by the orchestrator choice.
| Rule | Build consequence | Confirm with |
|---|---|---|
| NIST SP 800-53 baselines and FISMA oversight | Select the low, moderate or high baseline (149, 287 or 370 controls and enhancements) and map pipeline controls such as access, audit and configuration to it. Inspectors General may test those controls, so evidence must be retained per system. | Agency authorizing official and information security office |
| FedRAMP scope under OMB M-24-15 | A cloud-hosted pipeline service that stores or processes federal information for an agency may fall in scope. A system run only for one agency on its own cloud infrastructure, not offered as a shared service, is generally out of scope. Decide the hosting model before choosing tools. | Agency FedRAMP or cloud program office |
| FBI CJIS Security Policy v6.1 (effective 26 June 2026) | CJI in transit and at rest outside physically secure locations needs FIPS 140-3 validated modules or AES with at least 256-bit keys. Vulnerability scanning must run at least monthly. Configure NiFi TLS, Airflow connection encryption and backup encryption to meet this. | State CJIS Systems Officer and the agency's CJIS auditor |
| 32 CFR 2002.14(h)(2) for CUI on non-federal systems | Where a federal agency shares controlled unclassified information with a contractor or partner system, requirements follow NIST SP 800-171 unless the CUI category sets other safeguards. Pipelines that move CUI to partner systems inherit these safeguards. | Agency CUI program manager and the CUI category authority |
| ADA Title II WCAG 2.1 Level A and AA for published portals | Open data portals and public dashboards fed by pipelines must meet WCAG 2.1 A and AA from 26 April 2027 for entities serving 50,000 or more people, and from 26 April 2028 for smaller entities and special districts. Add accessibility testing to portal release steps and cover contracted content. | Agency ADA coordinator and legal counsel |
| NIS2 applicability for EU public administration | NIS2 covers central government entities regardless of size and lists public administration as highly critical. Member States may extend it to local bodies, and it excludes national security, public security, defense and law enforcement. Applicability drives incident reporting, supply chain and logging duties for pipeline operators. | National NIS2 competent authority and agency legal team |
| EU AI Act logging for high-risk AI fed by pipeline data | If pipeline outputs train or feed a high-risk AI system, Article 12(1) requires automatic event logging over the system's lifetime, and Article 19(1) requires at least six months of retention unless other law sets longer. Keep lineage and run logs in the same retention scope. | Agency legal team and AI system provider |
Where hosting rules change the architecture
Federal agencies face FedRAMP when a cloud service creates, collects, processes, stores or maintains federal information on the agency's behalf. Under the scope rules in OMB M-24-15, that test covers the hosting service, so an orchestrator, its metadata database and its staging storage are in scope if they hold federal data in a cloud environment. A system used only for one agency's operations on cloud infrastructure, and not offered as a shared service, can fall outside scope. The control baseline for the system follows the impact level set in its security categorization under NIST SP 800-53B.
- Inside the boundary: Pipeline runtimes, NiFi flow files, provenance repositories, logs and staging copies all hold federal information and belong in the authorization boundary.
- Provider authorization: If a cloud provider hosts any of these components, confirm that its service carries FedRAMP authorization at the impact level the agency requires.
- Single-agency exclusion: A self-run pipeline platform used only by one agency may sit outside FedRAMP scope, but the agency must document that determination.
- Shared platforms: A platform that serves several agencies or external customers is more likely to be treated as a shared service and in scope.
- Baseline selection: Low, moderate and high baselines in SP 800-53B select different control sets, so the security categorization drives the work.
Ask the compliance office for the written scope determination and a boundary diagram that lists every component before choosing a hosting model.
The FBI CJIS Security Policy sets minimum requirements for criminal justice information (CJI). Version 6.1 took effect on June 26, 2026, with a Requirements Companion Document that builds on the control structure of version 6.0. Any pipeline that moves CJI, including intermediate files, error queues and log output that contain CJI, has to meet the policy for those copies as well as the source system. Agencies and their contractors often underestimate how many copies a pipeline creates.
- Encryption strength: Version 6.1 requires FIPS 140-3 certified modules or FIPS 197 AES with at least 256-bit keys for CJI in transit, and at rest outside physically secure locations. The earlier 128-bit minimum no longer applies.
- Vulnerability scanning: The policy calls for vulnerability scanning at least monthly, mapped to control IDs in the SC and RA families, so scanning covers orchestrator hosts, workers and database servers.
- Operational copies: NiFi provenance data, Airflow task logs and dead-letter queues can hold CJI and need the same access control and encryption as the primary store.
- Hosting and third parties: Confirm with the agency that administers CJIS access how the policy applies to any cloud hosting, managed database or support vendor that can reach CJI.
- Version control: Confirm which version of the policy the agency has adopted and whether the companion document changes any requirement for your design.
Ask the agency for the list of systems that touch CJI and the version of the policy it will audit against before you design the data flows. The California DOJ bulletin on CJIS Security Policy version 6.1 is a useful starting point for the changes.
When a federal agency shares controlled unclassified information (CUI) with a contractor, grantee or other non-federal party, the agency's requirements determine the safeguards. Under 32 CFR 2002.14(h)(2), agencies use NIST SP 800-171 to set those requirements unless the CUI category's authority prescribes other safeguards. A pipeline run by a contractor that carries CUI therefore inherits the contract's safeguard standard, and the hosting choice decides which parts of the contractor's environment are assessed.
- Default standard: Unless the category's authority says otherwise, NIST SP 800-171 is the expected baseline for CUI on non-federal systems.
- Defense contractors: Under the CMMC program rule, a contractor pursuing Level 2 may store or process CUI in a cloud service only if that service is FedRAMP Authorized at the Moderate baseline or higher, or meets equivalent requirements.
- Non-cloud providers: A service provider that is not a cloud service is assessed within the contractor's own scope, so support vendors with access to the pipeline can bring their own environments into scope.
- Self-hosted scope: Running orchestrators inside the contractor's own assessed environment keeps CUI within a known boundary, but that environment must meet the assessment requirements in full.
- Category markings: Some CUI categories name their own safeguards, so the marking and its authority must be checked for each data set.
Ask the contracting officer for the CUI categories in scope and the safeguard standard written into the contract before selecting any hosting option.
The NIS2 Directive covers central government entities regardless of size and lists public administration as a sector of high criticality. Member States may extend it to local public administration. The directive does not apply to entities working in national security, public security, defense or law enforcement, and those exclusions are where many sensitive public pipelines sit. Whether a given pipeline falls under NIS2 depends on the entity, its mandate and the national law that transposes the directive.
- Central government: Central government entities fall in scope regardless of size, so their data pipelines carry NIS2 risk management and incident reporting duties unless an exclusion applies.
- Local administration: Municipalities and regional bodies are covered only where the Member State has extended the directive, so the national transposition must be checked.
- Security exclusions: Entities working in national security, public security, defense or law enforcement are excluded, which can move a pipeline out of NIS2 while bringing it under other national rules.
- Cross-agency exchange: Regulation (EU) 2024/903, the Interoperable Europe Act, may shape how agencies exchange data, so counsel should confirm whether it applies to the flows.
- Hosting location: NIS2 focuses on security and risk management, so hosting location is usually set by national rules and agency policy and should be confirmed separately.
Ask counsel for a written NIS2 status for each entity in the pipeline and a list of national hosting requirements before the architecture is fixed.
Which open source platforms fit agency pipelines
Agency pipelines usually need three things: a scheduler that can prove when and how each job ran, a controlled way to move data between zones, and a catalog that shows where published data came from. The platforms below cover those roles, and each has version, license or support conditions that should be checked before selection.
Apache Airflow for scheduled batch work
Airflow fits recurring batch jobs such as nightly case extracts and open data refreshes, with schedules, retries and run history kept on the agency's own servers. Airflow 2 reached end of life on April 22, 2026, so any new build should start on Airflow 3.3.2 and any existing Airflow 2 estate needs a migration plan. Confirm the metadata database (PostgreSQL or MySQL, not SQLite) and the executor model before design sign-off.
Apache NiFi for controlled transfer
NiFi suits ingestion from agency systems and transfers between network zones, where each hop must record what moved, when and from where. Its provenance records support audit questions about specific files and records. Use NiFi 2.x: the project states that 1.x cannot be maintained further, and migration from 1.x requires stepping through 1.27.0 and converting flow.xml.gz to flow.json.gz. Confirm the authentication method, since the Kerberos provider is deprecated.
Dagster for asset-oriented pipelines
Dagster models pipelines as data assets with declared dependencies, which helps when agencies need to show which tables a published dataset was built from and when each was last refreshed. It is a reasonable alternative to Airflow for teams that want asset lineage built into the orchestrator. Compare the team's skills and the number of existing Airflow jobs before choosing, because running two orchestrators adds operational and audit scope.
Airbyte licensing for connectors
Airbyte is often used for connectors to SaaS case tools and databases, but the platform repository is licensed under the Elastic License 2.0, which restricts providing the software as a managed service to third parties. An agency running it for its own use should have counsel confirm the license terms against its procurement and any contractor arrangements. Confirm which connectors are community or certified and whether they receive security updates.
Apache Superset for published dashboards
Superset is Apache-2.0 licensed and can serve dashboards to staff or public portals from data the agency already holds in its warehouse. Role-based access and row-level security must be configured per dataset so that restricted case or personal data does not appear in public views. Test the access model separately from the pipeline, because a correct load does not prevent an over-broad dashboard.
DataHub or OpenMetadata for lineage
Both DataHub and OpenMetadata are Apache-2.0 catalogs that record ownership, lineage and classification across pipelines, which supports records retention reviews and inter-agency data sharing agreements. Choose based on which connectors cover the agency's source systems and warehouse, and confirm how each captures lineage from Airflow and NiFi jobs. Lineage that depends on manual documentation will not hold up under audit.
How an agency rollout moves from inventory to operations
The sequence turns an open-ended pipeline idea into a scoped, classified and evidenced platform, with each phase ending in an output the agency can review and approve before the next one starts.
Inventory systems, datasets and data classification
DeployOpen works with the agency's data owners, records and security staff to list the source systems, case management tools, open data portals and partner agency exchanges in scope. Each dataset is tagged with its sensitivity, retention obligation and owner, including whether it holds criminal justice information, controlled unclassified information or personal data. The output is a data and system inventory with a classification for each flow. The agency approves the inventory and the classification before any design work begins.
Scope hosting and authorization boundaries
With the inventory in hand, the agency's security, legal and procurement teams confirm where the platform will run and which authorization regime applies. This covers the agency's own data center, a national or government cloud, or a cloud environment that the agency controls, and whether any hosted service is in the FedRAMP scope. DeployOpen documents the system boundary, the external connections and every place data may be stored or cached. The output is a signed boundary diagram and a list of open authorization questions for the agency's authorizing official to answer.
Map controls to the chosen baseline
The agency selects the control baseline that governs its systems, such as a NIST SP 800-53 baseline, the FBI CJIS Security Policy or a state framework, and DeployOpen maps each control to a concrete platform setting, process or log source. Typical areas include access control, audit logging, encryption of data in transit and at rest, vulnerability scanning and change management. Gaps are logged with an owner and a decision. The output is a control mapping workbook that the agency's security team accepts, with any accepted risks signed by the responsible official.
Prove provenance and retention on one pilot flow
DeployOpen builds one representative pipeline end to end, for example from a case management export to an internal analytics store, with orchestration in Airflow or NiFi. The pilot records lineage and run-level provenance, applies the agency's retention rule and shows how a failed run is detected and replayed without losing records. The agency's records officer and auditors review the lineage output and the retention job logs. The output is a pilot evidence pack and a go or no-go decision recorded by the agency.
Cut over to production with runbooks
Once the pilot is approved, the remaining flows move to production in an order the agency sets, with each flow promoted through the same change approval path. DeployOpen delivers runbooks for scheduling, failure handling, backfills, credential rotation, schema change handling and interagency exchange, and tests each runbook against the live environment. Rollback steps are rehearsed before the switch. The output is a production cutover record listing each flow, its approval, its test result and its runbook version.
Hand over operations and evidence
DeployOpen transfers configuration, source code, monitoring dashboards, access lists and documentation to the agency's operations team and trains the staff who will run the platform. The agency confirms which tasks stay in-house and which go to a support retainer, if one is chosen. Control evidence is packaged so that the agency's assessors can trace each control to a live setting and a log source. The output is an operations handover package and a signed acceptance by the agency, after which the platform is run under the agency's own change and incident processes.
Evidence auditors and inspectors ask for
This table maps each piece of evidence an assessor is likely to request to the pipeline component that produces it and the control family it supports. Use it to plan what the platform must record from day one, not as a statement that any specific audit will accept it. Auditors, inspectors and certification bodies draw the conclusions; DeployOpen prepares the systems, configuration and records they review.
| Evidence | Produced by | Control family or requirement | What to confirm |
|---|---|---|---|
| Provenance records for each transformation | NiFi provenance repository and Airflow task logs, with lineage metadata in a catalog such as DataHub or OpenMetadata | Audit and accountability, system and information integrity | Retention period of provenance data and whether it covers CJI or CUI fields |
| Airflow run history and failure handling | Airflow metadata database, task retry settings and alert routing | Contingency planning, incident handling | Metadata database is PostgreSQL or MySQL, not SQLite, and backups are tested |
| NiFi flow versions and change approvals | NiFi Registry or a Git-backed flow registry, with approval records in the change system | Configuration management, change control | Who can promote flows to production and how approvals are linked to versions |
| Encryption configuration for CJI or CUI paths | TLS settings in NiFi and Airflow, key management, and storage encryption configuration | CJIS SC-13 and SC-28; NIST SP 800-171 for CUI on non-federal systems | Modules are FIPS 140-3 validated and keys meet the 256-bit requirement in CJIS v6.1 |
| Access logs from NiFi and Airflow authentication | NiFi user and authorization logs for single-user, LDAP, OpenID Connect, SAML 2.0 or X.509 login; Airflow auth manager logs | Identification and authentication, audit logging | Which provider is in use, that logs are forwarded to a central store, and that retention matches policy |
| Vulnerability scan results | Scanner output for hosts, containers and the Airflow and NiFi versions in use | CJIS RA-5 (at least monthly scanning), vulnerability management | Scan cadence, remediation timelines agreed with the agency, and evidence of closure |
| Retention schedules and deletion records | Retention rules configured in storage, warehouse TTL settings and pipeline cleanup jobs | Records management rules set by the agency or applicable law | Which records schedule governs each dataset and who signs off on disposal |
| Version support status | Platform inventory showing Airflow and NiFi versions against vendor support dates | System maintenance and flaw remediation | Airflow 2 reached end of life on April 22, 2026, and NiFi 1.x can no longer be maintained, so confirm the agency accepts the upgrade path |
What do public sector pipeline projects cost and require?
What drives the cost of a public sector data pipeline project?
The biggest cost drivers are the hosting boundary, the number of source systems, the depth of records and lineage, the authorization work, and any public portal accessibility testing. Software licenses for Apache Airflow, Apache NiFi and Dagster are $0 because they are Apache-licensed, so the spend goes to people and infrastructure. Each cost component is paid for separately:
- Infrastructure in the agency's own cloud account, data center or government cloud (recurring)
- Engineering to build connectors, transformations and monitoring (one-time, then changes)
- Security documentation, testing and authorization support (one-time, then periodic)
- Support and upgrades under a retainer (recurring)
Ask any partner for a quote that itemizes these lines against a written fixed scope.
Can agency staff run a self-hosted pipeline platform themselves?
Often yes, but only after the agency has staff who can operate the platform day to day. Airflow and NiFi are both maintained by their projects, and the Airflow production deployment guide recommends PostgreSQL or MySQL rather than the default SQLite backend, which signals that a production setup needs real operations skills. Agencies that lack that skill set usually combine in-house ownership of data and approvals with a partner for setup, hardening and upgrades. Confirm the skills gap during scoping.
What does the agency need to provide for the project?
The agency needs named data owners who can confirm what each source contains and who may see it, the security approvals needed to connect systems and hosts, and representative test datasets that do not expose protected records. Schedule time from the people who approve change, because pipeline DAGs and flows often need the same review as other production changes. Ask the partner to list these dependencies in the written scope before work starts.
How long does a public sector pipeline project take?
Duration depends on the number of sources, the hosting boundary, and how many approvals sit in the path, so no fixed timeline applies to every agency. Scope the work in phases: one or two pilot feeds first, then additional sources, then portal publication and retention rules. Ask the partner for a timeline tied to deliverables and to the approval steps the agency controls, and confirm which assumptions it depends on.
What compliance work adds time and cost to a government pipeline?
Authorization and control evidence add the most work when data includes criminal justice information, controlled unclassified information or cloud-hosted federal data. The FBI CJIS Security Policy version 6.1 took effect on June 26, 2026 and requires FIPS 140-3 certified modules or FIPS 197 AES with at least 256-bit keys for CJI at rest and in transit outside physically secure locations. Under FedRAMP, a single agency's own system on cloud infrastructure may fall outside scope, so confirm the scope determination with the agency's authorizing official before costing the work.
What recurring work follows a public sector pipeline after launch?
Recurring work includes platform upgrades, dependency patching, vulnerability scanning, connector maintenance and periodic review of access and retention rules. Airflow 2 reached end of life on April 22, 2026 and gets no fixes, and NiFi 1.x can no longer be maintained, so plan version upgrades as a standing budget line. CJIS v6.1 requires vulnerability scanning at least monthly for systems in scope. Ask for a retainer scope that names upgrade windows and scan cadence.
Does DeployOpen provide compliance certification for a pipeline?
No. DeployOpen plans, implements, hardens and documents self-hosted pipelines and prepares systems, controls and evidence for compliance assessments, but independent auditors and certification bodies issue audit reports, attestations and certifications. Pricing follows a written fixed scope, and DeployOpen does not publish a price list, so the agency should request a quote after the scope is agreed.
What to ask a pipeline partner before a public sector build
Start by asking how the partner draws the compliance boundary before any connector is built. Ask which datasets are criminal justice information under the FBI CJIS Security Policy, which are controlled unclassified information, and which are ordinary public records, because each class sets different hosting, encryption and scanning duties. Ask who decides whether a cloud component falls within FedRAMP scope, and request the written reasoning, including whether the platform is offered as a shared service or used only for one agency's operations. Ask how they verify that encryption meets the CJIS Security Policy version 6.1 requirements for CJI in transit and at rest, and how monthly vulnerability scanning is evidenced for the Apache NiFi and Apache Airflow hosts.
Ask how the partner will move existing flows and DAGs to supported versions. Apache NiFi 1.x cannot be maintained further, and the migration path requires moving to NiFi 1.27.0 before 2.0.0, changing flow.xml.gz to flow.json.gz, and updating expression language and cache component names. Apache Airflow 2 reached end of life on April 22, 2026, so ask how Airflow 2 DAGs are tested against Airflow 3.3.2, which hybrid executors have been removed, and whether a rollback path exists. Ask which authentication providers they configure for NiFi (single-user, LDAP, OpenID Connect, SAML 2.0 or X.509) and how the Airflow auth manager maps agency roles to least-privilege permissions. Ask how provenance and retention are tested: request evidence that lineage records survive a replay, that retention jobs delete on schedule and that deletions are logged.
Ask how the partner monitors pipelines without SaaS tooling, including where metrics, logs and alerts are stored and who receives them when a job fails or a source goes silent. Ask for the on-call model and the escalation path, and confirm which parts are paid as a retainer and which are fixed-scope. For open data portals and case management front ends, ask how accessibility is tested against WCAG 2.1 Level A and AA, which the ADA Title II rule requires for state and local government web content, and whether testing is manual, automated or both. Finally, ask what evidence they hand over at the end of the build, such as architecture diagrams, configuration baselines, test records, runbooks, exit plans and documentation sufficient for a new team to operate the pipelines without the partner.
Deployment patterns for agency networks
An agency pipeline platform is judged on where each component runs, who can reach it, and how its state survives a failure. The patterns below cover the parts that most often decide whether a self-hosted Airflow and NiFi estate passes a security review and stays available.
NiFi cluster coordination
A NiFi cluster needs a coordination layer for state and leader election. Choose between an embedded ZooKeeper, an external ZooKeeper ensemble, or Kubernetes using ConfigMaps or Leases. Confirm which option your operations team can patch, back up and monitor inside the boundary. Also confirm that NiFi binds to 127.0.0.1 when no security configuration is provided, so the UI is not reachable until authentication is configured deliberately. Plan that configuration before the first node is exposed to agency users.
Airflow metadata store and schedulers
Airflow keeps its state in a metadata database, and that database is where run history, task outcomes and audit evidence accumulate. Use PostgreSQL 14 to 18 or MySQL in production, and never SQLite, which the project documents as test-only and a risk of data loss. Airflow supports more than one scheduler at once using database row-level locks, which removes a single point of failure. Confirm that the database is backed up, replicated and restorable within the agency's recovery objectives.
Executor choice for isolated workloads
The executor decides how tasks are isolated and scaled. KubernetesExecutor runs each task in its own pod, which suits mixed workloads that touch different classification levels. CeleryExecutor uses a shared worker pool, which is simpler to operate but requires tighter control over which workers can reach which data. Since Airflow 2.10.0, several executors can run in one deployment. Hybrid CeleryKubernetesExecutor and LocalKubernetesExecutor are no longer supported as of Airflow 3.0.0, so do not design around them.
Key management for CJI
Where criminal justice information moves through a pipeline, the CJIS Security Policy calls for FIPS 140-3 validated modules or AES with at least 256-bit keys for data in transit and at rest outside physically secure locations. Apply that rule to pipeline secrets, connection credentials, NiFi sensitive parameters and Airflow connections, not only to database volumes. Ask the agency's CJIS Systems Security Officer which key store and module validations it accepts before selecting a vault or HSM.
Zone-crossing gateway for exchange
Exchanges with other agencies or with a less-trusted zone should pass through one controlled gateway rather than direct connections from source systems. In NiFi, this is typically a dedicated site-to-site or secured HTTPS listener with OpenID Connect, SAML 2.0 or X.509 client certificate authentication, plus content inspection and logging at the crossing point. Confirm which identity provider the partner agency trusts and who approves each new flow that crosses the boundary.
Monitoring that stays inside the boundary
Pipeline monitoring often sends logs, metrics and alerts to a SaaS tool, which can move metadata about sensitive datasets outside the boundary. Keep metrics, log aggregation, alerting and the Airflow and NiFi audit trails on infrastructure the agency controls. Set retention for those logs to match the agency's records schedule and confirm who can read them. Treat alert payloads that name datasets or record counts as sensitive data in their own right.
How DeployOpen scopes and supports agency pipelines
DeployOpen plans, implements, hardens, documents and supports self-hosted Apache Airflow and Apache NiFi inside the agency's own data center, air-gapped network or authorized cloud account. The work starts with a written scope that names the source systems, case management and open data destinations, the network zones each flow crosses, the orchestrator and executor model, and the people who approve changes. Most engagements are fixed-scope, so deliverables and timelines are agreed before work begins. Agencies should expect the scope document to state which environment holds the Airflow metadata database, which holds NiFi flow definitions, and where credentials are stored, because those answers drive the rest of the design.
Evidence is where agencies often need the most clarity. DeployOpen prepares systems, controls and evidence for compliance assessments, such as configuration baselines, flow and DAG change records, access role mappings, and logging and retention settings for NiFi provenance and Airflow task history. DeployOpen does not issue audit reports, attestations or certifications. Independent assessors and authorizing officials make those determinations, so the scope should name the assessor and the control families the agency needs evidence for before implementation starts. Ask the partner to confirm in writing which artifacts will be delivered, in what format and to whom.
After go-live, the agency's operating risk moves to upgrades and continuous testing. NiFi 1.x is no longer maintainable, and Airflow 2 has reached end of life, so a retainer that covers version planning, patch windows and regression testing of key flows is worth pricing alongside the initial build. DeployOpen offers ongoing support, upgrades and continuous testing as retainers, which can be scoped separately from the implementation. Agencies should ask how the retainer handles security patches, how staging is kept in step with production, and how the partner's testing results are delivered to the agency's own security team.
