Data pipelines in air-gapped networks

The main extra cost is the offline supply chain: every container image, package, connector and rule set has to be fetched on an internet-connected side, verified, moved across the boundary and loaded into internal registries and repositories. Each step adds engineering, review and approval effort that an online build skips. Other drivers include building and maintaining an internal mirror, designing the transfer process with the security team, and testing every update in the isolated environment before it reaches production. Ask any partner to split the quote into design, mirror and transfer tooling, connector and pipeline work, and recurring update cycles, so you can see which parts are one-time and which recur. Also confirm whether any commercial editions or connectors are needed, since air-gapped support is often a paid feature in other products.

The timeline is set by the number of sources, the approval path for moving files across zones, and how many network zones the data crosses. Work usually starts with an inventory and a transfer design, then moves to building the internal mirror, deploying the orchestrator and connectors, and running a full test cycle offline. Security review and change approval often take longer than the engineering itself, so start those early. No duration applies to every environment, so request a schedule from your partner that lists milestones, dependencies on your approvers and the scope agreed in writing before work begins.

A team with Linux, Kubernetes or container operations, database administration and orchestrator experience can run the platform once it is built, provided it has a documented update procedure and tested rollback. The harder skills are offline package management, transfer validation and keeping a mirror current, which many teams have not practiced. Check three things before deciding: whether at least two people can operate the orchestrator and the transfer path, whether on-call coverage exists for failed batch jobs, and whether someone owns the approval step for each update. A partner can build the platform, document it and provide support under a retainer if you want continuity after handover.

Most core orchestration projects are open source without a paid edition from the project itself. Apache Airflow and Apache NiFi are licensed under Apache-2.0, and Dagster is also Apache-2.0. Airbyte's platform repository is licensed under the Elastic License 2.0, which restricts providing the software to third parties as a hosted or managed service, so confirm how your use fits that wording with your counsel. Metabase is AGPL outside its enterprise directory, and its air-gapped deployment is offered on its Enterprise plan. Review each component separately, including connectors, because a connector's license can differ from the platform's license.

Updates follow a controlled transfer cycle: the upgrade is downloaded on a connected system, its checksums and signatures are verified, the artifacts are copied into the isolated zone through your approved transfer path, loaded into internal registries and package repositories, and tested in a staging copy before production. Apache Airflow 2 reached end of life on April 22, 2026, and gets no fixes, so plan migrations to the 3.x line. NiFi 1.x can no longer be maintained, so NiFi 2 is the line to plan for. Ask your partner for a written update calendar, a rollback procedure and an owner for each approval.

The engagement needs four inputs from your side:

• Network diagrams showing every zone, boundary device and approved transfer path
• A source and destination inventory with data classifications, owners and volumes
• A list of named approvers for transfers, changes and production access
• Access to existing registries, identity systems and monitoring so the build can reuse them

For utilities or industrial sites, also provide the operational technology boundary rules, since NERC CIP-005-7 requires interactive remote access to pass through an Intermediate System with multi-factor authentication. Agree these inputs and the scope in writing before work begins.

Yes. DeployOpen plans, implements, hardens, documents and supports self-hosted open source platforms inside the client's own infrastructure, including air-gapped networks, meaning the client's own cloud accounts, data centers or isolated environments. The work covers the orchestrator, connectors, metadata store, registries used for offline images and packages, and the monitoring stack. Before any work begins, the team confirms which source systems, destinations and network zones are in scope, because an isolated enclave changes almost every design decision, from how images arrive to how alerts leave the network. Ask any partner to show which parts of the stack run without outbound connections and how they were tested.

Most DeployOpen engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. For an air-gapped pipeline, a useful written scope names the platforms (for example Airflow, Dagster, Airbyte, Kafka or NiFi), the source and destination systems, the transfer points between zones, the offline update process, the monitoring and alerting targets, the hardening baseline, the documentation set and the handover acceptance criteria. It should also state what the client must provide, such as hardware, approved software intake, network diagrams and named owners. Anything not listed is a change request, so agree on how changes are priced and approved before signature. Illustrative example: a scope that omits the offline package intake process will leave the team unable to patch after go-live.

DeployOpen does not publish a price list, and a quote follows from the written scope. The main drivers are the number of platforms and pipelines, the number of source systems and connectors that must be built or validated offline, how many network zones and transfer points exist, the depth of hardening and testing required, and the documentation and evidence the client expects. Costs fall into one-time work (design, installation, hardening, initial pipeline build, testing and handover) and recurring work (upgrades, offline update cycles, monitoring, and support). Ask for the quote to separate these, to list assumptions about the environment, and to state what happens if the client's network team changes the transfer process mid-project.

Many organizations can, provided they have skills in the chosen orchestrator, Linux and container operations, and the offline update process, and they have staff on call. The open source platforms in this stack carry no license fee, but the labor to operate them is real. Apache Airflow and Apache NiFi are Apache-2.0 projects with no paid edition from the project itself, so the cost of running them is staff time, infrastructure and any support contract the client chooses. A DeployOpen retainer covers ongoing support and upgrades, and a fixed-scope project can end with a documented handover so internal staff take over. Decide early which model fits, because an upgrade calendar that nobody owns is the most common reason pipeline stacks drift out of support.

Ongoing support and upgrades are available as retainers. For offline pipelines, that usually means reviewing vendor release notes, preparing and testing new images and packages in a staging enclave, moving them through the approved transfer process, applying the upgrade in a planned window, and checking that connectors, DAGs and flows still run. Support can also cover monitoring tuning, access reviews, backup restore tests and incident response. The support scope should state response expectations in writing, which DeployOpen would define in the agreement rather than assume. Ask whether the retainer includes a version support review, because upstream end-of-life dates, such as the Airflow 2 end of life on April 22, 2026, force upgrades whether or not anyone planned them.

DeployOpen runs security testing across applications and APIs, cloud and infrastructure, AI systems and agents, and red teaming. For an air-gapped pipeline stack, that testing can target the orchestrator web interface and API, authentication configuration, secrets handling for source credentials, the transfer and import path between zones, container images and their provenance, and the exposure of monitoring endpoints. Test plans should be agreed in writing, with clear rules of engagement and a remediation process. Note that testing can only confirm what is in scope on the test date, so re-testing after upgrades is a separate item to include in the scope or retainer.

DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those assessments. Evidence typically includes architecture and data-flow diagrams, the list of offline packages and images with their checksums, the transfer procedure and approval records, configuration baselines, access and change records, backup and restore test results, and alerting and monitoring configurations. For utilities, NERC CIP-013-2 asks for supply chain processes that verify software integrity and authenticity, which maps directly to the package intake step. Confirm with your assessor which artifacts they expect before the build starts, because retrofitting evidence is slower and less reliable than collecting it during delivery.

Ask for the scope in writing, including platforms, source systems, network zones and transfer points, and ask how each offline update is tested before it reaches production. Ask who performs the upgrades after go-live, how the partner documents the environment, and what evidence will exist at handover. Ask whether security testing is included, which parts of the stack will be tested, and how retests after upgrades are scoped. Ask how the partner will handle a licensing question, such as whether a connector or module requires a paid edition for air-gapped use, since some vendors reserve offline deployment for paid tiers. Finally, ask for references in the form the partner is permitted to share, and for the named engineers who would do the work.