Installing ClickHouse in an air-gapped network

Stable releases ship roughly monthly, so an internal intake process needs a fixed monthly cycle rather than ad hoc downloads. LTS releases ship twice a year and are supported for a year after release. Each month, someone should review the GitHub releases page, decide which stable or LTS line you will run, download the tgz archives or packages on the connected side, and record checksums. The ClickHouse production FAQ also recommends reading the changelog before every upgrade, because minor backward-incompatible changes can occur. Ask your team to budget that review as a recurring task, not a one-time setup. Your internal repository should keep the previous release available so you can roll back.

Check the ClickHouse security policy first, because it lists the versions that currently receive security updates and states that all other versions are unsupported. Keep your running line on that list at all times. When a security release lands, use the same intake path as a monthly release, but run it outside the monthly cycle: verify the archive, test it in a pre-production environment with representative data and queries, then roll it through the replicas. Build a standing runbook for this path before you need it, including who approves the transfer across the boundary and who signs off the change.

Upgrade one replica at a time so that each shard keeps at least one healthy copy serving queries. Replicated tables are coordinated by ClickHouse Keeper, and a recovered replica automatically syncs missing data from a healthy replica, so a restarted node catches up without manual data copying. Before each restart, confirm that Keeper has a quorum, that replication queues on the other replicas are not backed up, and that the node you are about to restart is not the only copy of anything. Upgrade the Keeper nodes with the same discipline. Keep the pre-production cluster on the same version path so you can rehearse the sequence. Your engineers should write the exact order of hosts into the change record.

An LTS jump crosses more than one stable line, so it needs a longer test window than a routine monthly update. LTS releases ship twice a year and are supported for a year after release, which gives you a planned landing point, but it does not remove the need to read every changelog between your current version and the target. Plan for schema or setting changes, configuration files that must be reviewed against the new defaults, and any queries that depend on behavior that changed. Run the full sequence on a pre-production cluster with representative data, record the results, and only then schedule the production change. Ask whether your team or a partner will own the changelog review and the test evidence.

Rollback depends on having the previous archive or package still in the internal repository and on a tested path back to it. Keep the prior version available until the new one has run cleanly for an agreed period. If a replica fails after upgrade, take it out of the rotation, restore the previous binary or package, and let the replica resync from a healthy copy. If a problem affects data, rollback to the binary will not undo writes, so the recovery path is a backup restore. This is why the backup test matters as much as the binary. Write the rollback decision points into the change record before the window opens, so the team does not debate them during an incident.

Yes. The backup documentation notes that replication alone does not protect against human error or corruption, and it warns that a backup never test-restored will probably not restore properly when needed. Take a full or incremental backup with BACKUP to local disk, S3-compatible storage or Azure Blob Storage before each change, then restore it to a spare cluster and confirm the row counts and key queries match. Repeat the restore test on a regular schedule, not only before upgrades. The effort is recurring: storage for backups, spare capacity for restore tests, and engineer time to verify results. Ask any partner to show how restore tests are scheduled, recorded and approved.

Your team can handle patching if it has the capacity to run the monthly intake, the test cluster, the upgrade sequence and the restore tests every cycle. Most of the cost is recurring labor and spare infrastructure, not the install itself. A partner can plan the intake process, write the runbooks, set up the internal repository and run upgrades under a fixed-scope agreement with written deliverables. DeployOpen plans, implements, hardens, backs up, upgrades, documents and supports self-hosted platforms inside your own networks, and ongoing support is available as a retainer. Ask for a quote based on the number of clusters, the release cadence you choose and the restore testing you require.

Verify every archive or package on the connected side before transfer, and again after it lands inside the boundary. Record the checksum of each file, compare it with the value published by the source, and keep the record with the change ticket. Check the version string against the release you approved, and store the signed package metadata alongside the file. For regulated environments, this control maps to the supply chain requirements for verifying software integrity and authenticity of vendor software and patches, including NERC CIP-013-2 for bulk electric system operators. Confirm with your compliance lead which evidence your assessor expects before you design the transfer process.

The main cost drivers are the number of clusters, the approval process for moving software across the isolated boundary, the repository tooling you need to mirror packages, and how much restore testing you require. A single small cluster with a fast transfer path is far simpler than several production clusters, each with its own Keeper nodes, backups and approval cycle. Ask for the quote to list each cluster, each environment (test and production), and each transfer step as separate line items, so you can see which parts scale and which are fixed.

Duration depends mostly on your transfer and approval process rather than on ClickHouse itself. The longest steps are usually getting an approved path for moving archives or packages in, getting security sign-off on each intake, and building a test environment that matches production. Those timelines are set by your own change and security processes, so confirm them with your security and change teams before scheduling. A fixed-scope engagement agrees the timeline in writing before work starts.

One-time costs include designing the mirror and repository, building the first install, verifying downloads, writing runbooks and running the first restore test. Recurring costs come from the monthly intake cycle: ClickHouse stable releases ship roughly monthly, so each release has to be downloaded, checked, approved, carried across the boundary and published internally. Patching work also recurs, because security updates must reach supported versions. Ask whether the quote covers one intake cycle or ongoing support, and whether that support is a retainer.

Yes, many teams do this in-house if they already run ClickHouse and have engineers comfortable with package repositories and Keeper. The gaps usually appear in the repeatable intake process, in restore testing and in keeping the internal repository current. Before deciding, check whether you have one person who can own monthly intake, whether your change process can absorb a monthly release, and whether someone can run a restore test on a spare cluster. A partner can take on any of those pieces under a written scope.

You will need current network diagrams that show the isolated segment, the boundary and the approved transfer path. You also need named owners for the platform, security approval, change control and backups, plus access to a test environment that resembles production. Confirm early which transfer media and scanning steps your security policy allows. Missing owners or an undefined transfer path are the most common reasons an installation stalls, so settle them before the quote is finalized.

Some frameworks do. For example, NERC CIP-013-2 requires supply chain procurement processes to address verification of software integrity and authenticity of vendor software and patches, which is the reason teams check every package crossing an isolated boundary. Your auditor or regulator decides which requirements apply to you. DeployOpen prepares systems, controls and evidence for compliance assessments, but does not issue audit reports or certifications. Ask your assessor which evidence they expect before the intake process is designed.

Most DeployOpen engagements are fixed-scope, and the quote follows from a written scope that lists the clusters, environments, deliverables, the transfer and approval steps you own, and any retainer for ongoing intake and patching. DeployOpen does not publish a price list. To compare quotes, ask each provider to state the assumptions behind the scope, what is excluded, who approves each change, and how a monthly release is handled after go-live. Scope, deliverables and timelines are agreed in writing before work starts.