ClickHouse for energy and utilities

Why ClickHouse fits energy and utilities analytics

Utilities and energy operators collect large volumes of smart meter readings, grid measurements and SCADA-derived data, and they often need to keep that history for years. ClickHouse can run this analytics workload inside infrastructure the operator controls, provided the design starts from the compliance scope. Which systems are in scope for NERC CIP or NIS2 determines where the cluster can sit, who can reach it and how its data moves, so that scoping comes before any schema or hardware decision.

For the team, the change is mostly in design. Meter and telemetry tables are typically ordered by device and time and partitioned so that old data can be dropped or moved without rewriting recent data. Retention then becomes a set of TTL rules that move older parts to cheaper storage tiers instead of an archive process run by hand. Isolation is a deployment decision: a cluster for grid-adjacent data can be placed on a separate network segment with its own access controls, rather than shared with corporate analytics. The server is licensed under the Apache License 2.0 and is open source, while ClickHouse Inc. sells ClickHouse Cloud as a separate managed service. Operators who cannot place telemetry in a third-party cloud can run the open-source server on their own hardware.

Before committing to a design, confirm with your compliance team which BES Cyber System impact levels and data flows apply to each dataset, and confirm which ClickHouse release line your support model will cover. The ClickHouse production FAQ recommends testing with representative data and queries before production use, which is the practical place to check whether the planned schema and retention rules hold up under real meter volumes.

Industry requirements and what they mean for the build

This table maps the energy-sector requirements in scope to specific decisions in a self-managed ClickHouse build. Read each row as a question to answer in your design, not as a statement that the platform is compliant. Applicability depends on your registration, impact categories and national transposition, so confirm those with your compliance team and regulator before design freeze.

Requirement	What it asks for	What it means for the ClickHouse build
NERC CIP-002-5.1a impact categories	Sort BES Cyber Systems into high, medium or low impact using bright-line criteria in Attachment 1	Decide first whether telemetry stores hold or reference BES Cyber System information. Keep the historian and analytics cluster out of scope where possible, and document the reasoning for the impact determination.
NERC CIP-005-7 Electronic Security Perimeter	Place routable-connected high and medium impact assets inside an Electronic Security Perimeter	Put ClickHouse nodes, Keeper nodes and replication traffic inside a defined perimeter. Plan replication and Distributed table traffic as internal flows that the perimeter must describe.
NERC CIP-005-7 Interactive Remote Access	Route all Interactive Remote Access through an Intermediate System with encryption and multi-factor authentication	Do not expose ClickHouse HTTP or native ports to remote analysts directly. Plan a brokered access path, and confirm the MFA method and session logging with your compliance team.
NERC CIP-013-2 supply chain risk management	Maintain plans covering vendor incident notification, vulnerability disclosure, software integrity and vendor remote access, reviewed at least every 15 calendar months	Record the ClickHouse release channel, verification of release artifacts, and vendor and partner notification paths in the plan. Schedule review cycles to fit the 15-month interval and track upgrade decisions against it.
NERC CIP-012-2 real-time data availability	Protect availability of Real-time data between Control Centers, effective 1 July 2026	If the cluster receives Real-time data between Control Centers, design replica placement and failover so a single site loss does not stop the data feed. Test replica recovery and document it.
NERC CIP-015-1 internal network security monitoring	Monitor internal network activity for the in-scope systems, enforceable from 1 October 2028 (FERC approved 26 June 2025)	Plan to send ClickHouse connection, authentication and query logs to a monitoring pipeline now, so the data exists when enforcement applies. Confirm the scope of CIP-015-2 with your auditor, since it extends monitoring to associated EACMS and PACS.
NIS2 Annex I energy coverage	List electricity, district heating and cooling, oil, gas and hydrogen as high-criticality subsectors	For EU operators, confirm whether your entity is in scope and whether it is classified as essential or important under national law. The classification affects supervision intensity, not the build itself.
NIS2 Article 21 risk-management measures	Apply risk-management measures to the entity's network and information systems, with the technical detail set by Implementing Regulation (EU) 2024/2690 for listed digital providers	Build the analytics platform to your own documented risk assessment. Cover access control, backup, encryption, logging, vulnerability handling and supplier security, and keep evidence that each measure operates.

Which environments fall under CIP and which fall under NIS2

CIP standards become mandatory in the United States once FERC approves them, and NERC and the Regional Entities monitor and enforce them. Whether a ClickHouse environment falls under CIP depends on how the operator's BES Cyber Systems are classified under CIP-002-5.1a. Classification uses bright-line criteria that sort assets into high, medium and low impact. High impact mainly covers qualifying control centers, medium impact includes, for example, generation of 1500 MW or more at one plant or transmission facilities at 500 kV or higher, and low impact covers other BES Cyber Systems at listed assets. Requirements then apply by Applicable Systems, including associated EACMS, PACS and PCA.

High impact cases: Historian or analytics servers connected to high impact control centers typically face the widest set of controls.
Medium impact cases: Systems with external routable connectivity to medium impact assets are scoped by CIP-005-7 and related standards.
Low impact cases: CIP-003-9 and the newer CIP-003 versions add controls for vendor remote access and remote user authentication.
Off-system cases: A ClickHouse cluster fed by exported, non-operational data may sit outside BES Cyber System scope if it has no operational path back.
Enforcement: There is no CIP certificate. The Compliance Enforcement Authority decides scope through audits and reviews.

Confirm the classification and the data flow diagram with your Compliance Enforcement Authority before you design the platform.

Designing ClickHouse tables for meter and grid time series

The first design decision is the ordering key of each MergeTree table, because it determines how much data ClickHouse reads for an interval query. For meter reads, a common starting point is an ordering key that leads with the meter or feeder identifier and then the timestamp, so that a query for one meter over one day scans a narrow, contiguous range of data. Put the high-cardinality identifier first only after testing, since a poor key forces scans across unrelated meters. Grid telemetry tied to a feeder or substation hierarchy may need a different leading column. Confirm the choice against the actual interval and reporting queries your billing, outage and load-forecasting teams will run, not against a generic time-series pattern.

The second decision is partitioning. Partitioning by month or day is typical for time series because it makes retention and drops cheap, but partitions that are too fine create many small parts and slow merges, while partitions that are too coarse keep expired data around longer than intended. Do not partition by meter identifier, since that multiplies partitions with the size of the fleet. Per-meter or per-feeder access is handled by the ordering key and, where needed, skip indexes or projections, which are covered in the performance and schema topic. Test partition granularity against a representative data volume in a pre-production cluster before fixing it.

For ingestion, the table engine should be ReplicatedMergeTree on every node that holds a shard, coordinated by ClickHouse Keeper, so that a meter data ingestion path can continue when one replica is unavailable. The replication guide recommends at least three Keeper nodes on dedicated hosts in production, and a recovered replica syncs missing data automatically. Replication does not protect against bad writes or deletions, so the backup plan must be designed alongside the table layout. Ask the platform team to confirm Keeper placement and recovery steps inside the isolated network before any meter data arrives.

Configuration that matters for utility workloads

Utility telemetry sets high expectations for availability, access separation and long retention, and these settings decide whether a ClickHouse cluster meets them. Each card names a decision your team has to make before data arrives, with the check to run before it goes live.

Replicated tables with Keeper

Meter and grid history should survive a node loss without a manual rebuild. Use ReplicatedMergeTree tables coordinated by ClickHouse Keeper. The replication guide recommends at least three Keeper nodes on dedicated hosts in production, with about 4 GB of RAM each. Confirm that Keeper does not share hosts with busy query nodes, and test a replica failure in your pre-production environment before loading live feeds.

External authentication for operators

Operations staff, engineers and vendors should sign in with identities your directory already governs, not with local ClickHouse passwords. ClickHouse supports LDAP directories, Kerberos, SSL X.509 certificates, an HTTP authenticator and JWT as external authenticators. Choose the method that matches your identity stack, and confirm how each one behaves when the directory is unreachable during an incident.

Roles that split operations, engineering and vendors

Operations, engineering and vendor accounts need different reach into the same tables. Use roles, row policies and settings profiles so that vendor users see only the datasets their contract covers, and engineers cannot alter retention or access settings. Write the matrix before creating users, then review it at each vendor access renewal. The access control documentation recommends the SQL-driven workflow for this.

Restricting the default user

A fresh ClickHouse installation starts with a default user that holds all permissions, which is a common path to an unintended open account in a utility network. Disable or restrict the default user before the cluster joins any network segment that carries operational data. Then verify with a test login that the restriction holds on every node, including nodes added later.

Backup and restore to separate storage

Replication copies a mistake as faithfully as it copies data, so it does not protect against deletions or corruption. BACKUP and RESTORE can write full and incremental backups to local disk, S3-compatible storage or Azure Blob Storage. Place backups in a location with separate credentials, and schedule regular restore tests on a spare cluster, since an untested backup often fails when it is needed.

Retention and cold tiers for history

Regulators and asset engineers often ask for years of telemetry, while recent data is queried most. Define retention per dataset and move older parts to cheaper storage through TTL rules and storage policies. Ask the team which datasets must stay queryable and for how long, since that answer sets both storage cost and the scope of any future restore.

Rollout phases for a utility deployment

The sequence moves a ClickHouse deployment from impact classification to a handed-over production cluster, with each phase closing on a written check. Utilities that follow the order can show their compliance owners what was decided, tested and restored before any production data arrives.

Classify systems and define scope
Start by deciding which data sources feed ClickHouse and how each one is classified under the CIP-002 bright-line criteria or the entity's NIS2 role. Compliance, OT security and the data owners agree on what the cluster will hold, such as smart meter reads, grid telemetry or SCADA-derived historian exports, and which of those are BES Cyber System Information or Real-time data. The output is a signed scope document that states the impact categories, the data classes and the systems that must stay out of the cluster. Confirm with your Compliance Enforcement Authority or internal compliance team how your registration applies before you treat any system as out of scope.
Design network and boundary placement
Decide where the ClickHouse servers and Keeper nodes sit relative to any Electronic Security Perimeter, and how data crosses into them. Map every ingestion path, such as a historian export, a message broker or a file drop, and every remote access route, then mark which ones would be Interactive Remote Access under CIP-005-7. The output is a network diagram with named flows, firewall rules and a list of the access paths that pass through an Intermediate System. Have the OT network team review the diagram before any hosts are built.
Build and test in pre-production
Install a pre-production cluster that matches the planned topology, then load representative data and run the queries that operators, planners and analysts actually use. Read the changelog for the chosen release before installing, because the production FAQ notes that minor backward-incompatible changes can occur, and record the version in your change records. The output is a test report covering ingestion rates, query results against known values, and any query that needs a schema change. Choose stable or LTS releases from the supported list in the security policy, and confirm the support window with your own vendor or support plan.
Configure authentication and access model
Restrict the default user, then build the access model with users, roles, row policies and settings profiles through SQL. Connect the identity source your utility already uses, such as LDAP, Kerberos or X.509 certificates, and document which accounts are service accounts and which are named people. Include a review of how privileged operators are granted and removed. The output is a tested access matrix where each role has a documented purpose, and a test showing that the default user cannot log in remotely. Map the controls to the relevant CIP requirements with compliance staff rather than assuming a mapping.
Set replication and prove backup restores
Deploy replicated tables coordinated by ClickHouse Keeper on dedicated hosts, then configure scheduled BACKUP jobs to the storage location your design approved. Run a restore onto a spare cluster and compare row counts and sample query results with production. The ClickHouse backup documentation warns that a backup never test-restored will probably not restore properly, so this check is the gate for production. The output is a restore test record with timestamps, the backup identifier and the comparison results. Note that replication alone does not protect against human error or corruption, so the backup evidence is a separate artifact.
Cut over and hand over operations
Move ingestion to production in a planned window, confirm that data arrives as expected, and keep the pre-production cluster available for rollback until the operations team signs off. Hand over runbooks for replica recovery, backup schedules, access reviews and upgrades, including the rule to read changelogs and test in pre-production before each upgrade. The output is a signed handover pack with monitoring in place, named owners for each runbook and a dated record of the first restore test completed on production-derived data. Set a recurring review of the scope document, since CIP and NIS2 requirements change over time.

Compliance considerations for CIP and NIS2 scope

Whether CIP or NIS2 applies depends on who the organization is and what its systems do, and that determination comes before any ClickHouse design work. Once scope is clear, the cards below show which controls a self-managed ClickHouse analytics platform touches and what evidence the team should be ready to produce. DeployOpen prepares controls and evidence, but auditors, regulators and the Compliance Enforcement Authority decide compliance.

Change and configuration control

CIP-007-6 covers system security management, and CIP-010 covers configuration change management and baselines. For ClickHouse, this means a documented baseline for server versions, configuration files, users and roles, and Keeper settings, with changes approved and recorded before they reach production. Check that upgrades, including the stable and LTS release lines, go through a pre-production environment with representative data and that the changelog is reviewed first. Confirm with your auditor which BES Cyber Systems your platform falls under, since an analytics cluster may sit outside those systems entirely.

Information protection for stored data

CIP-011 concerns protecting BES Cyber System Information, including its storage, transit and disposal. Smart meter, grid and SCADA-derived datasets can contain information that falls under this scope, so the team should identify which tables hold it before data lands in ClickHouse. Plan access through ClickHouse users, roles and row policies, restrict the default user, and decide how retention and disposal of older partitions are documented. Ask whether backups copied to S3-compatible or Azure Blob Storage destinations inherit the same protection requirements.

Vendor software integrity and remote access

CIP-013-2 requires supply chain risk management plans that cover vendor incident notification, vulnerability disclosure, software integrity verification and coordination of vendor-initiated remote access. For a self-hosted ClickHouse deployment, the open-source server is one vendor input, and any support provider is another. Record which ClickHouse release lines you run, check the security policy for supported versions, and verify package integrity at install. Document how a vendor or DeployOpen engineer gets remote access, through which intermediate path, and under which approval and logging, and confirm these terms against your own plan.

NIS2 risk management and incident reporting

Article 21 requires in-scope entities to apply risk-management measures, and Article 23 sets reporting obligations for significant incidents. For a ClickHouse analytics platform, the evidence set usually includes asset inventory, backup and restore test records, replica recovery procedures, access reviews and monitoring of Keeper health. Define how a ClickHouse outage or data integrity event is classified and who must be told, and check the deadlines in the transposition law of each Member State where you operate. Confirm which obligations apply to your entity type with counsel, because the directive is implemented nationally.

Implementing Regulation 2024/2690 applicability

Implementing Regulation 2024/2690 sets technical requirements for specific digital service categories, including cloud, data center and managed service providers, rather than for energy operators as such. An energy utility running ClickHouse on its own infrastructure is generally not the regulated provider the regulation describes, but a managed or cloud service supplier may be. Confirm with your national competent authority whether the regulation's requirements reach any service you buy or provide. Treat this as a scoping question to settle in writing before design begins.

ENISA guidance as non-binding evidence

ENISA published non-binding technical implementation guidance for Implementing Regulation 2024/2690 on 26 June 2025, including examples of evidence and mappings to ISO/IEC 27001:2022 and NIST CSF 2.0. Because it is non-binding, use it to shape evidence formats, such as configuration exports, backup restore test reports and access review logs, rather than as a checklist that determines compliance. Map each control to the artifact that proves it, and keep those artifacts current as the ClickHouse cluster changes. Ask your assessor which evidence formats they accept before collection begins.

What utilities ask before hiring a partner for ClickHouse

What drives the cost and effort of a self-managed ClickHouse deployment for utility data?

The main drivers are scope, the number of environments, the CIP impact classification of the systems that feed the platform, how isolated the network must be, and how much data you keep and for how long. A single production cluster ingesting smart meter data with a short retention window is a very different job from a multi-site deployment with a pre-production environment, replicated Keeper nodes and years of retained grid data on cheaper storage. Ask any partner for a written scope that lists each environment, each data source, the retention period per table and the systems in scope for NERC CIP review, because those items decide most of the quote. Expect a one-time build cost plus recurring costs for support, upgrades and testing if you choose a retainer.

Can our team run ClickHouse ourselves using the official documentation?

Yes, many teams do, and the official documentation covers replication, backup, access control and external authentication in enough depth to start. The harder part for utilities is the surrounding work: classifying the systems, documenting the boundary, proving that backups restore and keeping evidence for an auditor. The ClickHouse backup documentation itself warns that a backup never test-restored will probably not restore properly when needed, so a regular restore test on a spare cluster belongs in your plan from the start. Be honest about your team's capacity for on-call, upgrades and compliance evidence before you decide. A partner can take on part of that work under a written scope while your staff keep ownership of the grid data.

How long does a fixed-scope ClickHouse engagement take to agree?

Most DeployOpen engagements are fixed-scope, so the scope, deliverables and timeline are agreed in writing before any work starts. The time to agree that document depends on how quickly your OT, security and compliance owners can answer questions about data sources, impact classification, network zones and retention rules. Ask the partner for a written scope and timeline tied to named deliverables, and ask what you must provide and by when. Delays usually come from unresolved boundary questions rather than from the technical build, so settle the classification and network design early. DeployOpen does not publish a price list, so the timeline and cost follow from that written scope.

What do OT and compliance staff need to provide for a ClickHouse project?

OT and compliance staff provide the facts that the design depends on, and they own the decisions that carry regulatory weight. Typical inputs include an inventory of the data sources such as historian exports, SCADA-derived feeds and smart meter head-end data, a list of the systems that connect to them, the impact classification your team has assigned under CIP-002, and the network zones and Electronic Security Perimeter boundaries that apply. Compliance staff should confirm which evidence they need to keep and how long, and who signs off on access and change control. Plan for regular review time from these teams during the build, not only at the start.

Can ClickHouse be installed on an isolated network for a utility?

An isolated install is feasible in principle, but it depends on how the packages, container images and updates reach the environment, and on what your regulator expects for that boundary. ClickHouse server is open source under the Apache License 2.0, so the software itself can be installed inside your own network. Plan how security patches and the release channel will be moved across any air gap, and confirm the supported versions in the ClickHouse security policy before you choose a release line. Under CIP-005-7, routable-connected high and medium impact systems must sit inside an Electronic Security Perimeter, and any interactive remote access must pass through an Intermediate System with encryption and multi-factor authentication. Confirm with your NERC Regional Entity, your compliance team and ClickHouse how the isolated design will be reviewed, and whether NIS2 obligations apply through your national transposition law.

How are upgrades and support handled after go-live?

Upgrades and support after go-live are usually handled through a retainer, which covers recurring work such as patching, release testing, backup restore tests and monitoring reviews. ClickHouse publishes stable releases roughly monthly and LTS releases twice a year, and its security policy lists which versions receive security updates, so your retainer scope should name the release line you run and how quickly you expect a patch to reach a pre-production environment. Ask what changes are covered, what is excluded, how incidents are escalated and who is on call. The ClickHouse production guidance recommends reading the changelog before upgrading, because minor backward-incompatible changes can occur, so ask how that review is done and recorded for your compliance file.

What should we confirm with the vendor and the regulator before we start?

Confirm with the ClickHouse vendor which release line you will use, which versions are currently supported with security updates, and how replication and backup behave in your target topology. Confirm with your Regional Entity or compliance advisor whether the ClickHouse platform, its hosts and its management tools fall inside the CIP scope for your impact classification, and which standards apply, including the 2026 revisions such as CIP-003-9 and CIP-012-2 and the later virtualization revisions. For NIS2, confirm with your national competent authority how the directive has been transposed and whether your entity is classed as essential or important. Ask in writing, and keep the answers with the project record.

Running ClickHouse on isolated networks

An isolated ClickHouse deployment changes almost every routine task, so the decisions have to be made before the first install. Without direct internet access, the cluster cannot pull packages, container images or backup targets on demand. Decide how ClickHouse server packages, Keeper packages, operating system updates and any container images enter the enclave, who approves each transfer, and where the signed artifacts are stored for offline reinstalls. Confirm the release line you will pin against the ClickHouse production FAQ and the ClickHouse releases page, and plan to mirror the exact build you validated, not whatever is newest when a rebuild is needed.

Patch intake is where utilities most often lose time. Treat ClickHouse as software that supports a BES Cyber System or its associated monitoring, and document how vendor security notices are received, assessed and staged. CIP-013-2 expects supply chain risk management plans that cover vendor vulnerability disclosure and software integrity verification, reviewed at least every 15 calendar months, so record the hash or signature check performed on each transferred package. CIP-007-6 style controls for patch management and malicious code prevention apply to the hosts as well. Confirm with your compliance team which systems are in scope under the impact rating in CIP-002-5.1a before you decide how much of this chain is mandatory.

Backups also need explicit targets. ClickHouse BACKUP and RESTORE can write to local disk, S3-compatible storage or Azure Blob Storage, but an isolated site may only have one of these, so choose the target and its retention window in advance. A backup on the same storage as the cluster protects against little, and the documentation is clear that replication does not guard against human error or corruption. Schedule restore tests on a spare cluster built from the same offline packages, because a restore that has never been exercised in the enclave is the most likely failure. Finally, agree on how vendor support access works: any remote session needs to pass through the approved access path, and you should confirm that path with your Electronic Security Perimeter design under CIP-005-7.

How DeployOpen works on this

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, including air-gapped or restricted networks. For ClickHouse in energy and utilities, we start from the data sources, retention needs and regulatory scope, then build and test the cluster in your environment.

How we scope

Inventory the meter, grid and SCADA-derived data sources, volumes and query patterns that the ClickHouse cluster must serve.
Classify which datasets and network zones fall within NERC CIP or NIS2 scope, with your compliance and OT security leads.
Agree deliverables, environments and exclusions in writing before work starts, since most engagements are fixed-scope.

How we build

Design time-series tables, ordering keys, TTL rules and tiered storage for your retention periods and query workloads.
Deploy ReplicatedMergeTree replicas with ClickHouse Keeper nodes on dedicated hosts, following the vendor's replication guidance.
Configure external authentication, roles, row policies and restricted network paths sized to your electronic security perimeter design.

How we hand over

Test a BACKUP and RESTORE cycle on a spare cluster and record the measured restore results for your review.
Document the architecture, access model, evidence sources and runbooks that your auditors and operators will need.
Walk your operations team through upgrade planning, the supported release lines and the changelog review step.

How we support

Offer ongoing support, upgrades and continuous testing as a retainer after handover, with scope agreed in writing.
Plan version upgrades in a pre-production environment with representative utility data before touching live clusters.
Prepare systems and control evidence for your compliance assessment, while independent auditors issue any audit outcome.