Event streaming for energy and utilities starts with CIP scope, not throughput
The first decision for a utility or grid operator planning self-hosted event streaming for meter, sensor and SCADA-derived data is whether the streaming platform falls inside or outside NERC CIP scope. That answer, not throughput, determines which platform, hosting model and access controls are viable, so it should be settled before any broker sizing begins. In the United States, the CIP Reliability Standards become mandatory once FERC approves them, and NERC and the Regional Entities monitor and enforce compliance. Thirteen CIP standards are currently listed as mandatory and enforceable, so the scope question needs a written answer tied to specific standards, not a general assumption. This differs from most other industries because scope is set by bright-line impact ratings rather than by data sensitivity alone. Under CIP-002-5.1a, BES Cyber Systems are sorted into high, medium and low impact, and the criteria are explicit: high impact mainly covers qualifying control centers, while medium impact includes, for example, generation of 1500 MW or more at one plant. Under CIP-005-7, routable-connected assets of high and medium impact systems must sit inside an Electronic Security Perimeter, and all Interactive Remote Access must pass through an Intermediate System with encryption and multi-factor authentication. A streaming cluster that receives SCADA-derived data from such systems can therefore become part of that boundary's question, even when it is not a control system itself. Replication across control centers adds a second decision. A broker cluster that mirrors data between sites changes where in-scope information travels and who can reach it, so the team should document which topics cross each boundary, which replication links exist, and how vendor-initiated remote access is coordinated under CIP-013-2. Ask the partner to show the scope analysis for each topic and cluster before the sizing exercise, and confirm the registration tier and impact rating with your compliance lead and the applicable Regional Entity.
Which requirements shape the streaming build
This table maps each NERC CIP and NIS2 requirement that touches a streaming platform to the evidence an assessor expects and the build decision it forces. Read it from the top: the impact classification of the systems a topic carries determines where brokers, replicas and monitoring can sit.
| Requirement | What it asks for | Build implication |
|---|---|---|
| CIP-002-5.1a impact classification | Sorts BES Cyber Systems into high, medium or low impact using bright-line criteria, which sets the controls that apply to each system | Classify every source system before design. The impact level of the highest-rated system a topic carries drives where brokers may sit, not the broker's own role |
| CIP-005-7 Electronic Security Perimeter | Routable-connected assets of high and medium impact BES Cyber Systems sit inside an ESP, and Interactive Remote Access passes through an Intermediate System with encryption and MFA | Brokers that receive control-system data either sit inside the ESP or sit outside behind a defined boundary. Confirm how cross-perimeter topics are secured and who can reach brokers remotely |
| CIP-011-3 information protection | Requires documented protection of BES Cyber System Information, including storage, transit and disposal | Decide which payload fields count as protected information. Topic retention, replica locations and backups all hold that data and must be covered by the same handling rules |
| CIP-013-2 supply chain risk management | Plans covering vendor incident notification, vulnerability disclosure, software integrity and authenticity verification, and vendor remote access, reviewed at least every 15 calendar months | Verify release signatures and checksums for Kafka, Pulsar or NATS and connector plugins before deployment. Record the upgrade path and vendor contacts in the plan, and confirm any support provider's remote access is covered |
| CIP-015-1 internal network security monitoring | Internal network security monitoring, approved by FERC on 26 June 2025 and enforceable from 1 October 2028 | Design broker and replica network flows so monitoring sensors can see them. Confirm which monitoring tools can run inside the ESP, since the standard is not yet enforceable but the architecture is hard to change later |
| NIS2 Annex I energy coverage | Lists energy as a high-criticality sector with electricity, district heating and cooling, oil, gas and hydrogen subsectors | Confirm with counsel and the national competent authority whether the organization is in scope and which incident reporting duties apply to the streaming platform and its suppliers |
| Commission Delegated Regulation (EU) 2024/1366 | Sector-specific cybersecurity rules for cross-border electricity flows, supplementing Regulation (EU) 2019/943 | For cross-border operators, check whether replicated topics cross borders and which cybersecurity rules apply to the data path between control areas |
| Replication across control centers | Multi-site resilience expectations set by the entity's own recovery objectives and the CIP standards that apply to backup control centers | Replication links are part of the protected boundary. Each replica site needs the same impact-based placement and access controls as the primary, and failover tests should be recorded as evidence |
Which open-source streaming platforms fit utility data
Smart meter, grid sensor and SCADA-derived streams differ in volume, retention and how far they must be replicated, and those differences decide which broker fits. The cards below set out what each option offers for utility data and what to confirm in your own environment before you choose.
Apache Kafka for utility event backbones
Kafka is licensed under Apache-2.0 and has no paid edition from the project itself, so license cost is not the deciding factor. The supported releases are 4.3.1, 4.2.2 and 4.1.2, and Kafka 4.0 and later run in KRaft mode only, so any ZooKeeper-based cluster must migrate first. Brokers require Java 17. For utility data, check that your team can operate KRaft, patch Java and manage SASL or TLS on every listener that carries meter or grid data.
Apache Pulsar for multi-region retention
Pulsar is licensed under Apache-2.0 and separates stateless brokers from BookKeeper bookies that store data, so storage grows by adding bookies without adding brokers. That split suits long meter-data retention and large fleets of topics. Pulsar 5.0 LTS has active support until October 5, 2028 and security support until October 5, 2029. Confirm the metadata store you will run, since Oxia is recommended for new clusters, and confirm that your staff can operate three tiers: brokers, bookies and metadata.
Apache Kafka security controls
Kafka supports SSL/TLS between clients, brokers and tools, and SASL mechanisms including GSSAPI, SCRAM-SHA-512 and OAUTHBEARER. Security is optional in the product, so an unsecured or mixed listener is possible and must be closed by configuration and policy. In KRaft clusters, ACLs live in the metadata log, and resources without an ACL are limited to super users unless allow.everyone.if.no.acl.found is set to true. Verify that this setting stays false in any segment that touches control-center or substation-adjacent data.
Pulsar multi-tenancy and geo-replication
Pulsar is built for multi-tenancy with authentication, authorization and quotas, and it includes geo-replication across clusters. Utilities often separate operating companies, generation fleets and control center domains, so tenant boundaries can map to those divisions. Test how quotas and namespace policies behave when a single region's meter feed spikes, and confirm whether your replication design will carry CIP-scoped data across boundaries you have not approved.
NATS for lightweight telemetry
NATS is a candidate for small, high-fanout messaging such as edge telemetry or request and reply between field services. This source set does not confirm its licensing, release status, persistence model or clustering behavior, so none of those are stated here. Before you shortlist NATS, confirm its current license, its JetStream persistence and replication options, and whether it can be placed inside an ESP without crossing a boundary you need to defend.
RabbitMQ for command and task messaging
RabbitMQ is a candidate for work queues and command messages, such as dispatching a job to a field crew system, where per-message routing matters more than replay of long history. The supplied sources do not confirm its license, release cadence or retention limits, so verify them directly. Confirm whether your use is queuing or event retention, because that choice determines whether RabbitMQ or a log-based platform such as Kafka or Pulsar fits.
Should the streaming platform sit inside or outside the ESP?
Placing the streaming platform outside the Electronic Security Perimeter keeps the broker cluster, its administrators and its storage away from the high and medium impact BES Cyber Systems. Data crosses the boundary through defined, one-way or brokered flows, and the control environment only receives what it needs. The compliance owner should confirm with the impact analysis whether any data leaving the control environment is itself in scope under the CIP standards.
- Data flows: Document each flow from the BES Cyber System to the broker, its direction, protocol, port and the data elements it carries. Inbound commands to the control environment should be excluded or tightly limited.
- Intermediate System: Interactive Remote Access from engineers to the streaming platform must pass through an Intermediate System with encryption and multi-factor authentication, as CIP-005-7 requires for interactive access.
- Multi-factor authentication: Confirm which MFA method covers administrators of the broker, the metadata store and the monitoring console, and whether service accounts are excluded.
- Boundary evidence: Auditors typically sample firewall rules, the flow inventory and change records showing each rule was reviewed.
- Vendor access: Remote support into the cluster falls under CIP-013 supply chain controls, so document how vendor sessions are initiated and recorded.
Check that the flow inventory matches the live firewall configuration before the first assessment sample is requested.
Placing the streaming platform inside the perimeter means brokers, storage and administrative consoles sit within the same protections as the BES Cyber Systems they serve. This reduces the number of crossings but raises the number of assets that must meet the full set of applicable CIP requirements. Routable connectivity from high and medium impact assets must be inside a perimeter, so any broker that talks to those assets needs to be inside or behind a defined boundary.
- Data flows: Traffic between producers, brokers and consumers stays within the perimeter, which simplifies flow documentation but expands the number of internal rules to track.
- Intermediate System: All Interactive Remote Access to brokers, including from engineers working remotely, still passes through an Intermediate System with encryption and multi-factor authentication.
- Multi-factor authentication: Every administrator path into the cluster, including the Kafka or Pulsar CLI hosts and the monitoring stack, needs MFA where interactive remote access applies.
- Asset scope: Brokers and their hosts may themselves become in-scope assets, so the compliance owner should confirm the impact rating and the protections required for each.
- Virtualization: If the platform runs on virtualized hosts or containers, confirm how the revised CIP standards accommodated by FERC Order No. 919 apply and their effective date of 1 July 2028.
Confirm the asset inventory includes every broker, controller and management host before the placement is approved.
A hybrid design places a historian or aggregation tier inside the perimeter and a separate streaming cluster outside it, with the bridge carrying only aggregated or filtered values across the boundary. This keeps the bulk of the event volume and consumer fan-out away from the protected environment. The bridge becomes the most closely examined component because it is the path that every outside consumer depends on.
- Data flows: Map two sets of flows, one from BES Cyber Systems to the historian and one from the historian to the outside streaming cluster, and record the transformation applied at the bridge.
- Intermediate System: Interactive access to the bridge and historian must pass through an Intermediate System with encryption and multi-factor authentication, and sessions to the outside cluster should be separately recorded.
- Multi-factor authentication: Apply MFA to administrators of both tiers and to any operator who can change bridge configuration or filtering logic.
- Change control: Changes to what the bridge forwards are configuration changes to a boundary control, so they need the same review as firewall rule changes.
- Auditor samples: Expect requests for the bridge configuration history, the aggregation logic and evidence that unfiltered values are not forwarded.
Confirm with the compliance owner whether the aggregated values still carry enough information to make the bridge's output itself in scope.
How meter and sensor volume drives retention and storage
Storage needs in a streaming platform follow from four inputs: the number of meters and sensors, how often each one reports, the size of each message, and how long data must stay available for replay, analytics and compliance. Multiply those together and you have a daily and total byte count, but the retention period usually dominates the result. A reporting interval that is shortened from minutes to seconds, or a retention window that is extended to cover an audit requirement, changes the storage footprint far more than a modest change in device count. Before choosing between Kafka and Pulsar, measure your own meter and sensor rates, message sizes with the payload you actually plan to send, and the retention each consumer and each compliance owner needs. This document does not assume any volume figures, so treat every number you see in vendor material as something to replace with your own measurements.
In Kafka, retention is set per topic and storage lives on the brokers' disks, so growth means adding brokers or disk capacity alongside compute. Replication multiplies the stored bytes by the replication factor, so a high-volume telemetry topic with long retention can strain disks across the whole cluster. Pulsar separates these concerns: brokers route messages while Apache BookKeeper bookies hold persistent storage, and capacity grows by adding bookies independently of brokers. That split can suit utilities whose storage needs grow steadily while processing demand stays flat, but it also adds a component to operate. Whichever platform you choose, confirm with the vendor how retention is enforced, how tiered storage is handled in your version, and what happens to consumers when a backlog exceeds retention.
Retention for logs that support NERC CIP evidence is a decision for the compliance owner, not the streaming team, and it should be written down before topics are created. Ask that owner which records must be kept, for how long, and in which location, then translate that answer into topic-level retention, archive pipelines and access rules. Operating-system and platform changes also affect cluster operations. Kafka 4.0 supports only KRaft mode, so clusters still on ZooKeeper must migrate before upgrading, and brokers, Kafka Connect and tools require Java 17 from Kafka 4.0 onward. Plan these upgrades into your storage model, because a cluster that cannot be patched on schedule carries its own compliance risk regardless of how much data it holds.
How a utility streaming rollout runs in phases
The sequence moves from classifying which data sources and systems are in scope to a handed-over platform with tested failover and evidence the compliance team can review. Each phase closes with a written output that the next phase depends on.
Classify data sources and impact levels
Inventory every smart meter feed, grid sensor stream and SCADA-derived dataset, then record which systems produce and consume each one. Compliance and operations staff, with the entity's CIP lead, assign each BES Cyber System to a impact level under CIP-002-5.1a using its bright-line criteria, and confirm which assets are routable-connected. The step closes with a classified data inventory signed off by the CIP owner, so no stream is placed on a network before its impact rating is known. Ask the partner to show how they map feeds to this inventory before any broker is sized.
Design placement and network boundaries with ESP review
Decide whether the streaming cluster sits inside an Electronic Security Perimeter, outside it with a controlled bridge, or in a corporate network fed by one-way or brokered flows. Under CIP-005-7, routable-connected assets of high and medium impact BES Cyber Systems must sit inside an ESP, and all Interactive Remote Access must pass through an Intermediate System with encryption and multi-factor authentication. Draw the data flow diagrams, then review them with the entity's compliance and security teams. The output is an approved architecture diagram and a list of every crossing point, with the controls that protect each one.
Select the platform and supported version line
Choose between Kafka, Pulsar, NATS or RabbitMQ using the requirements already agreed, then pin a version from a supported release line. For Kafka, the downloads page lists 4.3.1, 4.2.2 and 4.1.2 as supported, and clusters still on ZooKeeper must migrate to KRaft before moving to 4.x. For Pulsar, confirm the support windows for the LTS line you choose, since Pulsar 4.0 LTS active support ends October 21, 2026. The output is a written version decision with a planned upgrade path, plus confirmation of the Java runtime required by that release.
Apply the security baseline before any data flows
Enable TLS on every client, broker and tool listener, and configure SASL with a mechanism your identity provider supports, such as SCRAM or OAUTHBEARER, or Kerberos where the directory requires it. Define ACLs for each producer and consumer group, and keep allow.everyone.if.no.acl.found set to false so resources without ACLs stay restricted to super users. Review CIP-013-2 vendor access and software integrity requirements for any remote support path. The step is complete when a configuration review confirms no unsecured listeners exist and each ACL traces to a named system owner.
Design replication across control centers
Define which topics replicate between control centers, which stay local, and how consumer offsets are translated after a switch. For Kafka, the geo-replication documentation describes cross-cluster mirroring options, and Pulsar offers its own geo-replication model. Set the replication direction for each stream and document which control center is authoritative for commands versus telemetry. The output is a replication map that the CIP owner and operations staff have reviewed, with each cross-site path also present in the ESP diagram from the earlier phase.
Test failover and restore procedures
Run planned switchover tests between control centers and a restore test from backup into an isolated environment. Record the observed loss, duplication and recovery time for each tested stream, and compare them with the targets set by operations. Include a test of revoked credentials and of a cluster rebuilt from configuration, since a restore that depends on undocumented settings will fail under pressure. The step closes with a test report listing each scenario, its result and the remediation owner for every gap found.
Hand over runbooks and collect compliance evidence
Deliver runbooks for routine operations, upgrades, certificate rotation, ACL changes, failover and restore, along with the on-call escalation path. Build an evidence folder mapped to the CIP standards in scope, covering configuration baselines, change records, access reviews, log samples and test reports, and agree its retention with the entity's records policy. Operations staff should perform at least one failover themselves under observation before handover closes. Independent auditors will assess this evidence, since DeployOpen prepares it but does not issue audit or compliance conclusions.
Integration points and replication patterns across control centers
Smart meter, grid sensor and SCADA-derived feeds reach a streaming platform through several integration points, and each one decides what crosses an electronic security perimeter. The cards below cover the adapters, the boundary, the schemas, the consumers, the replication path between control centers and the monitoring that supports CIP-015-1 expectations.
Source adapters for meters and SCADA-derived feeds
Meter head-ends, historians and SCADA-derived exports rarely speak Kafka or Pulsar natively, so an adapter layer translates them into topics. Each adapter should run in the zone where its source lives, publish only the fields the consumers need, and tag every record with the source system and collection time. Confirm which protocols each source exposes, who owns the adapter after handover, and whether adapter credentials are held in a vault rather than in configuration files. Adapters that sit inside a high or medium impact boundary need the same change control as the systems they read from.
Boundary brokers or one-way transfer
Where a broker must receive data from a less trusted zone, a boundary broker or a one-way transfer path separates the two sides. Decide which direction data may flow, whether consumers inside the perimeter can publish back, and how interactive remote access is kept outside the broker path. CIP-005-7 requires routable-connected assets of high and medium impact BES Cyber Systems to sit inside an Electronic Security Perimeter, so the design must be checked against the impact classification of each source before a topic is opened. Ask the partner to show the firewall rules and ACLs that enforce the direction.
Schema management across vendor feeds
Meter vendors, historians and grid applications change field names, units and encodings on their own schedules. A schema registry or an agreed schema contract keeps producers and consumers compatible, and it gives operations a record of what changed and when. Confirm how the chosen platform handles schema evolution, who approves changes to a shared topic, and how a breaking change is rolled out to consumers in both control centers. Version changes to the streaming platform itself should follow a tested upgrade path, and the Kafka upgrade notes set minimum versions and Java requirements that must be checked first.
Consumer groups for analytics and operations
Operations consumers, such as alarm and state estimation feeds, need low latency and strict access. Analytics and reporting consumers can tolerate delay and larger reads. Separate consumer groups, and where needed separate topics or tenants, let each class of consumer read at its own pace without affecting the other. Authorization should be explicit per topic and per group. In Kafka's StandardAuthorizer, resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so confirm that this setting is false in the target configuration and that each consumer group has a named owner.
Cross-site replication between control centers
Replicating topics between control centers keeps data available when one site fails, but it also copies data across a boundary that may be regulated. Kafka deployments typically use MirrorMaker 2 for this, and Pulsar offers geo-replication. Decide which topics replicate, which direction and whether the replicated copy sits inside the same perimeter as the source. Consumer offset translation and failover steps must be tested, not assumed. Ask the partner to document recovery objectives and the last full switchover test, and to confirm how replication traffic is encrypted and authenticated across the link.
Monitoring tied to CIP-015-1 expectations
CIP-015-1 covers internal network security monitoring and is enforceable from 1 October 2028. Streaming platforms generate their own evidence, including authentication events, ACL changes, broker health, replication lag and consumer lag, and that evidence should flow into the monitoring program rather than sit on the brokers. Confirm which broker and client logs are collected, how long they are retained, and who reviews alerts for replication failures and unauthorized access attempts. Plan now for the monitoring design so it can be shown to an auditor when the standard becomes enforceable.
Evidence auditors and regulators ask for from the streaming layer
This table maps the evidence a CIP assessment typically samples from a streaming layer to the component that produces it and the team that owns it. Use it to decide who prepares each item before an audit, and treat it as a starting point to confirm with your compliance team and assessor. DeployOpen prepares this evidence, but independent auditors assess it, and there is no CIP certificate.
| Evidence | Producing component | Owner |
|---|---|---|
| ESP diagrams and routable-connected asset inventory (CIP-005-7) | Network architecture documentation, firewall and segmentation configuration, cluster broker inventory | OT security and network engineering |
| Network flow records crossing the perimeter | Firewall and boundary device logs, Intermediate System session records for interactive remote access | Network operations and security monitoring |
| Broker and client ACLs and their change history (CIP-005-7, CIP-007-6) | Kafka StandardAuthorizer ACLs in the cluster metadata log, change tickets, version-controlled ACL definitions | Platform engineering with OT security approval |
| Authentication configuration, including MFA for remote access | SASL mechanism settings, identity provider integration, Intermediate System MFA logs | Identity team and platform engineering |
| Encryption settings for client, broker and inter-site traffic | TLS listener configuration, certificate inventory and rotation records, replication link encryption settings | Platform engineering and PKI owner |
| Software integrity and vendor patch records (CIP-013-2) | Release checksums and signature verification records, upgrade and patch change log, vendor notification register, supply chain risk plan review record | Supply chain risk owner and platform engineering |
| Audit logs and retention (CIP-007-6, CIP-015-1 where applicable) | Broker, authorization and admin logs shipped to a central store, retention policy settings, log integrity checks | Security operations, with retention set by compliance |
| Replication and failover test records | Cross-site replication status, switchover test plans and results, offset translation checks, recovery time findings | Operations and resilience team, with control center sign-off |
What utilities ask before hiring a streaming partner
What drives the effort and cost of an energy streaming build?
The main drivers are the number of data sources, the volume of smart meter and grid sensor events, how long data is retained, and how many environments must be built and tested. Cost also rises with the number of systems that consume the streams, the replication design across control centers, and the amount of CIP documentation and evidence the work must produce. Costs fall into one-time work (architecture, build, testing, cutover) and recurring work (hardware or cloud capacity, licenses where they apply, patching, monitoring and on-call). Ask for a written scope that separates these lines and states which environments, sites and data feeds are included. The Kafka and Pulsar projects are Apache-2.0 licensed with no paid edition from the project itself, so the license is not usually the cost driver. Ask the partner to show the assumptions behind any figure in the quote.
How long does a utility streaming rollout take?
The timeline depends on the impact classification of the systems that will connect, the number of sites, the depth of the security review, and how many change approvals the utility's own processes require. Work that touches BES Cyber Systems usually needs more time for design review, testing in a non-production environment, and coordinated cutover windows. Ask for milestones in the written scope with named dependencies on the utility's team, network changes and procurement. Because the CIP standards in force set specific dates, such as CIP-003-9 becoming enforceable on 1 April 2026 and CIP-012-2 on 1 July 2026, confirm with your compliance lead which requirements apply to the systems in scope before fixing dates. We do not give a duration here; the partner's written plan should state one.
What must the utility's own team provide before a streaming project starts?
The utility needs to supply several inputs that no partner can produce alone. These usually include:
- An impact analysis of the systems that will send or receive data, identifying high, medium or low impact BES Cyber Systems under CIP-002-5.1a
- Current network diagrams showing routable connections, Electronic Security Perimeters and any Intermediate System used for remote access under CIP-005-7
- A named CIP compliance owner who can approve design decisions and sign off on evidence
- Access to OT and IT change processes, vendor contacts and data owners for each feed
Without these, design decisions about where brokers sit and how they cross the perimeter will be guesses. Ask the partner to list these inputs in the written scope so both sides know what is required and by when.
Can our internal team run a self-hosted streaming platform after handover?
It can, if the team has or builds skills in Kafka or Pulsar operations, Linux, TLS and certificate management, monitoring, and upgrade planning. Kafka 4.0 supports only KRaft mode, and rolling upgrades are supported from the last three releases to the latest version, so an upgrade plan is a standing operational task. Pulsar offers LTS releases with 24 months of active support and 36 months of security support, which affects how often upgrades must happen. Ask the partner what the handover includes: runbooks, alert definitions, backup and restore procedures, and training. Also ask whether ongoing support is offered as a retainer, which is available from DeployOpen for its own engagements, and compare that with your team's on-call capacity.
How do CIP audits affect the build schedule?
Audits affect the build in two ways: the design must be defensible when an assessor reviews it, and the evidence must exist before the audit. Entities registered as a Balancing Authority, Reliability Coordinator or Transmission Operator are audited at least once every three years, so change records, access reviews, configuration baselines and supply chain documentation should be produced as the work happens rather than reconstructed later. CIP-013-2 requires supply chain risk management plans that cover vendor software integrity verification and reviews at least every 15 calendar months, which applies to the streaming vendor and partner. Ask the partner which evidence it prepares during delivery and how that evidence maps to the CIP requirements you are subject to. DeployOpen prepares systems, controls and evidence for compliance assessments but does not issue audit reports or certifications.
How do NERC penalties and audit cadence affect urgency?
Penalty exposure and audit frequency set the internal priority for compliance work, not the technical design itself. The maximum civil penalty is $1,584,648 per violation per day under 18 CFR 385.1602, and that figure is adjusted over time, so confirm the current value in the eCFR. Because a violation can continue across days, delays in documenting perimeters, access controls or supply chain controls can carry material risk. Ask the partner to identify which controls the streaming design affects and to flag any gap before go-live. Urgency should be set by the utility's risk assessment and the enforceable dates for each standard, which NERC publishes on its CIP standards page.
What does a partner's written scope need to cover for a utility project?
The written scope should list the data feeds, sites, environments and CIP-relevant systems included, along with the deliverables and acceptance criteria for each. It should state how replication across control centers is designed and tested, how the Electronic Security Perimeter boundary is handled, what documentation is delivered, and what the utility's team must provide. It should also name exclusions, such as any change to live SCADA systems, and explain how changes after signing are priced. DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, and most engagements are fixed-scope and agreed in writing before work starts. Ask for the written scope before comparing quotes.
Questions to ask a streaming implementation partner
Start with CIP experience: ask which CIP standards the partner has implemented for control centers, and how they document the impact classification of each BES Cyber System under CIP-002-5.1a before any broker is placed. A credible answer names the bright-line criteria they applied, shows a worked classification for a sample system, and says who at your organization signs off. Ask how they record whether streaming components are routable-connected to high or medium impact assets, and whether they can show the Electronic Security Perimeter diagram that CIP-005-7 would require. If they cannot explain how a Kafka broker or Pulsar bookie that touches SCADA-derived data would be classified, treat that as a serious gap.
Next, ask how they keep the platform's placement reviewable over time. Request the boundary decision record, the list of proxies, Interactive Remote Access paths and Intermediate System configuration, and the process for re-evaluating placement when a new topic or consumer is added. On upgrades, ask for a written path that starts from your current version and accounts for the Kafka 4.x requirement to run KRaft, the recommendation to move KRaft clusters older than 3.3.x to 3.9.x first, and Java 17 for brokers and Kafka Connect. If you run Pulsar, ask how they will schedule moves relative to the Pulsar 4.0 LTS active support end date of October 21, 2026 and the Pulsar 5.0 LTS dates. Ask who owns the CIP-013-2 vendor evidence for each patch.
Finally, ask for replication and failover evidence and for handover material. Request the most recent failover test report across control centers, including the consumer offset handling and the measured recovery results, and ask whether the test covered a loss of the primary site rather than a planned switchover. Ask for the runbooks, configuration baselines, ACL exports, monitoring dashboards and access logs they will hand over, and confirm that you receive them in a form your compliance team can produce to an auditor. The answer that should worry you is any partner who offers a failover demonstration but cannot provide the evidence file.
How DeployOpen scopes and supports a utility streaming platform
DeployOpen is a Chainscore Labs company. It plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside the client's own infrastructure, including its cloud accounts, data centers and isolated networks. For a utility, that means the streaming cluster that receives smart meter, grid sensor and SCADA-derived data runs where the utility already controls access, change management and records. DeployOpen does not operate a shared hosted service for this work.
- Own infrastructure: Brokers, storage and supporting services are deployed into the utility's data centers or accounts, under its network and identity controls.
- Full lifecycle scope: Planning, build, hardening, backup design, documentation and handover are all part of the work, not only installation.
- Security testing: Applications, APIs, cloud and infrastructure can be tested, and the same team can run red team exercises against the platform.
- Training and cyber ranges: Private cyber ranges and security training can be used to rehearse operator and incident response procedures.
- Limits of the role: DeployOpen prepares systems, controls and evidence for compliance assessments but does not issue audit reports, attestations or certifications.
Check which of these services are in your scope and confirm with the vendor that any utility-specific product features you depend on are available in the release you choose.
Most engagements are fixed-scope. Scope, deliverables and timelines are agreed in writing before work starts, so the utility knows what it is buying and which items are excluded. Because DeployOpen does not publish a price list, the quote follows from the written scope. Utilities should expect the scope document to name the environments, sites and data flows in play, since a streaming platform touching a control center differs materially from one serving only business analytics.
- Written scope first: Environments, sites, data sources, consumers, retention targets and acceptance criteria are recorded before implementation begins.
- Defined deliverables: Each deliverable, such as a reference architecture, runbook or test report, is listed with its acceptance evidence.
- Explicit exclusions: Work on OT devices, SCADA vendor systems or upstream meter head-ends is either in scope or stated as out of scope.
- Quote basis: Ask for the quote to break out design, build, testing, documentation and handover so the cost components are visible.
- Change control: Ask how scope changes are priced and approved once work is underway.
Before signing, confirm which party owns each interface, which environments are covered by the fixed price and which items would be billed as change requests.
After go-live, the platform still needs upgrades, patches, monitoring follow-up and periodic testing. These are available as retainers, which recur on a defined basis and cover the work that keeps the cluster supported. For a utility, the recurring element matters because Kafka and Pulsar release on their own schedules, and the utility's compliance obligations also change over time.
- Upgrade support: Kafka's supported releases and upgrade paths are documented by the project, so upgrade planning is a recurring activity rather than a one-off project.
- Continuous testing: Retainers can include recurring security testing, so new exposures are found between assessments.
- Operational handover: Runbooks, alert definitions and on-call procedures are handed to the utility's own staff, with the retainer covering escalation.
- Recovery exercises: Restore and failover tests can be scheduled as part of the retainer, with results recorded for the utility's evidence file.
- Retainer terms: Ask for the response expectations, coverage hours and what happens to upgrades that fall outside the retainer.
Confirm the service levels in writing. DeployOpen's documents and quotes, not this page, define what the retainer includes.
For utilities subject to NERC CIP, DeployOpen can prepare the systems, configurations and records that an auditor will sample, such as access lists, encryption settings, change records and test results. It does not decide whether the utility is compliant. The registered entity and its auditor make that determination, and the NERC compliance program sets the obligations. Evidence preparation is therefore scoped as a specific deliverable with named standards, not a general promise of compliance.
- Named control mapping: Map configuration and records to specific CIP requirements, such as the Electronic Security Perimeter rules in CIP-005-7 and supply chain expectations in CIP-013-2.
- Hardening baseline: Document TLS, SASL authentication and ACL configuration with the settings used, so they can be reviewed.
- Evidence pack: Produce the test reports, diagrams and change logs the utility's auditors are likely to request.
- Information protection: Handle configuration details under the utility's information protection program, consistent with CIP-011-3.
- Independent assessment: Ask the utility's compliance lead to confirm which evidence formats their auditor accepts before the work starts.
Check the mapping with your compliance team and registered entity documentation before relying on it.
