Apache NiFi for energy and utilities

Apache NiFi for energy and utilities: why the build starts at the control boundary

Utilities need field telemetry, substation data and equipment health signals in analytics and business systems, but those signals originate inside environments governed by NERC CIP. Apache NiFi fits as a routing and collection layer when the design treats the boundary between BES Cyber Systems and everything else as the first architecture decision, not a later review item. The flow design, the placement of NiFi nodes and the direction of every connection are set by where that boundary sits. Before any processor is placed, the team should confirm with its compliance lead which assets are high, medium or low impact under CIP-002-5.1a, and which of them have routable connectivity, because that determines which Electronic Security Perimeter the NiFi components may cross.

What changes compared with a generic NiFi build is the direction of data and the place where it lands. In most enterprise flows, data moves freely between systems. Here, the preferred pattern is outbound-only from the control environment, with collection points on the inside that push data toward a boundary component and never accept inbound sessions from the business side. Edge sites such as substations, generation plants and remote telemetry locations add a second constraint: the NiFi instance at each site must buffer through link loss and must not depend on a central cluster being reachable. Teams should confirm whether their MiNiFi deployments at these sites can run their flows without the central NiFi cluster, and document how buffered data is recovered after an outage.

The evidence auditors expect is different from what a generic build produces. Auditors test whether Interactive Remote Access passes through an Intermediate System with encryption and multi-factor authentication under CIP-005-7, whether vendor-initiated access and software integrity are covered by a supply chain plan under CIP-013-2, and whether changes to the flows are controlled and traceable. NiFi provenance, Git-versioned flow definitions and the access policies on each NiFi component can supply part of that evidence, but they do not establish compliance on their own. The outcome to aim for is a design where each flow can be shown to sit on one side of a defined perimeter, with records that a Compliance Enforcement Authority could follow.

Which energy and utility requirements shape the NiFi design

This table maps each NERC CIP and NIS2 requirement relevant to energy operators to the NiFi design decision it affects. Read each row as a question to settle before flows are placed: where the data sits, how remote access reaches NiFi, and where software comes from. The main takeaway is that NiFi placement and access paths are compliance decisions, not only data engineering choices.

Requirement	What it asks for	What it means for NiFi
CIP-002-5.1a impact categorization	Sorts BES Cyber Systems into high, medium or low impact using bright-line criteria in Attachment 1	Confirm the impact category of each system a flow touches before deciding which NiFi nodes may connect to it. Medium impact examples include generation of 1500 MW or more at one plant.
CIP-005-7 Electronic Security Perimeter	Requires routable-connected high and medium impact assets to sit inside an ESP, with Interactive Remote Access passing through an Intermediate System using encryption and multi-factor authentication	Place NiFi nodes that collect from control networks inside the ESP boundary. Route any operator access to NiFi UIs through an approved Intermediate System, not directly from corporate networks.
CIP-013-2 vendor and software integrity	Supply chain risk management plans covering vendor incident notification, vulnerability disclosure, software integrity verification and vendor remote access, reviewed at least every 15 calendar months	Record the NiFi release, checksums and any custom processor sources in your supply chain plan. Confirm vendor notification and vulnerability disclosure paths for any support partner who touches the deployment.
CIP-011-3 information protection	Requires protection of BES Cyber System Information through documented methods for storage, transit and disposal	Treat provenance repositories, flow configuration files and parameter contexts holding sensitive values as protected information. Limit where content and provenance data are stored and who can read them.
CIP-015-1 internal network monitoring	Approved by FERC on 26 June 2025 and enforceable from 1 October 2028, with CIP-015-2 extending monitoring to associated EACMS and PACS with an effective date of 1 October 2029	Plan NiFi flows that move network monitoring data to a monitoring platform early, because the enforcement date sets the window for building and testing that path. Confirm the scope version that applies to your entity.
CIP-012-2 real-time data availability	Adds protection of the availability of Real-time data between Control Centers, enforceable from 1 July 2026	Design inter-control-center NiFi flows with redundant clustered nodes and tested failover, since a single NiFi node failure should not interrupt Real-time data paths. Confirm the availability expectations with your compliance lead.
NIS2 Annex I energy coverage (EU)	Lists energy as a high-criticality sector with electricity, district heating and cooling, oil, gas and hydrogen subsectors	EU operators should confirm their subsector and national transposition before scoping NiFi. Apply incident and supply chain expectations to NiFi hosting, updates and support providers.

Where NiFi sits in a utility network: telemetry, SNMP and substation edges

The first decision is where the NiFi processing runs. A utility network typically produces three kinds of data: SNMP counters and traps from network gear such as routers, switches and serial-to-IP converters; telemetry from field devices and substation equipment such as protective relays, meters and remote terminal units; and the destinations that consume it, which are usually a historian, an analytics platform or a SIEM. A flow designer should map each source to the protocol and interface it uses, then ask which Electronic Security Perimeter (ESP) the source sits inside. Confirm each NiFi processor against your own equipment and protocol versions before committing to a design, because the NiFi documentation describes what a component can do, not what your devices and firmware expose. Treat any processor you cannot test against a representative device as an open risk.

Placement is a trade-off. A MiNiFi agent at the substation can collect locally, buffer during WAN outages and send only what the flow needs, which reduces the number of paths that cross the boundary. The cost is that every edge host becomes part of your configuration, patching and monitoring scope, and it must be hardened as a cyber asset in its own right. A central NiFi cluster outside the ESP is simpler to operate and easier to secure, but raw field traffic then has to cross the perimeter, and the cluster itself must be treated as a system that depends on that path. Most designs split the difference: local collection and buffering in the field, and aggregation centrally. Ask your compliance lead early which assets are in scope for each CIP standard, because that answer determines where the boundary can sit.

Traffic that crosses the boundary needs deliberate, one-directional design. Each flow path should be defined by its source, destination, protocol, port and data type, and should carry data only in the direction it was approved for. Inbound commands to field devices should not be possible through the telemetry path, and the NiFi cluster outside the perimeter should not hold credentials that reach back into control systems. Plan how the boundary crossing is inspected and logged, and document the data flow diagrams that your auditors will ask for. Use NiFi provenance and flow versioning as evidence of what moved and which flow version moved it, but confirm with your compliance team how those records need to be retained and protected under your own CIP program.

Choosing a NiFi topology for control and corporate networks

In this placement the NiFi cluster sits in the corporate network and receives data from the operational side only through a one-way diode or a controlled export point. Nothing in the corporate zone opens a session back into the control environment, so the cluster never needs a route into field equipment. The design question is what crosses the boundary and who controls it.

Isolation: Document the diode or export host as the only permitted path, with the data direction, protocol and the owner of the transfer host.
Authentication: LDAP tied to your corporate directory, or OpenID Connect or SAML 2.0 through your identity provider, usually works well here. X.509 client certificates suit machine-facing endpoints.
Coordination: An embedded or external ZooKeeper ensemble, or Kubernetes using ConfigMaps or Leases, both appear in the administration guide. Pick one and record the failure behavior.
CIP evidence: Show that the corporate cluster does not hold High or Medium impact BES Cyber System access paths, and capture the transfer control configuration.

Confirm with your compliance team which classification each exported data set carries before the diode rules are written.

Which NiFi modules and settings matter for utility data pipelines

A utility NiFi deployment is shaped by a small set of runtime, security, coordination and evidence choices made early. Each card below pairs one of those choices with the operational concern it affects on a substation, control center or corporate network, and the check to run before the flow goes live.

Java 21 runtime and Python caveats

NiFi 2 requires Java 21 as its minimum runtime, so field servers, golden images and patch baselines must be confirmed against that version before installation. Python 3.10, 3.11 or 3.12 is needed only for the optional Python-based processors, which the administration guide describes as a beta feature. In a utility, a beta processor on a pipeline that feeds operational data needs a documented risk acceptance and a test run against representative telemetry. Ask which processors your design depends on and confirm their status with the vendor docs before committing.

Flow Registry Clients with change review

NiFi 2 includes Flow Registry Clients for GitHub and GitLab that store flow versions in a Git repository. For utilities, this gives a reviewable change history for every collector, parser and routing rule that touches field or control-environment data. Pair the repository with a review gate, so a flow change is approved by someone outside the team that made it. Check which repository hosts are approved inside your environment, because a registry hosted outside the control perimeter may itself be an asset that needs assessment.

flow.json.gz and ENVIRONMENT parameter scope

NiFi 2 stores the flow in flow.json.gz rather than flow.xml.gz, and the variable registry scope was replaced by an ENVIRONMENT scope. Migrated flows that reference old variable names or distributed map cache components will fail or behave differently. For utilities, the practical risk is a silent change in which endpoint, credential or site identifier a flow uses after promotion from test to a production site. Verify each environment's parameter values and compare the exported flow against the expected endpoints before each promotion.

TLS and authentication with loopback default

Without a security configuration, NiFi binds to 127.0.0.1, so the UI is reachable only from the local host. Once you open it to operators, choose an authentication provider from single-user, LDAP, OpenID Connect, SAML 2.0 or X.509 client certificates, and avoid relying on the deprecated Kerberos provider. Operators who reach collectors at substations need the same controls as the control-center team, and shared administrator accounts should not be the norm. Confirm TLS certificate ownership, provider mapping and the authorization policies for each process group before go-live.

Cluster coordination choices

NiFi clusters coordinate through ZooKeeper, either embedded or external, or on Kubernetes using ConfigMaps or Leases for state and leader election. The choice decides how a utility handles a site outage, a control-center failover and the network path between nodes. An external ZooKeeper ensemble adds a component to patch and monitor, while Kubernetes-based coordination depends on the cluster's own control plane being in the same trust zone. Map every coordination path to the network segments it crosses, and test node loss during a planned maintenance window.

Provenance as evidence of data movement

NiFi provenance records where each piece of data came from, how it was transformed and where it was sent. For utility operators, provenance can show when a telemetry batch left a monitored boundary, which supports change and incident reviews under the CIP standards. Provenance is only as useful as its retention and access settings, and it can hold sensitive values if flows are not designed carefully. Decide the retention period with your compliance lead, restrict who can query it, and confirm that records are kept on a storage tier your audit team can reach.

How a utility NiFi rollout moves from asset inventory to audited operation

This sequence takes a utility from an inventory of field data sources to a NiFi deployment that the operator's CIP program has classified, bounded, tested and reviewed before it carries live telemetry. Each step produces a record that the compliance and operations teams can check.

Inventory data sources and classify impact
The operator's CIP program owner and OT engineers list every device, SNMP agent, telemetry feed and substation edge system the flows will touch, then map each one to a BES Cyber System category under CIP-002-5.1a. The output is a signed asset register that names the impact level for each source and records which systems are in scope for CIP requirements. The step is done when the CIP owner accepts the register and no flow is designed against an unclassified source.
Document the boundary and remote access paths
Network and security architects draw where NiFi nodes, edge agents, collectors and any data egress points sit relative to the Electronic Security Perimeter, and record every path that can reach them remotely. Where high or medium impact assets are routable-connected, they confirm that interactive remote access routes through an Intermediate System with encryption and multi-factor authentication, as CIP-005-7 requires. The output is a boundary diagram and a remote access inventory that the CIP program can test against. The step is done when each data path is either inside a documented perimeter or explicitly justified and approved.
Record vendor and supply chain obligations
Procurement and the CIP owner confirm which software components, including NiFi, its extensions, base images and any third-party processors, fall under the supply chain risk management plan. They check how the operator will receive vulnerability notices, verify artifact integrity and coordinate any vendor access, as CIP-013-2 describes at a high level. The output is a component list with provenance and an update process tied to the plan. The step is done when the plan's review cycle has an owner who will reassess it.
Build flows in a non-production environment
Engineers build the flows, parameters and any custom processors in a staging NiFi cluster that mirrors the production configuration without connecting to live control networks. Every change is committed to a Git repository through a flow registry client, with reviewed merge requests so the flow history can be traced for change management. The output is a versioned flow set, a parameter map that separates secrets from configuration, and a build record tied to a commit. The step is done when a fresh deployment from the repository reproduces the staging flows exactly.
Test failure, buffering and recovery
Test engineers simulate lost links to substations, slow collectors, duplicate telemetry and node failure in the cluster, and confirm that queues back-pressure without dropping data the operator must retain. They check that provenance records show what happened to each record and that the flows resume cleanly after a restart. The output is a test report with pass or fail results for each scenario and the queue limits chosen from those results. The step is done when every failure case has a documented expected behavior and the team has observed it.
Obtain security and compliance review
The security team reviews the cluster configuration, TLS and authentication settings, authorization policies, sensitive value handling and the logging of administrative actions against the CIP requirements in scope. The CIP program owner reviews the evidence package, which includes the asset register, boundary diagram, remote access inventory, flow repository history and test report. Findings are logged and resolved before approval. The output is a signed review record that names open items and their owners. The step is done when the CIP owner accepts the evidence and any remaining gaps have a tracked remediation. Confirm with the operator's own auditors which evidence they expect, since DeployOpen prepares evidence but does not issue audit reports.
Cut over with monitoring in place
Operations staff move the field sources to production flows in stages, starting with the least critical assets, while the monitoring and alerting for NiFi nodes, queues and provenance errors is already live. They keep the previous collection path available until each stage shows stable data delivery. The output is a cutover log, a monitoring dashboard the operations team uses, and a rollback procedure that has been rehearsed. The step is done when all in-scope sources report through production flows and the rollback path is confirmed to still work.
Hand over for ongoing operation
The operator's operations team takes ownership of the cluster, runbooks and change process, and the CIP owner schedules periodic reviews of the asset register, boundary and supply chain plan. Any flow change goes through the same Git review and staging test path used during the build. The output is a handover document, a runbook set and an agreed review calendar. The step is done when the operator's team has run one change through the full process without outside help.

Compliance considerations for NiFi in NERC CIP environments

A NiFi flow that touches a BES Cyber System or moves telemetry out of a control environment falls under NERC CIP obligations. The cards below cover the standards that most often shape the design, and the evidence a compliance team will ask for. DeployOpen prepares systems, controls and evidence for assessment, but does not issue audit results.

Perimeter and interactive remote access

CIP-005-7 requires routable-connected assets of high and medium impact BES Cyber Systems to sit inside an Electronic Security Perimeter, and it requires all Interactive Remote Access to pass through an Intermediate System with encryption and multi-factor authentication. For NiFi, confirm which nodes and the NiFi UI sit inside the perimeter, and whether any operator reaches the flow canvas from outside it. Document the path before the first deployment.

Supply chain and software integrity

CIP-013-2 requires supply chain risk management plans that cover vendor incident notification, vulnerability disclosure, and software integrity and authenticity verification, reviewed at least every 15 calendar months. For NiFi, this means recording the source and version of the core release, every custom processor, controller service and scripted component, and how each artifact was verified before it entered production. Ask your vendors how they notify you of vulnerabilities in their code.

Information protection for telemetry

CIP-011-3 governs how BES Cyber System Information is identified and protected, including when it is stored, transmitted or disposed of. SNMP traps, field telemetry and flow provenance records can contain system details that fall under this scope. Classify each data path in the flow design, restrict who can read provenance and content, and confirm that repositories and backups holding this data get the same protection as the source system.

Internal network monitoring timeline

CIP-015-1 covers internal network security monitoring and was approved by FERC on 26 June 2025, with enforcement from 1 October 2028. CIP-015-2 extends monitoring to associated EACMS and PACS and has an effective date of 1 October 2029. Plan NiFi's log and provenance outputs so they can feed a monitoring platform, and confirm the enforcement date that applies to your entity before you size that work.

Who runs the audit

There is no CIP certificate. Entities registered as a Balancing Authority, Reliability Coordinator or Transmission Operator receive a compliance audit at least once every three years, and the Compliance Enforcement Authority, a Regional Entity or NERC, monitors compliance through audits, spot checks, self-certifications and self-reports. Ask your compliance team which of these methods will apply to your NiFi environment, and what evidence each one will request.

What utilities ask before building NiFi pipelines

How much effort does a CIP-scoped NiFi build take, and what drives it?

Effort depends mostly on which BES Cyber Systems the flows touch, not on NiFi itself. Under CIP-002-5.1a, each system is sorted into high, medium or low impact using bright-line criteria, and the higher the category, the more controls apply to the flows that connect to it. The main drivers are the number of data sources, whether any flow crosses an Electronic Security Perimeter, how much evidence the compliance team needs for each control, and the number of field sites that need edge collection. Ask any partner to break the scope down by impact category and by flow, and to list which CIP requirements each flow is mapped against. No duration is given here because none has been supplied for this work; ask for one in the written scope.

Can our own team build and run NiFi for CIP-scoped data?

A team can do it in-house if it already has NiFi engineers, OT network knowledge and staff who understand the CIP requirements that apply to its systems. The gaps usually appear in three places: mapping flows to the Applicable Systems tables, producing evidence that auditors accept, and keeping NiFi patched and configured the same way across clusters. Many utilities split the work, keeping control of design decisions and evidence while bringing in a partner for build, hardening and documentation. DeployOpen plans, implements, hardens, documents and supports self-hosted NiFi inside the client's own infrastructure, under a written, fixed scope. It does not issue audit reports or attestations; independent auditors and the Compliance Enforcement Authority make those determinations.

What do our OT and compliance staff need to provide?

The build needs named people from three groups. OT engineers must document the field equipment, protocols such as SNMP, and the network paths each collector uses. Network and security staff must confirm where the Electronic Security Perimeter sits and which systems are inside it. Compliance staff must state which CIP standards apply, based on the entity's registration and impact categories, and which evidence they expect. Ask for these inputs before the scope is written, because each missing item becomes a change request later. Access to test systems and a documented change process are also needed, since changes to flows on in-scope systems are themselves subject to configuration management controls.

Can NiFi sit inside a BES Cyber System boundary?

The answer depends on the impact category and the connectivity of the system, and it must be decided by the entity with its compliance team. CIP-005-7 requires routable-connected assets of high and medium impact BES Cyber Systems to sit inside an Electronic Security Perimeter, and requires all Interactive Remote Access to pass through an Intermediate System with encryption and multi-factor authentication. Placing NiFi inside or across that perimeter changes which controls apply to it. A common pattern is to keep collection at the edge and move only the needed data outward through controlled boundaries. Confirm the placement against CIP-005-7 and your asset list before any design work starts, and record the decision in the design documentation.

How should we handle an existing NiFi 1.x estate?

Plan the move to NiFi 2 as a project with its own scope, because 1.x cannot be maintained further. The NiFi project explains that the 1.x line depends on Jetty 9.4, Spring Framework 5.3 and AngularJS 1.8, which cannot be upgraded within that line. The migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0, and changing the flow configuration file from flow.xml.gz to flow.json.gz. In NiFi 2.0 the variable registry expression language scope was removed and replaced by an ENVIRONMENT scope, and the distributed map cache components were renamed. Inventory custom processors and scripts first, since those are the most common breakpoints, and run the upgrade in a non-production cluster before touching in-scope systems.

What should a quote for a NiFi utility project include?

A quote should follow from a written scope that lists each element separately. Ask for the following:

The in-scope systems, grouped by impact category, and the CIP standards mapped to each flow.
Deliverables, such as flow designs, hardening records, runbooks and evidence packages.
Which costs are one-time, such as design and build, and which recur, such as patching, upgrades and support retainers.
Who performs each task, the client or the partner, and what the client must supply.
Assumptions about test environments, change windows and access.
Acceptance criteria and how changes to scope are priced.

DeployOpen does not publish a price list. A quote follows from the written scope.

Which NIS2 or EU rules matter for utilities outside the United States?

NIS2 Annex I lists energy as a sector of high criticality, covering electricity, district heating and cooling, oil, gas and hydrogen. Operators in EU member states should confirm how their national transposition applies to them. Commission Delegated Regulation (EU) 2024/1366 also establishes a network code on sector-specific cybersecurity rules for cross-border electricity flows, supplementing Regulation (EU) 2019/943. Because these rules differ from the NERC CIP framework, mapping NiFi flows should be done against each applicable regime separately. Confirm the obligations with your national authority and legal counsel before scoping work.

What to ask a partner before a NiFi utility engagement

Start with NERC CIP scoping, because it decides what the partner is allowed to touch. Ask which BES Cyber Systems your flows would connect to, and how the partner determines impact categories under CIP-002-5.1a before any NiFi node is placed near them. Ask whether they have scoped flows against Electronic Security Perimeters under CIP-005-7, including where a NiFi cluster sits relative to the perimeter and how Interactive Remote Access would reach it. Ask for a written list of the Applicable Systems their design assumes, and for the name of the person who will sign off that list. Confirm the scope against your own compliance program and your Compliance Enforcement Authority's expectations, since no partner can certify a CIP status on your behalf.

Next, ask how they document flows for auditors. A useful answer names the artifacts: flows versioned in a Git repository through a Flow Registry Client, change records tied to each commit, parameter and sensitive value handling, and provenance settings that show where telemetry came from and where it went. Ask how they handle the supply chain items in CIP-013-2, such as software integrity checks on NiFi and MiNiFi packages and vendor remote access to the platform. Ask who keeps the evidence and how long it is retained. Also ask about the Python processor beta status. The NiFi administration guide describes Python-based processors as a beta feature, so a partner should explain whether they keep those processors out of control-environment flows, which Python versions they support, and how they track changes to that beta.

Finally, ask how they support NiFi 2 upgrades and the NiFi Registry deprecation. NiFi 1.x can no longer be maintained because of its Jetty, Spring and AngularJS dependencies, and the migration path runs through 1.27.0 before 2.0.0, with the flow file changing from flow.xml.gz to flow.json.gz. Ask whether the partner will inventory your flows and custom processors for removed variable registry scope, renamed distributed map cache components, and Python usage before any upgrade begins. Ask how they plan to move from NiFi Registry to a Git-based Flow Registry Client, since NiFi Registry is planned for removal in NiFi 3.0. Request a written upgrade scope, a rollback plan and a test environment that mirrors your substation or control center segmentation.

How DeployOpen scopes and delivers NiFi for energy and utilities

DeployOpen plans, implements, hardens, documents and supports Apache NiFi inside the operator's own networks and data centers. For energy and utilities, we start from the operator's compliance boundary and network zones, then build flows, edge collection and access controls against a written scope agreed before work begins.

How we scope

Inventory the telemetry sources, SNMP devices and edge sites that the flows must reach.
Map each data path against the operator's impact classification and network zone boundaries.
Agree in writing which flows, environments, integrations and deliverables are in scope, and which are not.

How we build

Build NiFi flows for SNMP and telemetry collection with explicit error routing and buffering for unreliable links.
Configure TLS and an authentication provider suited to your identity stack, and test access paths before go-live.
Version flows in Git through a registry client, and test changes in a non-production cluster before promotion.

How we hand over

Deliver flow documentation, data path diagrams, configuration records and provenance and logging settings.
Document the boundary controls, change procedures and evidence your compliance team needs for its own assessment.
Walk your operators through monitoring, alerting, backups and recovery for the NiFi cluster and its repositories.

How we support

Offer ongoing support, planned upgrades and continuous testing as a retainer, scoped separately from the build.
Plan NiFi 1.x to 2.x migration work as its own written scope, covering flow file and configuration changes.
Confirm with us and your auditors which evidence is produced by DeployOpen and which is produced by your own teams.

Apache NiFi for energy and utilities