Apache NiFi for manufacturing

NiFi and MiNiFi can collect plant and machine data, buffer it over unreliable links and feed historians and analytics while keeping OT and IT networks separate. This page covers the requirements, design choices and rollout a manufacturer should plan.

On this page

Apache NiFi for manufacturing: why plant data pipelines need their own design

Apache NiFi, with MiNiFi at the edge, fits the work of moving plant and machine data out of production networks, but only when the design starts from the plant rather than from a generic integration template. Machine data sits in PLCs, MQTT brokers, local historians and edge devices, and every move across the boundary between operational technology (OT) and IT is a decision that someone in the plant must own. The engineering lead and the IT lead need to agree on which data crosses, in which direction, and through which gateway before any flow is built.

Generic data integration assumes reliable networks, stable schemas and a single owner for both ends. Plant data breaks each of those assumptions. Industrial links drop for hours, buffered data must survive restarts without duplicating or reordering readings, and the same tag can arrive with different units or quality flags depending on the device firmware. The segmentation of OT and IT networks also changes the architecture: the edge agent must initiate outbound connections, the hop into IT must be narrow and monitored, and the flows that handle control-adjacent data need change control that the plant's safety and quality teams can review. The ISA/IEC 62443 series is the usual reference for how zones and conduits are defined, and it should be read with the asset owner's own risk assessment.

Regulation adds a second layer. NIS2 lists manufacturing as a critical sector, so an in-scope manufacturer must be able to report significant incidents on a fixed clock, which means the pipeline needs logs and provenance that show what moved and when. The Machinery Regulation, which applies from 14 January 2027, requires machinery to be protected so that connecting another or remote device does not create a hazardous situation, so any new data path into a machine's control environment needs a documented safety review. Confirm with your legal and safety teams which of these obligations apply to your plants before scoping the work.

Which manufacturing requirements shape the NiFi build

This table maps the security, incident reporting, product and performance requirements that apply to plant data flows to what each one changes in a NiFi design. Read each row as a question to settle before the flow layout is drafted, since the answers decide where NiFi sits, what it may reach and how it is changed. The main takeaway is that NiFi placement and change control are compliance decisions as much as engineering ones.

RequirementSourceWhat it means for the NiFi build
Security levels and zonesISA/IEC 62443 series (ISA)Group NiFi and MiNiFi agents into defined zones, set the security level each zone needs, and document every conduit that crosses a zone boundary. Confirm the target levels with your OT security owner before choosing hosts.
Security risk assessment for the system designANSI/ISA-62443-3-2-2020 (ISA)Assess the NiFi cluster and edge agents as part of the system design, including what happens if a flow host or its credentials are compromised. Record the result so the design can be traced back to it.
Asset owner security programANSI/ISA-62443-2-1-2024 (ISA)Assign owners for NiFi hosts, flows, certificates and changes, and include them in the plant's security program. Clear ownership is usually what assessors check first.
NIS2 manufacturing scopeNIS2 Annex II, Directive (EU) 2022/2555If your organization is in scope as a manufacturer under Annex II, NiFi flows that carry operational or security data become part of the incident response chain. Confirm your entity classification with counsel.
NIS2 incident reporting timelinesArticle 23(4), Directive (EU) 2022/2555The early warning is due within 24 hours, the notification within 72 hours and the final report within one month. Flow logs, provenance and alerting must be retained and retrievable fast enough to meet those windows.
Cyber Resilience Act reporting dutiesArticle 71(2), Regulation (EU) 2024/2847Manufacturers' Article 14 duties to report actively exploited vulnerabilities and severe incidents have applied since 11 September 2026. Any NiFi-based product or connected equipment you ship needs a documented way to detect and report these events.
Connected device protection for machineryAnnex III, 1.1.9, Regulation (EU) 2023/1230Applies from 14 January 2027. Machinery must not enter a hazardous state when another or remote device connects. Design NiFi connections to machines as read-only or tightly bounded, and verify this with the machine builder.
OT performance, reliability and safetyNIST SP 800-82 Rev. 3, September 2023Treat NiFi as a component that must not disturb control traffic. Size queues and back-pressure so the flow degrades without blocking plant systems, and test failover under realistic load before go-live.

How MiNiFi, NiFi and the historian split the work

At the machine edge, MiNiFi agents sit close to PLCs, sensors, gateways and MQTT brokers. Their job is narrow: collect readings, apply light filtering, and hold data locally when the link to the plant network drops. Two agent builds are relevant here. MiNiFi Java 2.12.0 suits hosts that can run a Java runtime with enough memory for the JVM. MiNiFi C++ 1.0.0 suits constrained devices where a native binary fits better. The right choice depends on the device fleet, so inventory operating systems, memory, CPU and available runtimes before you select either build.

  • Local buffering: Agents queue records on disk while the uplink is unavailable, so a plant network outage does not automatically mean lost telemetry. Size the disk for the longest outage you must survive.
  • Protocol reach: Confirm which processors each agent build provides for your devices, such as MQTT subscriptions or the industrial protocol gateways in your plant, before committing to a design.
  • Light transformation only: Keep logic at the edge to filtering, tagging and compression. Heavier parsing and enrichment belong in the plant cluster, where changes are easier to test and roll out.
  • Fleet management: Plan how flow definitions reach each agent, how versions are tracked and how an agent is replaced when a device fails.

Check that every device class in scope has a supported agent build, and test buffering by cutting the uplink for a planned period before go-live.

Processors, controller services and settings that matter on the factory floor

Plant data reaches NiFi through a small set of processors, controller services and sizing decisions that determine whether readings arrive complete and in order. The cards below cover the settings that most often decide that outcome, and each one should be confirmed against the NiFi 2 component documentation for the processor versions you deploy.

MQTT consume and publish settings

Plant brokers and edge gateways exchange machine telemetry over MQTT, so the consume and publish processors need deliberate choices for QoS level, clean session behavior and topic structure. Map each site's topic tree to a stable pattern such as site, line, cell and tag, and keep wildcard subscriptions narrow enough that one line's traffic cannot flood a shared flow. Confirm which QoS levels and session options the MQTT processors in your NiFi 2 release expose, and test broker reconnection before production so you know what happens to in-flight messages during a network drop.

Record readers for OPC and vendor payloads

Machine payloads arrive as JSON, CSV, XML or vendor-specific formats, and they rarely match the schema your historian expects. Record readers and writers, backed by a schema registry or an explicit Avro or JSON schema, convert those payloads into one record model before routing. Decide early who owns each vendor schema and how a changed field is detected, because a silent change in a controller's output is one of the most common causes of bad historian data. Confirm the reader options available in your NiFi 2 release for each format you receive.

Back-pressure and content repository sizing

When a link to the plant or the historian fails, NiFi keeps accepting data until queues fill, so back-pressure thresholds and content repository capacity define how much data survives an outage. Calculate the buffer your longest expected outage requires from the message rate and average payload size measured on the line, then size the content repository and its disk to that figure with headroom. Set queue back-pressure limits so the upstream MQTT or OPC collector slows before the disk fills, and verify the retention and archive settings in the NiFi administration guide for your version.

Historian and time-series write paths

Writing to a historian or time-series database is where batching, ordering and duplicate handling meet the plant's data model. Choose processors that write in batches sized to the historian's ingest limits, and set retry behavior so a rejected batch is held and retried rather than dropped. Tag names, units and timestamps should be normalized before the write, and the timestamp source (controller clock or device clock) should be fixed in the design. Ask the historian vendor which write interfaces and batch sizes it supports, and confirm the matching processors in your NiFi 2 release.

Parameter contexts per site

A factory group usually runs the same flow in several plants with different brokers, historian endpoints and credentials. Parameter contexts keep those values out of the flow definition, so one versioned flow can be promoted to each site with only the site's parameters changed. Mark credential parameters as sensitive, scope contexts to the process groups that use them, and document which values an operator may edit on the plant floor. Confirm how your NiFi 2 release handles parameter providers and sensitive values before you standardize on this approach.

Flow health reporting

Operators need to know when a line stops sending data, and a silent flow can look healthy in NiFi while the plant has no readings. Reporting tasks and bulletins can expose queue depth, back-pressure events, processor errors and the time since the last successful write, and those signals should reach the team that owns the plant's data. Decide which thresholds page someone and which only appear on a dashboard. Confirm which reporting tasks and metrics your NiFi 2 release provides, and check how they are delivered to your monitoring stack.

Separating OT and IT networks without losing the data

The NiFi tier should sit in a deliberate position between the plant (OT) zone and the enterprise (IT) network, not inside either one. In the ISA/IEC 62443 zone and conduit model, the plant's data collection and buffering components usually belong to a demilitarized zone or intermediate zone, where the conduits crossing into OT and into IT are defined and monitored. The direction of every flow connection should follow the security design, not the convenience of the developer. Confirm the zone boundaries, allowed ports and conduit rules with the plant network owner and the team that maintains the ISA/IEC 62443 zone model before any firewall change is requested. The ISA/IEC 62443 series is the reference set for those decisions, and NIST SP 800-82 Rev. 3 covers the operational constraints that should shape them.

Where the security design permits it, prefer flows that initiate from the more trusted side toward the less trusted side. A pull model, where a NiFi node in the DMZ fetches from an OT-side MQTT broker or historian gateway, keeps the OT network from opening connections outward. A push model, where an OT-side MiNiFi agent sends to a NiFi listener in the DMZ, lets the plant buffer locally through link outages but requires a firewall rule that allows the plant to reach the DMZ. Choose one per data source and document the reason. Whichever direction is chosen, enable TLS on every listener and client connection, and confirm which of the supported authentication providers (LDAP, OpenID Connect, SAML 2.0 or X.509 client certificates) your identity team will operate before the first deployment. The NiFi administration guide also binds the UI to 127.0.0.1 when no security configuration is supplied, so keep the UI on loopback until TLS and authentication are in place, and reach it through a controlled tunnel during setup.

How a plant NiFi rollout typically runs

The sequence starts with knowing what data exists on each line and where the network boundaries sit, then proves the design on one line before any fleet-wide change. Each phase produces evidence that the next phase depends on.

  1. Inventory data sources and protocols per line

    The plant engineering team and the DeployOpen team list every data source on the pilot line and its neighbors: PLCs and machine controllers, MQTT brokers, edge devices, existing historians, and the protocols and data formats each one uses. The inventory records message rates, sample intervals, who owns each source, and which vendor or integrator must be involved for changes. It is done when every source has an owner, a protocol, and a named consumer, and the team agrees on which data is needed at the IT side and which stays on the machine side.

  2. Design zones and conduits

    Using the inventory, the team groups assets into zones by function and risk, then defines the conduits that carry data between them. The design follows the ISA/IEC 62443 model of zones and conduits and the risk assessment expected under ANSI/ISA-62443-3-2-2020. It decides where MiNiFi agents sit, which direction each conduit allows traffic, and which side initiates connections. The output is a reviewed zone and conduit diagram with a firewall rule list, approved by plant operations and IT security before any device is connected.

  3. Build and pilot one MiNiFi agent and one flow

    The team deploys a single MiNiFi agent on one line and a single NiFi flow that collects one data stream and delivers it to a test destination. Before installation, confirm the host meets the runtime requirement: the NiFi administration guide lists Java 21 as the minimum runtime for NiFi, so confirm the equivalent requirements for the MiNiFi build you choose with the vendor documentation. Parameters hold site-specific values such as broker addresses. The pilot is complete when data arrives at the destination with the expected fields, timestamps, and volume, and the flow is reviewed by both plant and IT staff.

  4. Test buffering and outages on a weak link

    The team deliberately degrades the pilot link with dropped connections, latency, and a full stop of the upstream network for a defined window. They confirm that the agent queues data on disk, that back pressure behaves as designed, and that nothing is lost or duplicated when the link returns. They also check that repository disk limits hold and that the operator sees alerts. Done means the team has written results for each failure scenario, with recorded gaps and fixes, and the plant accepts the buffer size and recovery behavior.

  5. Hand off to the historian and analytics

    Working with the historian and analytics owners, the team defines the data contract for each destination, including field names, units, time zones, and quality flags. The flow is extended to write to the historian or analytics platform, and error routes capture records that fail validation for review. Reconciliation checks compare counts and sample values between source and destination. The handoff is complete when the downstream owner signs off on the contract and a reconciliation report shows the agreed fields arriving intact.

  6. Version flows in Git and roll out the fleet

    Before expanding, the team stores the approved flows in a Git repository and deploys them through a controlled promotion process from test to production. NiFi 2 provides Flow Registry Clients for GitHub and GitLab that version flows in Git, which gives change history and review for each line. Each remaining agent is installed from a standard image and receives the versioned flow with line-specific parameters. The rollout proceeds line by line, and each line is closed only after its reconciliation and outage checks pass and the change is recorded in the repository.

Compliance and evidence manufacturers should plan for

Manufacturers collecting plant and machine data through NiFi and MiNiFi often fall under several regimes at once: OT security standards, EU incident reporting and product rules. The cards below cover the obligations that most directly shape flow design, network zoning and evidence retention. DeployOpen prepares the controls, configuration and records; independent auditors or certification bodies issue any assessment.

ISA/IEC 62443 zone and conduit evidence

The ISA/IEC 62443 series organizes industrial security into general, policy, system and component groups. For an NiFi collection point that bridges plant networks and IT, the system requirements of ISA-62443-3-3-2013 and the risk assessment in ANSI/ISA-62443-3-2-2020 help define the zones and conduits you must document. Asset owners also carry program duties under ANSI/ISA-62443-2-1-2024. Ask which zone each NiFi node sits in and where the conduit crosses, and keep those diagrams current with the flow.

NIS2 reporting timelines and fines

NIS2 Annex II lists manufacturing among critical sectors, covering NACE Rev. 2 divisions 26 to 30. In-scope entities must send an early warning within 24 hours, an incident notification within 72 hours and a final report within one month. Important entities face fines of at least EUR 7 million or 1.4% of worldwide turnover. NiFi provenance and log routing can help reconstruct incident timelines, so confirm that the logs you need are retained and reachable within those windows.

CRA Article 14 and Article 64

The Cyber Resilience Act applies to products with digital elements from 11 December 2027. Manufacturers' Article 14 duties to report actively exploited vulnerabilities and severe incidents already apply from 11 September 2026. Article 64 sets maximum fines of EUR 15 million or 2.5% of worldwide annual turnover, whichever is higher. If your company ships connected machines or gateways, confirm who on your side owns vulnerability intake and reporting, and whether NiFi-based data paths fall within your product scope.

NIST SP 800-82 Rev. 3 OT guidance

NIST SP 800-82 Rev. 3, issued in September 2023, supersedes Rev. 2 and covers SCADA, DCS, PLCs and building systems. It treats OT performance, reliability and safety as security requirements in their own right. For a NiFi deployment, this means tuning buffers, backpressure and failover so that data collection does not disturb control loops. Use the guide to set monitoring baselines for the NiFi hosts and to justify the isolation design to your OT engineers.

What manufacturers ask before hiring help with NiFi

What drives the cost and effort of a plant NiFi project?

The main drivers are how many data sources and plant areas are in scope, how many protocols and historian or analytics targets each flow must reach, and how much edge work is needed. Cost has two parts: one-time design, build, testing and handover, and recurring operation, patching and upgrades. Scope is harder to estimate when device lists are incomplete, when network zones are not documented, or when OT change control adds approval steps. Ask any partner for a written scope listing sources, targets, sites, edge agents, security zones and handover deliverables, then price against that list. Fixed-scope quotes follow from that document.

How long does a NiFi pilot take on a production line?

A pilot's length depends on the number of data sources, how quickly the plant can grant access to test networks, and how many approvals an OT change needs before a flow touches live equipment. Timing usually slips for reasons outside the flow itself: waiting for a maintenance window, confirming tag names with the historian team, or agreeing on where buffered data is stored during an outage. Ask for a timeline with named dependencies and a defined exit condition, such as a set of tags flowing through a test historian for an agreed period. No duration is given here; confirm it against your own plant schedule and change calendar.

What must the plant team provide before a NiFi engagement starts?

Plan to provide three things early: current network diagrams showing OT zones, conduits and the boundary to IT, a device inventory covering PLCs, gateways, MQTT brokers and sensors with their protocols and addresses, and named contacts for the historian, analytics and OT security teams. Also provide access rules for test environments, a change process for OT systems, and the data retention rules that apply to buffered telemetry. Gaps in these inputs are the most common reason fixed-scope work is re-quoted. Ask the implementation partner to list every input and its owner in the scope document.

Can our in-house team run NiFi after handover?

Yes, if the team is given the right materials and skills before the project closes. Run-time work includes monitoring cluster health, managing certificates and authentication, applying patches, handling disconnected nodes and full disks, and promoting flow changes through a controlled process. Ask for documentation of each flow's purpose, parameters, error paths and provenance settings, plus a recorded walkthrough and a supervised first change. Confirm the Java runtime and Python version requirements with your platform owners, since the NiFi administration guide sets Java 21 as the minimum runtime. Decide early whether support will be in-house, retained from a partner, or split.

How should a NiFi 1.x site plan the move to 2.x?

Treat the move as a planned migration, not an in-place upgrade. The NiFi migration guidance requires upgrading to NiFi 1.27.0 before moving to 2.0.0, and it requires changing the flow configuration file from flow.xml.gz to flow.json.gz. NiFi 2.0 also removed the variable registry expression language scope in favor of an ENVIRONMENT scope and renamed distributed map cache components, so flows and custom code that depend on them need changes. The NiFi project has also stated that 1.x cannot be maintained further, so the timing of the move is a risk decision. Inventory every flow and custom component, test them on a copy of the flow, and schedule the cutover around plant maintenance windows.

Is Apache NiFi free to license for a manufacturing plant?

The Apache NiFi project is released under the Apache License 2.0 (SPDX: Apache-2.0), and the project itself offers no paid edition. Licensing is therefore not the main cost. Plants still pay for infrastructure, engineering time, testing, security work, support retainers and the time their own staff spend operating the platform. Check any third-party connectors, historians or edge products separately, because their license terms are set by their own vendors. Confirm your organization's open-source use policy before adoption.

How do we keep OT and IT networks separate with NiFi in the middle?

Place NiFi so that the OT side connects outward only through defined, one-way or tightly controlled paths, and make the boundary a documented design decision reviewed by OT security. Common patterns include an edge agent such as MiNiFi inside the OT zone that buffers data and pushes it through an approved conduit, and a NiFi cluster in the IT or DMZ zone that consumes it. Avoid designs where IT systems open inbound sessions into control networks. The ISA/IEC 62443 series is a useful reference for zone and conduit thinking, and NIST SP 800-82 Rev. 3 covers OT security practice. Confirm the design against your own risk assessment before building.

What to ask a partner before signing a plant data scope

Start with the zone and conduit model. Ask the partner to show, in writing, which segments of the plant network hold the MQTT brokers, PLC-facing gateways, MiNiFi agents and NiFi nodes, and which conduits carry data between them toward the historian and the IT analytics platform. The useful answer names each crossing, the direction of traffic, the protocol and port, and who approves the rule. Ask how the design maps to the ISA/IEC 62443 zone and conduit concepts and to the risk assessment described in ANSI/ISA-62443-3-2-2020, and confirm that the partner will produce that assessment as a deliverable rather than assume it. Also ask where the NiFi cluster sits, because a cluster placed in the IT zone with write access back into OT is a different risk from one that only pulls data outward.

Next, ask how buffering and reconnection will be tested, and what evidence comes out of the test. A credible plan describes deliberate link cuts between the edge agents and the NiFi cluster, the broker restarts, the volume of data queued during the outage, and the point at which queued data is delivered in order with no gaps or duplicates. Ask which NiFi repositories and queue settings will be sized for that buffer, and how disk pressure on the content and provenance repositories will be monitored and alarmed before a full disk stops ingestion. Confirm whether the test runs on a staging copy of the plant topology or only on a lab bench. Any number the partner quotes for buffer duration should be tied to your measured data rates and your storage, not to a generic figure.

Finally, ask how flow changes are versioned, approved and scheduled. Find out whether flows are stored in a Git repository through a Flow Registry Client, who can commit and who can promote a change to the plant, and whether NiFi Registry is part of the design given its deprecation and planned removal in NiFi 3.0. Ask how OT change windows are agreed with operations and maintenance, how emergency rollback works if a flow misbehaves during a shift, and who signs off the change record. Request the handover list in writing: architecture and zone diagrams, the flow catalog with owners, runbooks for buffer and cluster recovery, the security configuration, backup and restore steps, and the test reports. Confirm which items are included in the fixed scope and which would be billed as later work.

How DeployOpen works on this

DeployOpen plans, implements, hardens, documents and supports Apache NiFi and MiNiFi inside your own plants, data centers and cloud accounts. We agree the scope in writing before work starts, so OT owners, IT architects and security teams know what gets built, tested and handed over for plant and machine data flows.

How we scope

  • Inventory the MQTT brokers, PLC gateways, historians and edge devices that feed each planned flow.
  • Map which data crosses the OT and IT boundary, in which direction, and under which approvals.
  • Agree deliverables, timelines and acceptance tests in a written, fixed-scope statement before build starts.

How we build

  • Design flows for store-and-forward buffering so data survives unreliable plant links and broker outages.
  • Configure MiNiFi agents and NiFi clusters with TLS, authentication and provenance enabled from the first build.
  • Test replay, ordering, duplicate handling and failover against representative machine data before production use.

How we hand over

  • Document flow design, parameters, network paths, certificates and recovery steps for your operations team.
  • Store flow definitions in a Git-backed registry with change records your reviewers can audit.
  • Walk your engineers through runbooks for patching, backups and disconnected-node recovery before handover.

How we support

  • Offer ongoing support, upgrades and continuous testing as a retainer after handover.
  • Plan NiFi 1.x to 2.x migration work as a separate, scoped engagement with its own deliverables.
  • Prepare evidence for your own auditors and certification bodies, which issue the assessments themselves.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.