How Apache NiFi JVM heap tuning keeps flows stable under load
When a NiFi node slows down, stalls or throws memory errors under production load, the JVM heap and garbage collection settings are usually the first place to look. The heap holds FlowFile attributes, in-flight processor objects and queue metadata, and the garbage collector decides how often and how long the JVM pauses to reclaim that memory. A team needs to tune these settings when a node's heap runs close to its limit during normal traffic, when long collection pauses show up in the logs, or when a flow that worked in testing starts failing once real volumes arrive. NiFi runs as a JVM process, and the administration guide lists Java 21 as the minimum runtime, so confirm your installed Java version before changing any memory flags.
The heap is set in the bootstrap.conf file in the NiFi conf directory, where the java.arg entries control the initial and maximum heap size and the garbage collector arguments. Changes there apply on restart, and they should be made one node at a time in a cluster so the rest of the cluster keeps processing. Heap size is only one input. Large FlowAttribute values, big attribute maps on many FlowFiles and queues that grow past their thresholds all put pressure on the same memory, so heap changes that ignore flow design tend to move the failure rather than remove it. Treat the heap work as one part of a wider performance plan that also covers repositories, operating system limits and queue behavior, and record the baseline heap usage and GC activity before you change anything.
Before you change settings, collect the evidence: the heap usage and GC logs for the affected node, the queue sizes at the time of the slowdown, and the sizes of the largest attributes that flow through the slow processors. Ask your team or partner to confirm the GC logging options available for your Java build, and confirm how the administration guide describes the heap dump arguments for out-of-memory errors so you can capture a dump when a failure happens. Plan a test window, define what a stable node looks like in metrics, and change one setting at a time so you can tell which change helped.
Heap, garbage collection and CodeCache options in bootstrap.conf and when each fits
This table compares the bootstrap.conf JVM arguments the NiFi administration guide covers: heap sizing, the garbage collector, CodeCache and heap dump on OutOfMemoryError. Read each row as a decision: what the setting controls, when it fits and what to confirm against the guide for your installed version. The main takeaway is that heap size is only one of several JVM settings, and the others change how NiFi behaves under sustained load.
| Setting | What it controls | When it fits | What to confirm |
|---|---|---|---|
| Initial and maximum heap (-Xms, -Xmx) | Memory the JVM reserves for FlowFile attributes, queues and processor objects | Set both to the same value on a dedicated host so the heap does not resize under load | Current argument lines and the default values shipped in your installed bootstrap.conf |
| Garbage collector selection | Which collector runs and how it pauses application threads | Use the collector the guide documents for your Java 21 runtime, and change only after measuring GC pauses | The collector named in the guide for your version and any GC logging arguments it recommends |
| GC logging arguments | Records collection events so pauses can be correlated with throughput drops | Enable during tuning or incident review so heap pressure can be traced to specific periods | Log file location, rotation and disk space on the repository volume |
| CodeCache sizing | Memory for JIT-compiled code, which the guide links to sudden slowdowns under sustained, extremely high throughput | Consider tuning when throughput is high for long periods and performance drops abruptly, not during normal bursts | The CodeCache arguments and thresholds the guide describes for your version, and whether they are needed at all |
| Heap dump on OutOfMemoryError | Writes a heap dump file when the JVM cannot allocate memory | Enable on every production node so the evidence exists when an out-of-memory event happens | Dump directory, free disk space sized for a full heap dump and the handling of sensitive content in the dump |
| Large attributes and queues | Consume heap independently of the arguments above, regardless of how they are set | Keep attribute values small and rely on swapping and back pressure to limit in-heap queue growth | Flow design review of attribute sizes and connection thresholds before raising heap limits |
| Restart after changes | Whether new arguments take effect | Apply changes in a maintenance window, one node at a time on a cluster | Node disconnect and reconnect behavior in your cluster during the restart |
How to set heap size and GC in bootstrap.conf step by step
This sequence changes NiFi's JVM heap and garbage collection settings in a controlled way, so each change is measured against a known baseline before it reaches the whole cluster.
Record the current heap and GC baseline
Before editing anything, the administrator captures how the node behaves today: heap usage over time, garbage collection frequency and pause length, and whether the node has logged out-of-memory errors. Use the node diagnostics available in the NiFi UI and the JVM garbage collection log, if enabled, on the node you plan to change. Note the current
java.argvalues inbootstrap.confso you can roll back. The step is complete when you have a written baseline for one representative node and a copy of the original file.Edit the heap arguments in bootstrap.conf on one node
On a single node, the administrator changes the initial and maximum heap sizes (
-Xmsand-Xmx) and any garbage collection arguments inbootstrap.conf, keeping changes small enough that a regression is easy to attribute. Avoid changing several unrelated settings at once. If you want a heap dump on an out-of-memory error, add the heap dump arguments documented in the administration guide at the same time, so you can reproduce a failure later. The step is complete when the diff against the saved copy shows only the intended lines.Restart the node and confirm the JVM flags
Restart NiFi on the changed node only, and let it rejoin the cluster before you read anything. Confirm the flags took effect by checking the startup log for the JVM arguments it prints, or by reading the runtime values in the node diagnostics. If the flags are missing or the node fails to start, restore the saved
bootstrap.confand restart again. The step is complete when the logged or reported heap limits match what you wrote and the node shows as connected in the cluster view.Observe the node under representative load
Let the node run the flows it normally carries, including the busiest period you know about, and watch the same measures you recorded at baseline: heap use after collections, GC frequency and pause time, and any out-of-memory or repeated full-collection events in the logs. Compare the results with the baseline rather than with an expectation. If the test traffic is not representative, say so in the change record, because a short quiet window can hide the problem. The step is complete when you have a comparison that shows whether the new settings helped, hurt or made no measurable difference.
Check code cache under sustained throughput
If the node sustains very high throughput, check whether the JVM code cache is filling and whether performance drops suddenly during peak periods. The administration guide notes that CodeCache settings may need tuning under this kind of load, so review it against your observed behavior rather than adjusting it preemptively. Record any code cache change as a separate step on the same test node so its effect is isolated. The step is complete when you can state either that code cache was not a limit under your load or what setting you changed and what you observed afterward.
Roll the change across the cluster
Apply the validated
bootstrap.confchange to the remaining nodes one at a time, restarting each and waiting for it to reconnect before moving on. Keep the saved original file available for each node until the cluster has run through a full peak period. Confirm each node's startup flags with the same check used on the first node. The step is complete when every node reports the new heap settings, all nodes are connected, and the monitoring baseline has been re-measured across the cluster.
Which tuning path fits a standalone node, a cluster or sustained high throughput
A single development, test or low-volume NiFi node usually runs well on the heap size you set during installation. The goal on this path is to confirm that the default settings in bootstrap.conf match the machine and the data volume, and to avoid oversizing the heap in ways that leave too little memory for the operating system and the repositories. Tuning here is mostly a sizing check, not a performance project.
- Match the heap to the host: Confirm that the
-Xmsand-Xmxarguments inbootstrap.confleave memory for the operating system, the repositories' file caches and any other services on the same machine. - Keep minimum and maximum equal: Setting both to the same value avoids resizing pauses during early operation, which is a common choice for steady workloads.
- Check the runtime version: Confirm the JVM is Java 21, the minimum runtime the administration guide lists.
- Watch for pressure, not guesses: Only change heap settings after you see repeated out-of-memory errors, long garbage collection pauses or a steadily rising heap in logs.
Before you change anything, confirm the current heap settings, the Java version and the data rate you expect in production, and ask whether this node will ever carry production traffic.
A production cluster runs many flows with different shapes: small record processing, large file movement and occasional bursts. Heap decisions on this path must hold across every node and across the flows that compete for memory at the same time. The main question is not the peak of one flow but how queues, attributes and processor concurrency behave together under normal and degraded conditions, including when a node is disconnected and its work moves to others.
- Size every node the same way: Nodes with different heap settings produce different failure points, so review the
bootstrap.confvalues across the cluster together. - Account for queued data: Large queues and big attributes keep objects on the heap, so tune after you review back pressure and swapping behavior for the busiest connections.
- Plan for failover load: Confirm that the remaining nodes can carry the load of a lost node without sustained memory growth.
- Collect garbage collection evidence: Enable GC logging on each node and compare pause times across nodes before changing collector options.
- Change one setting at a time: Test each heap or collector change in a staging cluster that mirrors production flows.
Before a change, ask for a per-node record of heap usage and GC pauses over a representative period, and confirm the flows that drive the largest attribute and queue sizes.
A node under sustained, extremely high throughput is the situation where the administration guide says CodeCache settings may need tuning to avoid sudden performance loss. The signal here is not a single error but a pattern: throughput that holds steady for long periods and then drops without an obvious cause. Heap changes alone may not resolve it, so this path starts with evidence from the JVM and the flow before any argument changes.
- Look for a sudden drop under steady load: Sustained load that degrades abruptly, rather than gradually, is the pattern the guide associates with CodeCache pressure.
- Check CodeCache usage first: Review JVM code cache metrics during peak periods before changing
-XX:ReservedCodeCacheSizeor related arguments. - Confirm heap is not the cause: Rule out heap exhaustion with garbage collection logs and heap history so you do not treat a memory problem as a code cache problem.
- Capture a heap dump on failure: Add the
-XX:+HeapDumpOnOutOfMemoryErrorand dump path arguments inbootstrap.confso you have data when an out-of-memory error occurs. - Retest at full load: Validate each change under the same sustained throughput that exposed the problem.
Before tuning CodeCache, ask for JVM metrics from the period of degradation and confirm whether the same node also shows heap or garbage collection pressure, since the two paths can overlap.
Security and operational risks of heap dumps and JVM changes
Heap dumps and JVM argument changes both alter what NiFi holds in memory and who can see it. The cards below cover the data exposure, storage and change control, and post-change monitoring that should be decided before any heap or garbage collection setting reaches production.
Heap dumps hold live flow data
A heap dump is a snapshot of everything in the JVM at the moment of failure, so it can contain FlowFile attribute values, content held in memory and parameter values resolved during processing. Treat a dump as sensitive data in the same class as the flows it came from. Before enabling dumps through bootstrap.conf, confirm which sensitive parameters and attributes your flows carry and whether they can appear in memory.
Sensitive properties and credentials in memory
NiFi decrypts sensitive property values, such as passwords and keys held by controller services, so they exist in plaintext in the heap while components run. A dump taken during an out-of-memory event can expose them even though they are encrypted at rest. Check which controller services hold credentials, limit who can read dump files, and plan rotation for any secret that may have been captured.
Dump storage needs access controls
Heap dump files are large and are written to the path configured for the JVM, often on a shared or general-purpose volume. Restrict the directory to the NiFi service account, keep it off repository disks so a dump cannot fill the content or provenance volume, and place it on storage that your encryption and backup standards cover. Ask your security team which access logging applies to that path.
Retention limits for dump files
Dumps accumulate after repeated failures and are easy to forget. Define a retention period and an owner who deletes or archives dumps after analysis, and record when each dump was captured and which node produced it. Your data classification policy should decide whether dumps may leave the environment for vendor analysis, and whether that transfer needs approval.
JVM argument changes need change control
Heap size and garbage collection flags in bootstrap.conf change behavior on every node and usually need a rolling restart. Route each change through your normal change process with the previous values recorded, a documented restart order for the cluster, and a tested way to restore the earlier file. A change that looked safe in a test flow can behave differently under production throughput.
Monitor GC and memory after every change
A new setting can look correct at startup and still degrade under sustained load, so watch garbage collection frequency and pause time, heap occupancy after collections, and code cache usage for a full business cycle after each change. Compare results with the values recorded before the change. Set alerts that fire on rising post-collection heap use so that a slow leak is noticed before it becomes an out-of-memory event.
What teams ask before tuning NiFi heap and garbage collection
What drives the cost of NiFi heap and garbage collection tuning?
The main driver is how much analysis the heap needs, not the change itself. Setting the heap size in bootstrap.conf takes little time, but finding the right value depends on flow design, the number of FlowFiles in queues, how large your attributes are, and how many nodes you run. Costs come from two sources: your own engineers' time to collect data and test changes, and any outside help you buy under a fixed-scope written agreement. Ask any partner for the scope in writing, including which flows are reviewed, which heap dumps are analyzed and what test runs are included. DeployOpen does not publish prices, so a quote follows from that scope.
How long does heap and garbage collection tuning usually take?
Duration depends on how many environments, nodes and flows you have and how many tuning cycles the workload needs. A single node with a stable flow can often be adjusted in a few test runs, while a multi-node cluster with large queues needs longer observation under real load. Plan time for collecting garbage collection logs, changing one setting at a time, and repeating the test under peak conditions. Any specific duration should come from the written scope and from your own test environment, not from a general rule.
What does our team need to provide for this work?
You need access to the NiFi hosts, the bootstrap.conf file, the NiFi application logs and garbage collection logs, and a test environment that can reproduce your load. You also need someone who knows the flows well enough to explain which processors handle large attributes or large batches. Give the team the current NiFi version, the Java runtime in use and a record of recent out-of-memory events. Without representative load, heap settings tuned on a quiet test system may not hold in production.
Can our own engineers tune the NiFi heap without an outside partner?
Yes, many teams can change heap settings and read garbage collection logs with their own staff. The work is manageable when the flow design is stable, the team can reproduce load in a test system and someone can read heap dumps. Outside help is more useful when problems come from flow design, such as oversized attributes or queues that grow without limit, because those need changes to the flows as well as the JVM. Decide early who owns production changes and who signs off on them.
How do we recognize a heap problem in NiFi?
Common signs include repeated OutOfMemoryError entries in the application log, long garbage collection pauses, throughput that drops over time without a matching change in data volume, and nodes that become unresponsive under load. Check whether heap use rises between collections and does not fall back. Also check whether queues are large and whether attributes on FlowFiles are unusually big, since both can consume heap. Configure NiFi to write a heap dump on out-of-memory errors, as the Administrator's Guide describes, so you have evidence for analysis.
Could a CodeCache problem look like a heap problem?
Yes, the symptoms can overlap. The Administrator's Guide notes that under sustained, extremely high throughput the JVM CodeCache settings may need tuning to avoid sudden performance loss. A sudden drop in throughput under steady load, without heap exhaustion, can point to CodeCache rather than the heap. Compare the garbage collection logs and JVM metrics against the throughput history before changing any setting, so you change the right value.
When should we bring in outside help for NiFi heap tuning?
Bring in outside help when out-of-memory errors recur after you have changed the heap, when the cause is unclear from logs, or when the fix depends on flow redesign. It also helps when a production cluster is at risk and your team lacks time for controlled testing. DeployOpen plans, implements, tunes, documents and supports self-hosted open-source platforms inside your own infrastructure. Ask for a written scope that names the environments, the flows reviewed, the heap dump analysis, the test runs and the handover material before work starts.
What tuning the heap changes for your flows and your team
A larger heap is not a free fix for a slow or unstable NiFi node. Every gigabyte assigned to the JVM in bootstrap.conf is memory the operating system cannot use for the page cache, and NiFi's repositories depend heavily on that cache. The FlowFile, content and provenance repositories read and write large volumes of data, and when the host starts swapping or evicting cached pages, throughput can fall even though the heap looks healthy. The practical decision is how much memory the node has after the OS, the repositories and any co-located agents are accounted for, and whether the heap you want still leaves room for those. Ask your platform team to confirm the memory budget per node before anyone raises -Xmx, and record the reasoning next to the change.
Heap settings and flow design have to be reviewed together, because most heap pressure starts in the flow. Large attribute values, FlowFiles that carry big payloads in attributes rather than content, and connections that queue far past their back pressure thresholds all keep objects alive on the heap. A heap increase can hide these patterns for a while and then fail later at a higher volume. Assign one owner for flow design and one for JVM settings, and require that anyone who adds a processor pattern that buffers data, or raises a queue threshold, states the expected heap impact. Garbage collection behavior should be checked against the same flows after each change, not in isolation.
The upgrade path changes the baseline your team tunes against. NiFi 2 requires Java 21 as the minimum runtime, according to the NiFi administrator's guide, so any heap or garbage collection options carried over from a 1.x bootstrap.conf need to be checked against the new runtime before go-live. Flags that were valid on an older Java release can be removed or behave differently, and the upgrade is a good moment to re-measure rather than copy old values forward. Plan the Java upgrade, the NiFi upgrade and the JVM retune as one change with a rollback point, and confirm the exact supported JVM options in the NiFi System Administrator's Guide for the release you deploy.
How DeployOpen scopes and delivers NiFi JVM tuning
DeployOpen approaches Apache NiFi JVM heap and garbage collection tuning as a written, fixed-scope change to your running NiFi, starting from how your flows actually use memory. We plan, build, test, document and hand over the settings in the infrastructure you control, and we can support them afterward under a retainer.
How we scope
- We review node sizes, current bootstrap.conf arguments, flow designs and queue and attribute patterns before proposing any heap value.
- The written scope names the NiFi nodes, JVM versions, garbage collector choice and the load profiles that the tuning must cover.
- We agree on the test evidence, such as GC logs and heap dump handling, that will count as acceptance before work starts.
How we build
- We change heap, garbage collector and CodeCache arguments in bootstrap.conf in a staged order, one node or environment at a time.
- We test under representative load, including sustained high throughput, and record garbage collection behavior against the agreed baseline.
- We configure heap dump capture on OutOfMemoryError and confirm that dumps land on storage with enough space and access controls.
How we hand over
- We deliver a runbook that explains each setting, the reason for its value and the signals that should trigger a review.
- We walk your operations team through reading GC logs, interpreting heap dumps and rolling a configuration change across the cluster.
- We provide a change record with before and after measurements so your team can audit the decision later.
How we support
- Ongoing support is available as a retainer, covering re-tuning after flow changes, NiFi version upgrades or new node hardware.
- We can review the effects of new large attributes or queue growth on heap use and recommend flow changes alongside JVM changes.
- Continuous testing is available as a retainer, so heap and garbage collection settings are checked again as your workload grows.
