Choosing JVM and Java settings for Apache Kafka before go-live
Since Kafka 4.0, brokers, Kafka Connect and the command-line tools require Java 17, while clients and Kafka Streams applications need at least Java 11. Teams standing up or upgrading self-hosted Kafka need to choose a Java runtime that meets the broker floor, then set a heap that stays modest so the operating system page cache does most of the work. This page covers choosing the Java version, sizing and setting the heap, and planning Java patch updates as part of security maintenance. It does not cover operating system tuning such as file descriptor limits, or hardware sizing for disks and memory. Confirm the Java versions supported by your target release in the Kafka 4.3 upgrade notes before you pick a runtime.
Start by inventorying the Java runtime on every broker, controller and Connect host, because mixed versions across a cluster are a common source of surprises during upgrades. For each host, record the Java distribution, the exact patch level and how it was installed, since package-managed and vendor builds may update on different schedules. Then choose one approved distribution and version for the whole platform. Set the heap to a fixed size and keep it well below the host's total memory, leaving the rest for the page cache that serves reads and buffers log segments. Use the G1 garbage collector settings documented for your Kafka release as the starting point, and test any change in a non-production cluster. Ask your Java vendor and your internal platform team to confirm which build and support terms they will use, rather than assuming the default package applies.
Java patch releases carry security fixes, so the runtime needs the same change control as Kafka itself. Plan a recurring review of Java security updates, decide who approves them, and make sure the broker and Connect restart process can be run as a rolling change without losing availability. Record the Java version in your configuration management and in the evidence you keep for internal reviews, so you can show which runtime each broker ran on a given date. Before go-live, rehearse one full patch cycle on a non-production cluster, including a restart of every node type, and confirm that your monitoring alerts on heap pressure and garbage collection pauses. These checks tell you whether the heap setting still fits the workload after the first months of production traffic.
JVM options and when each fits
This table compares Java runtime choices for Kafka brokers, Kafka Connect, clients and Kafka Streams applications. Read each row against your own release line and workload, then confirm the exact Java requirement in the upgrade notes for the Kafka version you plan to run.
| Option | When it fits | Trade-off | What to confirm in the release documentation |
|---|---|---|---|
| Java 17 for brokers and Connect | Brokers and Connect workers on Kafka 4.x where you need the minimum Java level the project requires | Java 17 is the stated minimum for brokers, Connect and tools since Kafka 4.0, so it leaves no headroom above the floor | That Java 17 is listed as supported for the exact Kafka release you install, and the patch level your Java vendor ships |
| Java 21 for brokers and Connect | Teams that standardize on a newer long-term Java release and want a supported runtime above the minimum | Your build, agent and monitoring tooling must also run on 21, which can add testing work during upgrades | Whether the release notes for your Kafka version list Java 21 as tested or supported, and any known issues with your JVM vendor |
| Java 25 for brokers and Connect | Organizations that have already moved their platform Java baseline to 25 and need one runtime across services | Newer runtimes may have fewer vendor builds and fewer confirmed combinations with agents and plugins | Whether Java 25 is documented for your Kafka release line, rather than assuming support from the Java release date |
| Clients and Kafka Streams on Java 11 or later | Client applications and Streams jobs that cannot move to the broker runtime on the same schedule | Clients can lag behind the broker runtime, so you maintain two Java baselines and two patch cycles | The minimum Java version for the client library and Streams version you pin in your build files |
| Heap sized by workload | Brokers with large partition counts, heavy client connection churn or large metadata footprints | Heap beyond what the workload needs reduces page cache, which Kafka relies on for reads and writes | The heap guidance in the Kafka operations documentation for your release and your measured broker memory use |
| G1GC as the documented collector | Standard broker deployments where the project's documented garbage collector settings apply | Tuning beyond the documented flags requires load testing and GC log review on your own hardware | The current JVM performance settings in the Kafka documentation for your release, including any changes between minor versions |
| Java patch level and vendor builds | All environments, since Java security fixes arrive in patch releases on your schedule | Patch updates need a restart plan and a rolling broker procedure to avoid downtime | Your Java vendor's support and security update policy, and whether your Kafka release is tested against that patch level |
Why a larger heap does not make Kafka faster
Giving Kafka a larger heap rarely improves throughput, and it often makes brokers less stable. Kafka keeps its log data on disk and relies on the operating system page cache to serve recent reads and absorb writes. Memory the cache cannot use is memory the broker does not benefit from, so an oversized heap takes RAM away from the one layer that makes consumer reads fast. The heap should hold broker objects, request state and metadata, not the message data itself.
A large heap also lengthens garbage collection pauses. When the collector must scan and reclaim a big heap, request threads can stall, and those stalls appear as produce and fetch latency spikes, replication falling behind, or brokers dropping out of the cluster after a session timeout. Teams often see these symptoms first on dashboards and blame the network or the disks. Check GC pause logs and request latency together before changing any setting, and confirm which heap size your broker version and workload need with your own load tests rather than copying a figure from another cluster.
Set heap from measured broker behavior: heap usage after full garbage collection, pause times under peak load, and request queue latency. Host memory alone is the wrong input, because a host with a lot of RAM still needs most of it for the page cache. Plan the page cache as a separate budget, using the reasoning in the Apache Kafka hardware sizing page, and treat the heap as the smaller, measured allocation that remains. Revisit the setting after traffic grows, partition counts change or the Java version is updated.
How to set heap and G1GC for a Kafka broker
This sequence confirms the Java version the Kafka release requires, sets heap and G1GC options through the startup variables, and checks GC behavior on one node before the change reaches the whole cluster. The result is a tested, documented JVM configuration that each broker runs with.
Confirm the Kafka release and Java requirement
The platform owner confirms which Kafka release the cluster will run, using the supported releases listed on the downloads page, and checks the Java requirement for that release in the upgrade notes. Since Kafka 4.0, brokers, Kafka Connect and tools require Java 17. Done when the team has a written pair of Kafka release and Java major version, approved by the people who own the cluster.
Install an approved Java build on every host
The infrastructure team installs a Java build from the vendor or Linux distribution the organization approves, at the major version confirmed in the previous step. Install the same build on every broker and controller host so that GC behavior is comparable across nodes. Run the Java version command on each host and record the full build identifier. Confirm the approved source publishes patch-level updates, because security fixes in the JVM arrive that way and an unpatched runtime leaves the broker exposed.
Set heap and G1GC through startup variables
The engineer sets the heap size and the garbage collector options in the environment that the Kafka start scripts read, typically the variable for heap options and the variable for JVM performance options. Set the minimum and maximum heap to the same value so the heap does not resize under load. Select G1GC and add GC logging flags. Leave the rest of the host memory free for the operating system and page cache. Confirm the variable names against the documentation for your release. Done when the running process shows the intended flags, which the team checks with the JVM flag inspection tool on the started process.
Start one broker and capture GC logs
The engineer restarts a single broker with the new settings while the controller quorum stays healthy and the other brokers keep serving traffic. The broker runs under representative producer and consumer load for a window the team defines in advance, with GC logging writing to a known location. Confirm the broker rejoins the cluster and that no partitions remain under-replicated after it returns. Done when the GC log covers the full load window and the broker is healthy.
Review pauses and heap occupancy on the test node
The engineer reads the GC log and compares pause times with the latency target the service owners set for producers and consumers. Check heap occupancy after collections, since a baseline that climbs over time points to a leak or an undersized heap. Check the free memory left on the host, because too little page cache weakens the broker's read path and too much heap leaves the OS short. Done when the team signs off on pause behavior and heap headroom, or adjusts the settings and repeats this step.
Roll the change across the cluster one node at a time
The operator applies the same settings to the remaining brokers and controllers, restarting one node at a time. Before each restart, confirm that the cluster has no under-replicated partitions, and wait for the restarted node to catch up before moving to the next. Compare GC logs across nodes to catch host-specific differences. Keep the previous settings available so the change can be reversed. Done when every node runs the new configuration, the cluster reports healthy broker and quorum status, and the change is recorded in the configuration management system.
Why patch-level Java updates matter for Kafka security
The JVM is the runtime under every Kafka broker, so a vulnerability in it sits underneath TLS termination and SASL authentication. The cards below cover how to learn about vendor security releases, how to roll them through a cluster without losing availability, and how to keep the runtime version visible in change records.
Track vendor security releases
Subscribe your platform team to the security and update notices of the Java distribution you chose, and record which build each broker, controller and Connect worker runs. Patch-level releases of the JVM can fix flaws in TLS, cryptography and networking code that Kafka relies on, so a missed notice leaves the trust boundary exposed. Ask the vendor how far back it backports fixes and how it publishes them. Confirm the release cadence for your chosen distribution before you write it into your patch policy.
Schedule rolling JVM restarts
A JVM patch takes effect only after the process restarts, so each broker and controller needs a rolling restart. Restart one node at a time, wait for under-replicated partitions to return to zero and for controllers to report a healthy quorum, then move to the next node. Plan the order so the active controller is restarted last, and rehearse the procedure on a non-production cluster first. Ask your operator or vendor whether the rolling process is automated or manual.
Keep clients on supported runtimes
Patching the broker does not cover applications that embed Kafka clients or run Kafka Streams. Check each client's Java runtime against the minimum listed in the upgrade notes for your Kafka release, and track applications still on older runtimes as a separate risk. Brokers, Kafka Connect and tools require Java 17 from Kafka 4.0, while clients and Streams applications need at least Java 11. Inventory these workloads so that a runtime retirement does not leave an unpatched consumer holding credentials.
Record the Java build in change records
Store the full Java version string, including the build and vendor, in every change record that touches a broker, controller or Connect host. When a security notice arrives, you need to answer quickly which nodes run an affected build and when each was last restarted. Capture the version from the running process rather than from the install package, because a host can hold a newer package while the service still runs the old one. Keep these records with your Kafka version and configuration changes.
Questions teams ask about Kafka JVM settings
Can we set the heap size alone and leave other JVM settings at defaults?
Setting the heap is a reasonable first step, but it rarely covers the whole job. Kafka brokers depend heavily on the operating system page cache for reading and writing log segments, and that memory sits outside the JVM heap. Garbage collector choice, pause behavior and the Java runtime version all affect latency under load. Review the G1GC settings in the Kafka operations documentation as a baseline, then confirm them against the release you run. Changing only the maximum heap without measuring garbage collection pauses and broker request latency leaves the most likely failure points unexamined. Plan a heap change as a measured experiment in staging before it reaches production.
What makes a JVM change costly to test and roll out?
The cost comes from three areas that recur with every change:
What does the team need in-house before changing Kafka JVM settings?
The team needs four things before it changes JVM settings on a production cluster. First, someone who can read garbage collection logs, interpret heap usage and recognize allocation problems. Second, a staging cluster with hardware close enough to production that results carry over, plus a way to replay or simulate production traffic. Third, change windows with on-call coverage for the restart period. Fourth, operators who already perform rolling broker restarts and can confirm partition leadership and replication health between steps. If one of these is missing, ask a partner to cover that gap in the written scope, or ask your own team to build it first.
How long does a JVM rollout take for a small cluster compared with a large one?
The rollout time grows with broker count, because each broker restarts in turn and the operator waits for replication and leadership to settle before moving to the next one. Controllers add their own steps, since the quorum must stay healthy throughout. A small cluster with few brokers finishes quickly, while a large multi-site cluster can take far longer because every restart needs verification. No duration is stated here because it depends on your hardware, traffic, replication settings and how much verification you require. Ask any partner for a rollout plan that lists each step, the checks at each step and the rollback point, then compare the duration against your change calendar.
Does the Java version have to match the Kafka version?
The Java version is set by the Kafka release, so check the upgrade notes before you choose a runtime. According to the Kafka upgrade notes, since Kafka 4.0 brokers, Kafka Connect and tools require Java 17, while clients and Kafka Streams applications need at least Java 11. Confirm the exact runtime list for the Kafka release you plan to run, because the supported runtimes can change between releases. Your client applications may stay on an older Java version than the brokers, but each client team should verify its own runtime against the client library it uses.
Why do Java patch updates matter for Kafka security?
Java patch updates fix vulnerabilities in the runtime that Kafka uses for its network, TLS and authentication code. Kafka secures client, broker and tool connections with SSL/TLS and SASL, so a flaw in the runtime can affect those controls even when Kafka itself is configured correctly. Treat Java patching as a recurring operational task with its own cadence, not as a one-time setup step. Each patch needs the same test and rolling restart cycle as a settings change, so budget for it in the same change process. Confirm your patch process with your security team and document how quickly critical runtime fixes reach production.
When is it worth bringing in a partner for Kafka JVM tuning?
A partner is worth considering when your team lacks JVM or Kafka operations depth, when production latency problems have no clear cause, or when you run several clusters that need the same controlled process. DeployOpen implements, hardens, supports and upgrades self-hosted open-source platforms in your own infrastructure, and most engagements are fixed-scope with scope, deliverables and timelines agreed in writing before work starts. Ongoing tuning, patching and upgrades can also be covered by a retainer. Ask for a written scope that names the tests, the staging environment, the rollout steps and who performs the restarts, so you can compare it with doing the work in-house.
Official Kafka sources for Java requirements
Before you pick a Java version or heap size, read the official Kafka material that states the requirements for your release. Start with the Kafka downloads page, which lists the supported releases and their dates, so you can confirm that the release you plan to run is still maintained. Then read the upgrade notes for the release line you are targeting, since they state the minimum Java version for brokers, Kafka Connect, tools, clients and Kafka Streams applications. Treat those notes as the authority for your version, and confirm the exact Java build your vendor or security team has approved against them.
The security overview is the next read once the runtime is settled, because JVM choices affect how the TLS and SASL settings behave in practice. Kafka supports SSL/TLS for encryption and SSL or SASL for authentication, and the mechanisms and listener configuration are documented there. Read it alongside your own inventory of Java runtimes on broker and controller hosts, including any bundled JDKs in container images. Ask your team to confirm which JDK distribution provides patch-level security updates, how quickly it ships them, and whether your change process can roll a patched runtime through a Kafka cluster without breaking the rolling upgrade path the release plan describes.
How DeployOpen scopes and delivers Kafka JVM work
DeployOpen treats the Java version, heap size and garbage collector for a Kafka cluster as part of the platform design, not an afterthought. We agree the runtime baseline in writing, build and test it in your infrastructure, and hand over settings your team can maintain through each Java patch release.
How we scope
- Confirm the Kafka release line, the Java version your brokers, controllers and Connect workers will run, and any client Java versions to support
- Record host memory, the heap and page cache split, and the GC settings each node role will use in the written scope
- Agree how Java patch updates will be tested, approved and rolled out, and who in your organization signs off on them
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts
How we build
- Set the JVM options for brokers, controllers and Connect workers through the deployment configuration, with each role documented separately
- Validate heap sizing and GC behavior under representative load and record the GC log and pause results as evidence
- Confirm the chosen Java build is in the supported set and that the runtime is patched to the vendor's current update level
- Keep configuration in version control so every change to heap or GC flags has a reviewable history
How we hand over
- Deliver a runbook covering the Java version, heap sizing rationale, GC flags, and the procedure for applying patch-level Java updates
- Document how to read GC logs and which alert thresholds your team should monitor for pauses and heap pressure
- Walk your platform and operations engineers through the upgrade steps for both Kafka and the Java runtime
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped in writing
- Retainer work can include reviewing Java security updates, re-testing heap and GC settings after Kafka upgrades, and updating the runbook
- Ask any provider which Java builds they will patch, how quickly they respond to security advisories, and how changes are tested before rollout
