Apache Kafka deployment and infrastructure

Installing and running Kafka on servers, containers and Kubernetes: KRaft bootstrapping, the official images, hardware, OS and JVM settings, node pools, OpenShift, restricted networks and multi-site layouts.

Bootstrapping a new KRaft-mode Kafka cluster

Standing up Kafka 4 from scratch: generating a cluster ID, formatting controllers with kafka-storage as standalone or with initial controllers, controller.quorum.bootstrap.servers, formatting brokers and verifying the quorum.

Running Apache Kafka in Docker with the official image

Using apache/kafka images beyond a laptop: pinning version tags, configuration through environment variables, persistent volumes for log and metadata directories, listener and advertised listener settings, and why the native image is not for production.

Apache Kafka hardware sizing

Sizing broker and controller hosts: memory for page cache with the 30-second buffering rule of thumb, disk throughput as the usual bottleneck, separate data drives, network for replication and how many controllers to run.

Linux and filesystem tuning for Kafka brokers

OS settings that matter for Kafka: file descriptor limits, vm.max_map_count for many partitions, XFS versus EXT4 and their mount options, noatime, socket buffers for WAN links and leaving flush to the OS.

JVM and Java settings for Apache Kafka

Choosing a Java version and heap for Kafka: supported Java 17, 21 and 25, the documented G1GC settings, heap versus page cache, and why patch-level Java updates matter for security.

Running Apache Kafka on OpenShift

Deploying Kafka on OpenShift with the Strimzi operator: operator installation, Routes for external clients, security context constraints, storage classes and the support differences between community Strimzi and vendor builds.

Deploying Apache Kafka in air-gapped networks

Running Kafka without internet access: mirroring release archives and container images, building Connect images with connectors in advance, internal certificate authorities, and update processes for patch releases.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.