Building producers, consumers and stream processors on Kafka: idempotence and transactions, Kafka Streams in production, Spark, client libraries, HTTP access, retries and offsets.
Why retries can duplicate records and how the idempotent producer stops it: the default settings, the configurations that silently disable it, ordering guarantees and what idempotence does not cover.
Read-process-write without duplicates: transactional.id, sending offsets inside the transaction, read_committed consumers, zombie fencing and the limits of exactly-once with external systems.
Operating Kafka Streams services: sizing instances and threads, standby replicas and rack-aware assignment, exactly_once_v2, internal topic management, monitoring metrics and resetting applications.
Managing stateful Streams apps: RocksDB-backed stores and their changelogs, restore time after failures, memory and disk sizing, RocksDB metrics and keeping state on persistent volumes in Kubernetes.
Serving reads from Streams state: querying local stores, discovering which instance holds a key, exposing an RPC layer between instances and handling rebalances during queries.
What the server-side Streams rebalance protocol changes: production readiness in Kafka 4.2, how to enable it, monitoring with kafka-streams-groups and compatibility with older brokers and apps.
Reading and writing Kafka from Spark: the spark-sql-kafka package, offsets and checkpoints, at-least-once writes and deduplication, delegation token authentication and SASL_SSL settings.
Using Spring Kafka in Java services: KafkaTemplate and listeners, transactions and exactly-once, error handlers, non-blocking retries with dead letter topics and Micrometer observability.
Choosing non-Java Kafka clients: librdkafka and the confluent-kafka libraries for Python, Go and .NET, feature parity with the Java client, idempotence and transactions, and version alignment with brokers.
Giving HTTP-only clients access to Kafka: the Strimzi HTTP Bridge, the Karapace REST proxy, authentication in front of them, consumer instance handling and throughput limits compared with native clients.
Handling poison messages without blocking partitions: blocking versus non-blocking retries, retry topics with delays, dead letter topics, ordering trade-offs and replaying dead letters.
Choosing how consumers commit progress: auto-commit defaults, synchronous and asynchronous manual commits, at-least-once versus at-most-once processing, auto.offset.reset and committing inside transactions.
What to know before building on ksqlDB: its Confluent Community License and the managed-service restriction, self-hosting implications, and open-source options such as Kafka Streams and Apache Flink SQL.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.