Integration pages for the systems most often wired to a self-hosted Kafka cluster: Kubernetes operators, change data capture, stream processors, analytics databases, data flow tools and identity providers, each based on the documentation of the project that provides the integration.
Running Kafka on Kubernetes with the Strimzi operator: cluster, topic and user custom resources, Connect and MirrorMaker managed the same way, listener security options, GitOps workflows and what the operator does not take off your hands.
Streaming database changes into Kafka with Debezium source connectors on Kafka Connect: supported databases, connector deployment, topic layout and the operational work of snapshots, schema changes and connector failures.
Processing Kafka topics with Flink through KafkaSource and KafkaSink: offset starting points, checkpoint-based offset commits, exactly-once sinks using Kafka transactions and the transactional ID setup they need.
Loading Kafka topics into ClickHouse with the Kafka table engine and materialized views: consumer groups and parallelism, formats, at-least-once semantics and durability settings for analytics pipelines.
Moving data between NiFi flows and Kafka with ConsumeKafka and PublishKafka: the Kafka connection controller service, record versus per-message processing, offset commits and exactly-once patterns.
Authenticating Kafka clients and brokers with OAuth tokens from Keycloak or another OIDC provider using SASL/OAUTHBEARER: JWKS validation on brokers, client credentials on clients, TLS requirements, and mapping token principals to ACLs.
Loading Kafka topics into Druid for real-time analytics: the Kafka indexing service extension, supervisors and task management, exactly-once ingestion by partition offsets, and sizing ingestion tasks.
Scaling consumer deployments on lag with the KEDA Kafka scaler: lagThreshold, the partition count cap and allowIdleConsumers, SASL and TLS authentication, and avoiding rebalance churn while scaling.
Wiring a Fineract core banking system to Kafka: enabling external events, the Avro event format and ordering, at-least-once delivery and idempotent consumers, and using Kafka for COB remote partitioning.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.