Streaming Kafka topics into Apache Iceberg tables
Landing Kafka data in a lakehouse with the Iceberg Kafka Connect sink: catalog choices, the control topic and commit coordination, exactly-once delivery, multi-table routing and schema evolution.
Apache Kafka Connect connectors and pipelines
Building data pipelines with Kafka Connect: open-source connectors for lakehouses, databases and search, error handling, transforms, exactly-once delivery, Kubernetes deployment and connector licensing.
Writing Kafka topics to databases with the Debezium JDBC sink
Replicating change events into PostgreSQL, MySQL or other databases: upsert mode for idempotent writes, primary key modes, delete handling, schema evolution limits and error routing.
Apache Kafka and MongoDB with the MongoDB Kafka Connector
Moving data between Kafka and MongoDB with the official source and sink connectors: change streams on replica sets, sink write models, converters and deployment on Connect workers.
Indexing Kafka topics into OpenSearch
Feeding search and log analytics from Kafka with Aiven's Apache 2.0 OpenSearch sink: index naming, document IDs, schema handling, TLS to OpenSearch and handling mapping errors.
Apache Camel Kafka connectors
Using Apache Camel Kafka Connector when no dedicated connector exists: the catalog of sources and sinks such as RabbitMQ, JMS, MQTT and S3, configuration style, versioning and production readiness checks.
Kafka Connect error handling and dead letter queues
Keeping pipelines running through bad records: errors.tolerance, retries and timeouts, logging error context, routing failures to a dead letter queue topic with context headers and reprocessing them.
Kafka Connect single message transforms
Reshaping records inside Connect: the included transforms and predicates, routing and field changes, when SMTs are enough and when a stream processor is the better place for the logic.
Exactly-once delivery in Kafka Connect
Getting exactly-once from Connect pipelines: read_committed for sinks, enabling exactly-once source support with two rolling upgrades on existing clusters, extra ACLs and which connectors can actually use it.
Running Kafka Connect on Kubernetes with Strimzi
Operating Connect clusters through Strimzi: building images with connector plugins, managing connectors as custom resources, secrets for credentials, scaling workers and upgrades.
Managing Kafka Connect plugins
Installing and controlling connector plugins: plugin.path layout, plugin discovery modes and service_load compatibility, the plugin migration script, isolating versions and only installing trusted code.
Kafka Connect connector licensing: Apache 2.0 vs Confluent licenses
Checking connector licenses before you depend on them: Apache 2.0 connectors from Debezium, Apache projects and vendors, the Confluent Community License restriction, enterprise-licensed connectors and open-source substitutes.
