What moving off Confluent Platform to Apache Kafka involves
Moving a self-managed Confluent Platform estate to Apache Kafka is a two-part job. The brokers and topics move onto Apache Kafka, which is licensed under the Apache License 2.0 and has no paid edition from the project itself, so the core cluster can run in infrastructure you control without a vendor license. Schema Registry, REST Proxy, ksqlDB and some connectors do not come across automatically, and each needs its own decision. Topic data, consumer offsets and ACLs can be carried with documented tooling, but the effort depends on how much of your estate relies on those optional components. Start by listing which Confluent pieces each team actually uses, because that inventory sets the scope more than the broker count does.
Schema Registry has the most direct replacement path. Apicurio Registry, licensed under Apache 2.0, exposes a Confluent Schema Registry compatible API at /apis/ccompat/v7 and /apis/ccompat/v8, both listed as fully supported, so clients may need only a registry endpoint change. Karapace is another Apache 2.0 option, and its Schema Registry is API-compatible with Confluent Schema Registry 6.1.1, an older baseline you should test against. Karapace also implements the Kafka REST API, so one project may address two components. ksqlDB has no supplied drop-in replacement, so you must decide whether its queries move to Kafka Streams applications, another stream processor or are retired. Confirm each choice against your client versions before cutover.
Connector license checks are the step most often left until late. Each connector you run, whether published by Confluent or a third party, carries its own license and support terms, and some may sit under Confluent Community or Enterprise terms instead of Apache 2.0. Read the license file of every connector and Confluent component in the inventory, and record which ones your organization may run self-hosted and which need a replacement. Pair that review with security work: the SASL mechanisms your clients use, such as SCRAM-SHA-512 or OAUTHBEARER, and the StandardAuthorizer ACLs in the metadata log must be rebuilt for the new cluster, and client access should be tested after cutover. Ask any implementation partner to base its quote on that written inventory.
Which Confluent Platform capabilities map to Apache Kafka and which need a substitute
This table compares the main Confluent Platform capabilities with their Apache Kafka or open-source equivalent and notes what changes during migration. Read the Apache Kafka column as the core that moves over directly, and treat rows marked for confirmation as work to scope before cutover. The main takeaway is that the broker, ACL and consumer model carries over, while the surrounding tools need explicit substitutes.
| Confluent capability | Platform equivalent | Migration notes |
|---|---|---|
| Brokers and topics | Apache Kafka brokers and topics under Apache 2.0 | Topic configuration, partition counts and retention settings carry over, but confirm each topic's settings against the target cluster before cutover. |
| Consumer groups and offsets | Apache Kafka consumer groups and committed offsets | Offsets can be moved with planned consumer switchover; test resets and duplicate handling in a staging environment first. |
| ACLs and SASL credentials | Kafka SASL mechanisms (SCRAM-SHA-256, SCRAM-SHA-512, GSSAPI, OAUTHBEARER) and StandardAuthorizer ACLs | Recreate ACLs and credentials in the target cluster. Without ACLs, resources are restricted to super users unless the allow-everyone setting is changed. |
| Schema Registry | Apicurio Registry or Karapace (both Apache 2.0) | Apicurio Registry offers Confluent-compatible endpoints at /apis/ccompat/v7 and /apis/ccompat/v8. Karapace is API-compatible with Confluent Schema Registry 6.1.1. Confirm the version your clients need. |
| REST Proxy | Karapace REST (Apache 2.0) | Karapace implements Kafka REST. Confirm endpoint and version compatibility with each HTTP client before switching. |
| ksqlDB | No direct Apache Kafka equivalent in this scope | Confirm a substitute with each team, such as Kafka Streams applications or another stream processor. Rewrite queries and test state stores. |
| Connect workers and connectors | Apache Kafka Connect and MirrorMaker 2 for replication | Check each connector's license and availability before migration. Connect runs on Java 17 for Kafka 4.x. |
| Control Center style monitoring | Open-source metrics and dashboards (confirm tooling) | Confirm a substitute for alerting, lag views and cluster visibility. Rebuild dashboards before retiring Control Center. |
| ZooKeeper-based deployments | KRaft mode only in Apache Kafka 4.x | ZooKeeper-based clusters must migrate to KRaft before upgrading to 4.x. Confirm the migration path first. |
Which parts are Apache 2.0 and which sit under Confluent licenses
Apache Kafka itself is released under the Apache License 2.0 (SPDX: Apache-2.0), and the Apache Software Foundation project offers no paid edition. Confluent Platform, however, is a bundle of components that carry their own terms. Some are under the Confluent Community License, some under the Confluent Enterprise License, and some connectors or plugins may carry separate commercial or source-available terms. The Kafka broker you move to is not the component that usually creates the license question. The surrounding tooling is.
Start with a component inventory for every host, container image and build pipeline in the estate. For each item, record the product, version, package source, and the license file or terms that govern it. Cover Schema Registry, REST Proxy, ksqlDB, Kafka Connect workers, Confluent-branded connectors, and any proprietary transforms, converters or authentication plugins that were added by your teams or by a vendor. Also record which items are only used in a test or staging environment, since the license question still applies to them if they are copied into production. Confirm the terms with Confluent and with each connector publisher rather than assuming them from the package name or from a repository README.
The inventory decides what can be kept, replaced or retired. A component under terms that permit your planned use can stay in place. A component whose terms do not fit the planned deployment, such as a hosting model or a number of nodes, needs a replacement or a retirement date. Replacement candidates are covered in the sibling sections on Schema Registry, REST Proxy and ksqlDB, and connector choices are covered elsewhere on this page. The reading of any license, and whether your planned use is permitted, is a legal question. Involve your legal and procurement teams before the cutover plan is fixed, and ask DeployOpen to document each finding so that your counsel can review it against the source terms.
Documented tools for each part of the move
A move off Confluent Platform splits into separate parts: moving topic data and consumer state, keeping schemas available, and bringing cluster metadata onto KRaft. Each part has a documented tool or document, and each tool covers only some of the work. Check every card against your own inventory before you choose a sequence.
MirrorMaker 2 for topics and offsets
MirrorMaker 2 copies topics, their configuration, consumer group offsets and ACLs between clusters while preserving partitioning. It is built on Kafka Connect, so it runs as a connector workload you operate. It does not move Schema Registry subjects, Connect connector definitions, ksqlDB state or client code. Test offset translation on a replica topic, confirm ACLs arrive as expected, and rehearse a consumer cutover before production traffic moves.
Apicurio Registry compatibility API
Apicurio Registry is Apache 2.0 and exposes a Confluent Schema Registry compatible API at the /apis/ccompat/v7 and /apis/ccompat/v8 paths, both listed as fully supported. It lets producers and consumers keep their existing Schema Registry client configuration while the registry changes behind it. It does not migrate subject history automatically in every case, so verify schema IDs, compatibility modes and serializer behavior for each subject before switching clients.
Karapace for Schema Registry and REST
Karapace is an Apache 2.0 implementation of Kafka REST and Schema Registry, and its Schema Registry is API-compatible with Confluent Schema Registry 6.1.1. Choose it when you need REST Proxy style HTTP access as well as schema management. Confirm the Confluent client version and serializer settings in use, because compatibility is stated against 6.1.1 rather than later Confluent releases. Test every REST endpoint your applications call, since ksqlDB and Connect REST paths are not covered by this card.
Kafka upgrade notes for ZooKeeper to KRaft
The Kafka upgrade notes say ZooKeeper-based clusters must move to KRaft before upgrading to 4.x, because Apache Kafka 4.0 supports only KRaft mode. Clusters older than 3.3.x should first reach 3.9.x. Use the documented migration phases to plan the controller rollout, and confirm your target release against the downloads page. This card covers the metadata change only; security and client settings that depend on ZooKeeper still need their own review.
Confluent licensing and component inventory
Before you pick a tool, list every Confluent component in use and its license. The Apache 2.0 Kafka broker does not include Schema Registry, REST Proxy or ksqlDB, so each of those needs its own replacement decision. Confirm the license terms for each connector and component in your environment with the vendor, because licensing is set per component and is not implied by the broker license.
How the cutover runs in phases with a parallel run
The sequence moves a Confluent Platform estate to self-hosted Apache Kafka in stages, so each layer is proven on the new cluster before any client or component is switched and before the old estate is retired. Every stage ends with a written check that lets the team either continue or fall back.
Inventory clusters, topics, schemas and licenses
Start by listing every Kafka cluster, topic, partition count, retention setting, ACL, schema subject, Connect connector, producing and consuming client, and Confluent component in use, including which ones run under the Confluent Community or Enterprise license terms. Your team and the partner review this together, with the license owner signing off on which components are in scope. The step closes with a migration inventory that names an owner for each item and flags any component with no open-source replacement planned.
Build the target Kafka cluster
Deploy the new cluster on a supported Apache Kafka 4.x release in KRaft mode, with Java 17 for brokers and Kafka Connect, and with TLS and SASL in place before any client connects. Confirm the chosen SASL mechanism against your identity source and set authorization on the StandardAuthorizer so that resources without ACLs are not open by default. The step is done when a test client can authenticate, produce and consume over the secured listener and the cluster metadata passes your health checks.
Replace Schema Registry and REST endpoints
Deploy the replacement for Schema Registry and any REST Proxy use, selecting an open-source option after checking its compatibility with the Confluent API version your clients call. Copy schemas from the existing registry into the new one, then test serialization and deserialization for each subject against both registries. The step closes when every subject resolves to the same schema ID and version behavior, and when compatibility rules for each subject match the source registry.
Replicate topics and consumer offsets
Run MirrorMaker 2 from the source cluster to the target so that topics, their configuration, consumer group offsets and ACLs are copied continuously with partitioning preserved. Check that replicated ACLs grant the same access as the source and that consumer offsets advance on the target as they do on the source. The step is complete when lag and replication health are stable and a sample of consumer groups shows matching offset positions for the topics in scope.
Move clients in waves with a parallel run
Group producers and consumers into waves by business owner and risk, and switch each wave to the new bootstrap servers and registry endpoints while both estates stay live. Run each wave in parallel with the old path for a period agreed in advance, comparing message counts, schema errors and processing results between the two. The step is done when a wave shows no unexplained differences and its owner signs off in writing.
Cut over with a defined rollback window
Stop writes to the source for the remaining clients, let MirrorMaker 2 drain, and confirm offsets and topic contents match before the final switch. Keep the source cluster and its reverse replication available for a rollback window agreed before cutover, with a documented decision point for returning traffic. The step closes when all production traffic runs on the new cluster and the rollback decision has been recorded as closed.
Decommission Confluent components after checks
Before turning off Confluent components, confirm with the license owner that no remaining use requires them, and check the data retention obligations that apply to topics, schemas and logs held in the old estate. Archive or delete data according to those obligations, then remove the components and their access. The step is complete when a signed decommission record lists each retired component, the license status and the location of any retained data.
What drives the cost and duration of the move
This table lists the factors that change the size of a Confluent Platform to Apache Kafka move and what to ask for in a quote. Read each row as a question to answer about your estate before comparing proposals, since most quotes move with these inputs. The main takeaway is that the migration build and cutover are one-time work, while running, upgrading and patching the platform recur for as long as you operate it.
| Driver | What it changes | What to ask for in a quote |
|---|---|---|
| Clusters and topics | Each cluster needs its own cutover plan, and topic counts drive partition mapping, ACL rebuilds and MirrorMaker 2 or replication work. | An inventory of clusters, topic counts and partition counts, with the replication approach for each. |
| Schemas and compatibility modes | Every subject, version and compatibility setting must be exported and validated against the replacement registry, which takes longer when many subjects use non-default modes. | Subject and version counts, the compatibility mode per subject, and the replacement registry and its API version check. |
| ksqlDB queries | Persistent queries, tables and streams must be rewritten or replaced with a stream processor, and each stateful query needs its own state and output validation. | A list of persistent queries and their outputs, with the target technology for each and the validation method. |
| Connectors and license status | Each connector must be checked for license terms, replaced or rebuilt if its license or vendor does not fit your plan, and retested with its converters and offsets. | A connector inventory with version, source license, the owner of the connector and the replacement or retention decision. |
| Client applications and languages | Each producer and consumer needs client library changes, configuration updates, and a test run in each language or framework in use. | Counts of applications by language and client library, plus the owners who sign off on cutover. |
| Security model migration | Moving from Confluent RBAC or existing ACLs means mapping roles to Kafka authorization rules, reissuing credentials, and testing access after each step. | A mapping of current RBAC roles and ACLs to target ACLs, the authentication mechanism chosen, and access tests planned. |
| Java runtime upgrades | Kafka 4.0 and later require Java 17 for brokers, Connect and tools, so hosts, images and application runtimes may need upgrading. | Confirmation of the Java version on every broker, Connect worker and client, and the upgrade work included. |
| Parallel run | Running both platforms during cutover adds infrastructure, duplicate consumption checks, offset handling and a longer test window. | Whether a parallel run is included, its duration, and who monitors data equality during it. |
| Recurring operations | Running brokers, registries, Connect, upgrades and patching continues after cutover and is usually priced as a retainer or staff time. | Support hours, upgrade cadence, patch handling and who owns on-call response after go-live. |
What usually goes wrong and how to prevent it
Most failed Confluent Platform exits trace back to a few assumptions: that the replacement registry behaves like the original, that replicated data and offsets stay in step, and that security and licensing carry over unchanged. Each card below names one of these risks and a check the team can run before cutover.
Schema registry behavior differs
A replacement registry can accept the same API calls yet resolve compatibility modes, subject naming or schema IDs differently from the Confluent Schema Registry you run today. Apicurio Registry offers a Confluent-compatible API under /apis/ccompat/v7 and /apis/ccompat/v8, and Karapace targets API compatibility with Confluent Schema Registry 6.1.1. Replay a sample of production schemas and serializer calls against the candidate registry, and compare compatibility check results for every subject before producers switch.
Offsets out of step with data
Copying consumer group offsets without the matching topic data, or replicating data without the offsets, causes consumers to skip or reprocess messages. MirrorMaker 2 can replicate topics, consumer groups with offsets and ACLs, but only if the source and target stay in a consistent state during cutover. Pause writers, let MirrorMaker 2 catch up, translate offsets, then confirm committed positions on the target for every critical group.
ACLs and credentials mis-scoped
Kafka's StandardAuthorizer keeps ACLs in the cluster metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. Confluent-era principals, SCRAM users and SASL mechanisms rarely map one to one onto the new cluster, so producers and consumers can fail or receive broader access than intended. Export the source ACL list, rebuild it as explicit rules, and test each client identity with a denied-by-default check.
Hidden ksqlDB dependencies
Teams often discover ksqlDB late because a persistent query, a materialized table or a dashboard reads from it without a clear owner. Kafka itself does not provide a ksqlDB replacement, so each query must be rewritten as a Kafka Streams application, a connector pipeline or another stream processor. Inventory every ksqlDB cluster, persistent query and downstream reader, and assign a replacement owner and test before cutover is scheduled.
Connector licenses block design
Confluent Community and Enterprise licenses cover different components, and some connectors carry their own terms, so a target design that works technically may not be permitted. Before cutover, list every connector, its license, its version and whether it runs on the target Connect workers in your infrastructure. Confirm each license with the publisher or your counsel, and record the result in the design document so the decision is not left implicit.
Java and Kafka version mismatch
Kafka 4.0 and later require Java 17 for brokers, Kafka Connect and tools, while clients and Kafka Streams applications need at least Java 11. The upgrade notes also recommend that KRaft clusters older than 3.3.x move to 3.9.x before 4.x. Check the source platform's Java runtime, client library versions and connector builds against these requirements, and test each client on the target runtime before switching traffic.
No tested rollback path
A cutover without a rehearsed reverse path leaves the team stuck if consumers misbehave or schemas fail after the switch. Define the rollback trigger, keep the source cluster and its registry running and writable until the agreed exit point, and keep MirrorMaker 2 or equivalent replication flowing in the reverse direction for the rollback window. Rehearse the full reversal in a staging copy and record how long each step takes in your environment.
Questions teams ask before moving from Confluent Platform
What does it cost to move from Confluent Platform to Apache Kafka?
The cost is driven by the written scope, not by a list price, because DeployOpen does not publish one and a quote follows from the scope agreed in writing. The main components are the planning and inventory of your current estate, the rebuild of brokers and controllers on Kafka, the replacement of Schema Registry, REST Proxy and ksqlDB where they are used, connector checks, client cutover, and documentation and handover. Costs rise with the number of clusters, topics, schemas, connectors, clients and consumer groups that must move, and with the number of environments that must be validated. Ask for the quote to state which items are one-time work and which are recurring, such as support or upgrades under a retainer. Apache Kafka itself carries no license fee, since it is released under the Apache License 2.0 (SPDX: Apache-2.0), so the spend sits in people, infrastructure and any support you buy.
How long does a Confluent Platform to Apache Kafka move take?
The duration depends on how many clusters and workloads are in scope, how many clients can be switched without downtime, and how much of the estate depends on ksqlDB or custom Schema Registry behavior. The work usually runs in phases: inventory, a parallel target cluster, replication of topics and consumer offsets, staged client cutover, and decommissioning. Your own change windows and approval cycles often set the pace more than the technical work does. Ask for a timeline in the written scope that names each phase, the environments covered and the cutover approach for each application. Kafka's own release policy matters too: the project lists 4.3.1, 4.2.2 and 4.1.2 as supported releases, and you should confirm which target version your platform team will standardize on before the plan is fixed.
What must our team provide before the engagement starts?
Your team provides access, facts and decisions. Expect to supply an inventory of brokers, Connect workers, Schema Registry instances, ksqlDB applications, topics, ACLs, client applications and their owners, plus the infrastructure where Kafka will run. You will also need to name an approver for cutover windows, a contact for each application team, and the security owner who signs off on TLS, SASL and authorization design. Access to licensing terms for each Confluent component and connector in use is needed so the license review can be completed. Without these inputs the quote will carry more assumptions, and the timeline will carry more risk. Confirm the exact list during scoping, because it depends on which Confluent products you actually run.
Can our internal team migrate Confluent Platform to Apache Kafka alone?
An internal team can do it, provided it has people who already operate Kafka in production, understand KRaft, and can own security and client changes across many application teams. The hard parts are rarely the broker installation. They are the cutover order, the replacement of Schema Registry and ksqlDB behavior, the verification of offsets and ACLs after each phase, and the rollback path if a consumer misbehaves. Many teams choose a mixed model: their own staff keep ownership of applications and approvals while a partner handles the platform build and the test plan. Decide this by comparing the skills you already have against the list in the scope, and ask a partner to show how the work would be split between your team and theirs.
Can Karapace or Apicurio Registry replace Confluent Schema Registry?
Both are open-source options that expose Confluent-compatible behavior, and each should be tested against your own schemas and clients before you commit. Apicurio Registry is licensed under Apache 2.0 and offers a Confluent Schema Registry compatible API at /apis/ccompat/v7 and /apis/ccompat/v8, which its documentation lists as fully supported. Karapace is an Apache 2.0 implementation of Kafka REST and Schema Registry, and its Schema Registry is API-compatible with Confluent Schema Registry 6.1.1 according to its repository. Confirm which API version your serializers use, how compatibility modes are set, and whether your data contracts depend on any Confluent-specific feature. Run serializer and deserializer tests against the candidate registry before cutover.
What happens to our ksqlDB workloads during the move?
ksqlDB is not part of Apache Kafka, so each ksqlDB query, stream and table has to be reviewed and either rebuilt on another stream processor or kept on its own runtime pointed at the new cluster. The inventory should list every persistent query, its input and output topics, its state stores and the consumers that depend on its output. Some teams rewrite these as Kafka Streams applications, which are part of the Apache Kafka project, while others choose a different processing tool. Ask for the decision on each query to be recorded in the scope, along with a test that compares outputs between the old and new pipelines. Pausing or rewriting ksqlDB is often the longest item on the plan, so it should be sized early.
How does licensing affect the Kafka connectors we run?
Apache Kafka is under the Apache License 2.0, but connectors are separate artifacts, and each one carries its own license, which may be open source, source-available or commercial. Some connectors shipped with Confluent Platform fall under the Confluent Community or Enterprise license, so moving the broker does not automatically clear the connectors. Build a connector register that lists each plugin, its version, its license, its vendor and whether it is used in production. Then decide, for each one, whether to keep it, replace it with an open-source alternative or buy a license. Ask for this review to be part of the written scope and for the license findings to be confirmed by your legal or procurement team, since DeployOpen prepares the evidence but does not give legal advice.
How DeployOpen works on this
DeployOpen plans, implements, customizes, integrates, documents and supports self-hosted open-source platforms inside your own infrastructure. For a Confluent Platform move, we start by mapping every component your estate actually uses, then agree the replacement path, the cutover plan and the acceptance tests in writing before build work begins.
How we scope
- We inventory the brokers, Schema Registry subjects, REST Proxy clients, ksqlDB queries and connectors in use today.
- We record which Confluent components are licensed under Confluent Community or Enterprise terms and which are removed from the target.
- Engagements are fixed-scope, so deliverables, timelines and responsibilities are written down before work starts.
How we build
- We stand up the target Apache Kafka cluster in your environment and run a Schema Registry replacement alongside it for testing.
- We check each connector's license and plan a replacement or retirement for any that cannot run on the target.
- We test producers, consumers, schema compatibility and access controls in a staging environment before any production cutover.
How we hand over
- You receive configuration, runbooks, cutover records and the test evidence gathered during the migration.
- We document the replacement for each Confluent component so your team can operate the new estate without us.
- Handover includes a review of rollback steps and the decommissioning checklist for the old Confluent services.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after the migration closes.
- Support can cover Kafka upgrades within the 4.x line, schema registry maintenance and connector updates.
- Ask any provider to confirm in writing which post-migration tasks are inside the quote and which are optional.
