Confluent Schema Registry migration to Apicurio or Karapace

What moving off Confluent Schema Registry involves for a self-hosted Kafka team

A team that runs Kafka on its own infrastructure and stores Avro, Protobuf or JSON Schema subjects in Confluent Schema Registry can move to an Apache 2.0 or other open-source registry, but the work is mainly about preserving schema IDs and compatibility history while clients are repointed. Effort depends on three things: how many subjects and versions exist, how many client languages and deployment models read and write them, and how strictly old records must stay readable after the switch. The Kafka platform itself is licensed under the Apache License 2.0 (SPDX: Apache-2.0), and registry choice is a separate decision from broker choice, so the broker cluster does not have to change to make this move.

Schema IDs are the part that most often breaks. Serialized records carry a schema ID in their wire format, so every ID referenced by a record still in a topic, a retained log segment or a consumer offset must resolve after cutover. Confirm with the chosen registry whether it can reproduce existing IDs on import, and whether its Confluent-compatible endpoints return the same IDs, versions and subject names that serializers expect. Also inventory compatibility settings per subject and global defaults, since a registry that accepts an import without carrying those rules forward changes what producers are allowed to publish. The Apicurio compatibility API and Karapace documentation each describe their own coverage, and the gaps should be checked against your actual serializer libraries rather than assumed.

Ask any partner for a written scope that lists subjects by format, the client inventory with serializer versions, the cutover method (dual-run, read-only freeze or staged repointing), the rollback plan, and the verification steps that compare subject lists, versions and schema IDs between old and new registries. Those items determine cost. Setup of the new registry is usually a small share of the work, while the client and verification effort grows with each team and language involved. Both the one-time migration and any ongoing hosting or support are priced from that written scope, and DeployOpen does not publish a price list.

Which Confluent Schema Registry capabilities map to Apicurio and Karapace

This table compares the Confluent Schema Registry features that existing clients depend on with their Apicurio and Karapace counterparts. Rows marked as items to confirm need checking against the current documentation of each project before you commit to a target, and the migration note in each row says what to test.

Capability	Apicurio Registry	Karapace
Subjects and versions	Artifacts and versions with groups; confirm how Confluent subjects map to artifact IDs and groups before export	Subjects and versions exposed through the Confluent-style REST paths; export subjects and versions in order and compare counts after import
Schema IDs in serialized records	Confluent-compatible API can serve IDs, but confirm that imported schemas keep the original global IDs embedded in existing records	Global schema IDs are kept in the Kafka-backed store; confirm ID continuity on import before retiring the source registry
Compatibility modes	Compatibility rules are configured per artifact or globally; confirm the transitive variants your consumers rely on	Per-subject and global compatibility settings with the Confluent mode names; confirm transitive modes behave as your tests expect
Supported formats	Avro, Protobuf and JSON Schema are documented; confirm the exact versions and any format-specific rule behavior	Avro, Protobuf and JSON Schema are documented; confirm support for each format your topics use
Normalization and references	Confirm whether normalization and schema references resolve the same way as Confluent before assuming identical results	Documented normalization limits apply; test schemas that use references, imports or nested definitions before cutover
Serializer configuration	Confluent serializers can point at the Confluent-compatible endpoint; confirm the URL, auth properties and any Apicurio-specific serializer settings	Confluent serializers can point at the Karapace URL; confirm client library versions and the subject naming strategy used by each producer
REST access and authentication	Confirm which auth methods the compatibility API accepts and whether they match your identity provider setup	Basic auth and TLS options are available; confirm how they integrate with your access model and gateway
Contexts and multi-tenancy	Confirm support for contexts or groups as used by your subjects before relying on them	Confirm whether context-qualified subject names are supported in your target version

Export, import and API paths to confirm before the first move

Each path below carries a different slice of what the source registry holds: subjects, version history, schema IDs and compatibility settings. Confirm in the target project's own documentation what each path preserves before you rely on it, because a path that copies schemas but reassigns IDs can break existing consumers without any visible error.

Confluent-compatible API on Apicurio

Apicurio Registry exposes a Confluent-compatible API so that existing serializers can keep calling the registry while you change the endpoint. This path lets producers and consumers continue to resolve schemas during a transition. Confirm which Confluent endpoints the Apicurio version you deploy implements, how it maps schema IDs, and whether serializer settings such as the ID handler need to match your current configuration.

Karapace REST API compatibility

Karapace implements a REST API designed to match the Confluent Schema Registry surface, so clients that call subjects, versions and compatibility endpoints can often be pointed at it directly. The compatibility is at the API level, not a guarantee for every edge case. Test the endpoints your clients actually call, including error responses and the handling of deleted versions, against a copy of your registry before cutover.

Export and import options per project

Each project documents its own export and import tooling, and the options differ in what they carry. Some carry subject history and schema IDs, others only the latest version per subject. Ask the project which artifacts its import tool writes, whether it preserves global IDs and version numbers, and whether compatibility rules and subject-level configuration come across. Record the answers in your migration plan before selecting a path.

Scripted reads from the source REST API

If no supported importer fits, a script can read subjects, versions, schema text, references and config through the source registry's REST endpoints and replay them into the target. This gives you full control over ordering and ID handling but puts the correctness burden on your team. Confirm which endpoints return schema IDs and references, and write a verification step that compares source and target for every subject before you accept the result.

Compatibility settings and deleted versions

Compatibility modes set at the global or subject level are a separate piece of state from the schemas themselves, and an export may not carry them. Soft-deleted versions are another gap, since they may be hidden from normal reads yet still referenced by old records. Confirm in each target's documentation whether it migrates compatibility configuration and deleted versions, and decide how you will reconcile any difference before producers switch.

Confirm the serializer and client path

Serializer libraries resolve schemas through a registry URL and an ID lookup, so the migration path depends on the client library version you run. Check the client documentation for the registry API it calls, whether it supports the target's endpoints, and which configuration keys select the target. Pin the library version in the migration test environment so the result you verify is the one you ship.

Why existing schema IDs in Kafka records decide the migration design

The schema ID embedded in each record decides how the migration is designed. Confluent-style serializers write a schema ID into the header of every message, and a consumer uses that ID to fetch the schema from the registry. Every ID already present in a topic that consumers may still read must resolve in the new registry to the same schema, or those records become unreadable. Before choosing between Apicurio and Karapace, list which topics hold records written under the current registry, how long those records remain retained, and which consumers may read them after cutover.

Teams usually take one of two paths. The first reproduces the original IDs in the target registry, so existing bytes on disk and in retained topics keep their meaning without any rewrite. This keeps the data path untouched but depends on the target registry honoring the source IDs when it imports subjects and versions, and on the ID space not colliding with IDs the new registry assigns later. The second re-serializes records into the new registry's IDs, which removes the dependency on preserved IDs but requires a rewrite job, a verified mapping table and a cutover window in which both old and new IDs can appear. Confirm with the chosen registry whether imports keep original IDs, how its Confluent-compatible endpoints assign IDs for new schemas, and whether ID ranges can be pinned.

Retention and compaction change the risk. With time-based retention, old IDs age out and the migration window shrinks as the retention period passes, so a team may choose to wait. Compacted topics keep the latest value per key indefinitely, which means an old ID can stay live in a topic for years, and a re-serialization job must cover those records or the registry must keep the original IDs. Also check whether consumers cache schemas by ID across restarts and whether any downstream store, such as a sink table or an archive in object storage, keeps the header bytes. Those copies sit outside Kafka and are the easiest to miss in an inventory.

How a registry migration runs from inventory to decommissioning

The sequence moves schemas from Confluent Schema Registry to the new registry without changing the schema IDs that existing records carry, and ends with a source registry that can be shut down on evidence rather than assumption. Each step closes with a record the team can review later.

Inventory subjects, versions and compatibility settings
The platform team lists every subject, its versions, global and per-subject compatibility modes, and the schema types in use (Avro, Protobuf or JSON Schema). They also record which producers, consumers and serializer libraries and versions read or write each subject, and which topics they map to. Confirm the client library versions against the registry APIs each target supports, since the Apicurio ccompat API and Karapace compatibility differ in detail. The step is done when the inventory is a single reviewed file that the owning teams have signed off, and every subject has a named owner.
Run a pilot on non-critical subjects
Choose a small set of low-risk subjects with a known consumer group and a team willing to test. Export them, register them in the target registry, and point a test producer and consumer at the new endpoint. Use this pilot to confirm that serializers fetch schemas correctly, that compatibility checks behave as the team expects, and that the team understands how error responses differ. The pilot closes with a written list of differences found, each with an owner and a fix or an accepted risk, plus a go or no-go decision for the wider migration.
Export subjects and verify schema IDs
Export every subject version from the source registry, preserving the original numeric schema IDs wherever the target registry allows it. Load the schemas into the target and compare IDs, subject names, version numbers and schema text for each entry. Produce an ID comparison report that lists matches, mismatches and any schema the target normalized differently. Serialized records in existing topics reference these IDs, so a mismatch means old messages cannot be decoded. The step is done when the report shows zero unexplained mismatches and the team has a verified backup of the source registry's storage.
Run both registries in parallel
Keep the source registry running while the target serves reads, and decide deliberately which side receives new writes during the overlap. A common pattern is to register new schemas in both registries through the same pipeline and compare the results after each change. Track any drift in IDs or compatibility outcomes and stop the overlap if drift appears. Keep per-client test results for each serializer library, covering produce, consume and schema evolution cases. The step is done when several representative schema changes have passed on both sides with matching outcomes, and the team has recorded each client's result.
Cut over producers and consumers
Move clients in groups, starting with consumers so they can read data written under either registry, then producers. Change the registry URL and credentials in configuration, roll out each group with a rollback plan that restores the old URL, and watch deserialization errors and schema lookup failures during the rollout. Record each service's cutover time, configuration change and error metrics before and after. The step is done when every client in the inventory reads and writes against the target registry and no client still calls the source endpoint in logs or traffic records.
Observe the source and then decommission it
Keep the source registry in read-only mode for a defined observation period agreed in advance by the platform, security and data owners. During that time, confirm through access logs that no client requests reach it, and that consumers can still decode older records from topics retained under the old IDs. Ask the owners to confirm the length of the observation period in writing, because the research supplied for this page does not set one. Then retire the source, remove its credentials, and archive the final export, the ID comparison report and the per-client test results with the change record.

How to switch producers and consumers without breaking deserialization

Java applications that use the Confluent Avro, Protobuf or JSON Schema serializers are the most common case. The change is often a registry address and, where used, a credential, because the serializer reads these from configuration. Before changing production, confirm that the target endpoint answers every subject, schema and compatibility call your serializer version makes.

Registry URL: Change schema.registry.url in producer and consumer configuration to the new endpoint, and keep the change in the same deployment unit as other client settings.
Credentials: Carry over basic.auth.credentials.source and basic.auth.user.info, or your equivalent, and confirm the new registry accepts the same credential format.
Deploy order: Move consumers first so they can read schema IDs written before and after the switch, then move producers so new records are written against the new registry.
Test: Run a consumer group against a topic that holds records written before cutover, decode each record, and compare the output with stored expected payloads.

Check that each serializer version in use is listed as compatible with the target registry in its documentation, and record the versions in the change ticket.

What drives the cost and duration of a registry migration

This table lists the factors that change how much work a move off Confluent Schema Registry takes, and how each one affects effort, whether the work is one-time or recurring, and what to ask a partner in a quote. Read it before comparing quotes, since the same platform can be a small or a large project depending on these inputs.

Driver	Why it changes effort	Question to ask in a quote
Number of subjects and versions	More subjects and versions means more export, ID mapping and verification work, and a longer parity check before cutover.	How many subjects and total schema versions are in scope, and how will each be counted and verified?
References between schemas	Schemas that import other schemas must be migrated in dependency order, and broken references are a common cause of failed registration.	How many schemas use references, and how will reference graphs be exported and rebuilt?
Mix of Avro, Protobuf and JSON Schema	Each format has its own compatibility behavior and normalization details, so each needs separate test cases against the target registry.	Which formats are in use per subject, and which formats does the target registry support for our versions?
Client applications and languages	Every producer and consumer that resolves schemas through the registry needs a serializer or URL change, and each language's client library needs testing.	How many client applications, in which languages, and who owns each one for the switch?
Retention and compaction on schema topics	If the registry stores schemas in a Kafka topic, retention and compaction settings decide how much history must be preserved and copied.	Where is schema data stored today, and what retention or compaction settings apply to it?
Required compatibility rules	Per-subject rules must be read, reapplied and tested on the target, and any rule the target does not match changes release planning.	Which compatibility modes are set per subject, and which of them must match exactly after the move?
Authentication and high availability	Access control, TLS, and clustered deployment on the new registry add setup, testing and recurring operations work.	What authentication and availability design is required, and who runs it after handover?
Compliance evidence needs	Assessors may ask for records of the migration, schema change history and access controls, which adds documentation and retention work.	What evidence must be produced for the migration, and in what format will it be delivered?

What usually goes wrong in schema registry cutovers and how to prevent it

Most schema registry cutovers fail in the gaps between the old and new systems rather than in the registry software itself. The six risks below cover identifiers, rules, endpoints, parallel-run data, feature gaps and evidence, and each has a check the team can run before it signs off the move.

Schema IDs resolving to different schemas

Serialized records carry a schema ID in their payload, so the new registry must return the same schema for every ID the old one issued. After import, the team should fetch a sample of IDs from both registries and compare the schema text byte for byte, including IDs for older versions of each subject. A changed ID mapping can make existing records undecodable without any error at produce time. Keep a written mapping of source ID to target ID and treat any mismatch as a stop condition.

Compatibility rules not carried over

Compatibility mode is set per subject and is easy to lose during export, since a bulk import may create subjects without re-applying their settings. A subject left at the default mode can accept a schema change that the old registry would have rejected. Export the mode for every subject, including global defaults, and compare the lists after import. Then run a test submission of a known incompatible schema to confirm the new registry rejects it.

Clients still pointing at the old endpoint

Producers and consumers hold the registry URL in configuration, often in several services, jobs and build pipelines. Any client left on the old endpoint keeps registering schemas in a registry that is no longer the system of record, so the two diverge. Inventory every service that uses a serializer, including batch jobs and tools, and record the URL each one reads. Confirm the old endpoint receives no requests for a full business cycle before decommissioning it.

Records written during the parallel run

A parallel run means producers and consumers may register or read schemas in both registries for a period. Records written with IDs from the new registry during that window can be unreadable by consumers still configured against the old one. Decide in advance which registry is authoritative for writes at each stage, and test a consumer that reads records from each side. Keep consumers able to resolve both ID sources until every reader has moved.

Serializer features the target does not support

Serializers can depend on API behavior beyond basic registration, such as particular formats, reference handling, normalization, or specific endpoints. A feature the target only partly implements may fail only on some message types, which makes it easy to miss in a happy-path test. List the serializer libraries and their versions in use, then check each feature against the target's documentation and the vendor or project notes. Confirm the behavior with a test that uses the production serializer configuration.

Decommissioning before audit evidence is captured

Once the source registry is shut down, its configuration, change history, compatibility settings and access logs may be gone. Auditors reviewing schema change control often ask for that history for the period before the cutover. Export the source data, configuration and logs to retained storage, record checksums, and confirm with your compliance lead which evidence the assessment requires. Keep the export read-only and tested for restore before the source is removed.

Questions teams ask before moving off Confluent Schema Registry

What does a migration off Confluent Schema Registry cost?

The cost is mostly the time of people who are paid to do the work, so it is set by scope rather than by the registry software, which is open source. Apicurio Registry and Karapace carry no license fee of their own, but the work has several parts: inventorying subjects and versions, exporting them, verifying that schema IDs still resolve, changing or re-deploying clients, running parallel reads and writes, and operating the new registry afterward. Each part is paid for either from your own engineering hours or from a partner's fixed-scope fee. The recurring cost is the infrastructure and on-call time for the registry you run. Ask any quote to separate one-time migration work from ongoing support, to list which client applications are in scope, and to state how cutover testing and rollback are priced.

How long does a schema registry migration take?

Duration depends on how many subjects, versions, clients and languages are involved, and on how many teams must change code or redeploy. A registry with a few subjects owned by one team is a very different project from one with hundreds of subjects, many producers and consumers in several languages, and no current owner for half of them. Schedule risk usually comes from client owners who are not reachable, from consumers that cannot be redeployed quickly, and from compatibility checks that fail when old and new registries are compared. Ask for a written plan with phases, entry and exit criteria for each phase, and the dependencies on other teams stated explicitly.

What must our internal team provide before a migration starts?

Three inputs decide whether the work can proceed on schedule. First, a subject inventory: every subject, its schema type (Avro, Protobuf or JSON Schema), its versions, its compatibility setting and any content rules. Second, named client owners for each producer and consumer, with authority to approve a redeploy and a rollback window. Third, test environments that mirror production topics and registry settings closely enough to rehearse export, ID checks and client switching. Your team should also confirm who holds credentials for the existing registry and who will own the new one after handover. A partner can help build the inventory tooling, but only your teams know which clients are live and which are abandoned.

Can our team migrate the registry without a partner?

Yes, a capable platform team can do this internally, especially for a small number of subjects with clear owners. The work is well defined: export subjects and versions, reproduce them with the same IDs in the target registry, and move clients in a controlled order. The hard parts are less technical than organizational, such as coordinating redeploys across teams and proving that no serialized record can fail to resolve. A partner is most useful when the inventory is large, when compliance evidence of the cutover is needed, or when internal staff are committed to other work. Ask a partner for a written scope that lists what they will test, what they will hand over and what your team must still run.

Should we choose Apicurio Registry or Karapace?

Choose based on the clients you must keep working and on the storage and operating model you want. Apicurio Registry offers a Confluent-compatible API through its ccompat layer, which helps existing serializers continue without code changes, and it supports several storage options and a richer governance feature set. Karapace is a Confluent-compatible schema registry and REST proxy that stores schemas in a Kafka topic, which suits teams that want fewer moving parts. Confirm each candidate against your actual serializer versions, the formats you use, the normalization and content rules you depend on, and the authentication and high availability model your platform requires. Test both with a copy of your subjects before deciding.

Do existing Kafka records need to be re-serialized?

Usually not. Each serialized Kafka record carries a schema ID in its header or payload prefix, and consumers use that ID to fetch the schema. If the new registry keeps the same IDs for the same schemas, existing records stay readable without rewriting topics. The migration therefore centers on preserving IDs, not on converting data. Verify this with a sample of old records from each topic before cutover, and confirm that the target registry returns the same schema for each ID. Re-serialization becomes necessary only when a schema must change in an incompatible way or when IDs cannot be preserved, and that decision belongs in the data contract review, not in the migration itself.

What should we ask for in a written quote?

Ask for a scope that names every subject set, client application and environment included, the export and ID-preservation method, the verification steps with their acceptance criteria, and the cutover and rollback plan. Request a breakdown that separates one-time migration work from recurring operations such as upgrades, monitoring and support, and that states what is excluded, such as changes to client code owned by other teams. Ask how the partner will handle schemas whose owners cannot be found, and what documentation and runbooks you will receive at handover. Quotes should follow from this written scope rather than from a general estimate.

How DeployOpen works on this

DeployOpen approaches a move from Confluent Schema Registry to Apicurio or Karapace as a planned platform change inside your own infrastructure. We start by inventorying subjects, versions, serializers and consumers, then agree the target registry, compatibility settings and cutover approach in writing before any work begins.

How we scope

Written scope lists the subjects, contexts, serializer clients and consumer groups that the migration covers.
The scope states which target registry is chosen, and why, based on the Apicurio or Karapace compatibility review.
Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
Ask for the quote to name each client language and serializer library that must be switched.

How we build

We export subjects and versions and verify that every schema ID referenced by existing records resolves in the target.
We rebuild compatibility rules per subject and test them against sample producer and consumer versions.
We switch clients in a staged order, with rollback steps defined for each serializer configuration change.
Test results, configuration and migration scripts are documented as deliverables in your repositories.

How we hand over

Handover covers runbooks for registry deployment, backup, access control and schema change review.
Your team receives a record of each cutover step, the verification results and open items.
We document which client applications still point at the old registry and when they were retired.
Knowledge transfer sessions walk your engineers through the registry API, rules and failure modes.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after handover.
Retainers can cover registry version upgrades, compatibility rule reviews and new subject onboarding.
Confirm with us during scoping which post-migration support level fits your change calendar.
Security testing of the registry endpoints can be added to a retainer or scoped separately.

Migrating from Confluent Schema Registry to Apicurio or Karapace