When to finalize the Kafka metadata version after a KRaft upgrade
Finalize only after the cluster has run stably on the new binaries, and only once you have verified that brokers, clients and monitoring behave as expected. The finalize step is bin/kafka-features.sh upgrade --release-version followed by the target version. Until you run it, the cluster keeps its previous feature level, which is what gives you a clean way back. In KRaft mode, Kafka controls the feature level through metadata.version, managed with bin/kafka-features.sh, rather than through inter.broker.protocol.version used in ZooKeeper-era clusters. So the decision is about the metadata version, not the broker protocol setting you may remember from earlier upgrades.
The supported path is a rolling upgrade from each of the last three releases to the latest version, as described in the Apache Kafka time based release plan. Upgrade brokers one at a time, watch them for a soak period you define, and only then finalize. Confirm the exact sequence for your starting release in the Apache Kafka 4.3 upgrade notes before you plan the window, because the notes govern what each release requires.
Before you finalize, agree on the checks that will count as stable: broker health and under-replicated partitions, controller behavior, producer and consumer error rates, and the client versions you inventoried earlier. Record who has the authority to run the finalize command and what would stop it. Treat the finalize step as a deliberate change with its own change record, because once it runs, some downgrades are no longer possible without a restore.
What changes in the 4.x release line and what each change means for finalizing
This table lists the 4.x changes that affect a KRaft upgrade and the final metadata version step. Read each row as a question to answer before you run the finalize command on your own cluster. Feature-level details for each release belong in the upgrade notes for that version, so confirm them there before you set a target.
| Change | Impact on finalizing | Action needed |
|---|---|---|
| Supported releases are 4.3.1, 4.2.2 and 4.1.2 | Your current and target releases determine which metadata version you can finalize to and how long a rolling path stays supported. | Confirm which release you are on and pick a supported target from the downloads page before you start. |
| KRaft-only operation since 4.0 | There is no ZooKeeper fallback. Finalizing is the point where the cluster commits to the new feature level, so a KRaft cluster must already be healthy before you begin. | Confirm the cluster runs in KRaft mode and that controllers and brokers are all on the new binaries before finalizing. |
| Rolling upgrades from the last three releases to the latest | The time-based plan defines which starting versions can roll to the latest release, which sets where your rollback window begins. | Check the release plan for your starting version and note the version you would return to if you roll back. |
| Java 17 for brokers and Connect | Mixed Java runtimes during the rollout can complicate troubleshooting, and a runtime problem may look like a feature problem after finalizing. | Run brokers and Connect workers on Java 17 before the rollout and keep runtime versions recorded per node. |
| Java 11 minimum for clients and Streams | Clients on older runtimes still work with the cluster, but they may not reflect the feature level you enable at finalizing. | Inventory client and Streams runtimes and test them against the upgraded cluster before finalizing. |
| metadata.version controls the feature level | Finalizing sets the cluster feature level through kafka-features rather than inter.broker.protocol.version, so this is the setting you must verify. | Run bin/kafka-features.sh to view the current level and confirm the target before you run the upgrade command. |
| Feature-level details per release | Which features each metadata version enables, and which can be reversed, varies by release and is not set out in this table. | Confirm the feature list and downgrade limits for your target release in its upgrade notes before scheduling the finalize step. |
What finalizing the metadata version enables and why it is one-way
Finalizing a KRaft upgrade turns on the new metadata version for the whole cluster. In KRaft mode, Kafka uses metadata.version to set the feature level that every broker and controller agrees on, and it manages that level with bin/kafka-features.sh, not with inter.broker.protocol.version. Until you run kafka-features.sh upgrade --release-version, the cluster can run mixed binaries from the old and new release, which is what makes the rolling broker upgrade safe. Once the metadata version is raised, the cluster begins to rely on behavior that the older binaries do not implement, so the old binaries stop being a valid target state. Treat finalization as a change to the cluster's feature level, not as a routine restart step.
The features gated by each metadata version are release-specific. Do not assume the list from a previous release or from memory. Before you finalize, read the upgrade notes for your target release and list every change that the notes tie to the new feature level, including any configuration default that changes once the level is active. Confirm with the project documentation and your own test cluster which of those changes can be reversed and by which tooling. The reversibility of a given feature is not a property you can infer from the fact that a rolling upgrade is supported. Record the answer for each feature in your change plan, and state clearly where you could not confirm it.
Plan the rollback window around that one-way point. Before you run the finalize command, decide how long you will keep the old binaries available, who can approve a rollback, and which checks must pass during the soak period, such as client error rates, consumer lag and controller health. Use a short illustrative schedule only as a planning aid, and set real durations from your own change calendar. After finalization, a rollback means reverting binaries and, where the notes allow it, the metadata version, so confirm that path on a staging cluster built from the same release before production.
Which steps can be reversed after finalizing and which cannot
Until you run the finalize step, the metadata version still reflects the previous release, so the cluster can keep operating with the earlier binaries. The Kafka upgrade notes describe upgrading brokers one at a time, which means each broker can be returned to the previous release in the same broker-by-broker sequence while the feature level has not been raised. This is the stage where a rollback is simplest and carries the least risk to stored data.
- Broker binaries: Roll each broker back one at a time and confirm cluster health before moving to the next broker.
- Controllers: Handle controller nodes with the same care as brokers and confirm the rollback order in the upgrade notes for your release.
- Client traffic: Confirm that clients still work against the older broker version before you revert the last broker.
- Configuration: Keep a copy of the previous server configuration and any settings you changed for the new release.
Check that the rollback sequence was tested in a non-production cluster before the production rollout starts, and that the person running it knows when to stop.
Running the feature upgrade with kafka-features raises the metadata version, which sets the feature level the cluster operates at. Once that change is committed, you cannot assume that returning the binaries alone restores the earlier state. Feature-level downgrade is limited and depends on each feature, so the target release notes in the Apache Kafka 4.3 upgrade notes are the authority for what you can return to.
- Binary rollback alone: Do not assume an older binary can run against a metadata version it does not understand. Confirm this in the notes before relying on it.
- Feature-level downgrade: Check whether each feature you enabled lists a downgrade path for the release you plan to return to.
- No assumed matrix: The facts available for this page do not include a full downgrade matrix, so each feature needs its own confirmation against the release notes and your test results.
- Supported paths: Rolling upgrades are supported from each of the last three releases to the latest version, so a return to an older release outside that window should be confirmed with the project before you depend on it.
Record the metadata version before and after finalization so that any rollback decision is based on the cluster's recorded state rather than memory.
The hardest rollback is the one where something already depends on the new behavior. Once clients, Connect workers, Streams applications or stored data use a newly enabled capability, reverting the cluster can break those workloads, even when a broker downgrade is technically possible. This risk grows with every day the cluster runs at the new feature level, so the time between finalization and confirmation should be short and deliberate.
- Client dependence: Identify which client libraries and versions use newer protocol behavior before you finalize, and record them in your inventory.
- Stored data: Check whether records or metadata written under the new feature level can still be read by the older release.
- Connect and Streams: Confirm that connector plugins and Streams applications run on the release you would roll back to, especially if their upgrades were completed in the same window.
- Operational tooling: Scripts, dashboards and admin commands built against new output or flags may need reverting along with the cluster.
Before you finalize, list every workload that touches the cluster and mark which ones would block a rollback, so the team can decide whether the rollback window is acceptable.
What to check before running kafka-features upgrade
Finalizing is the point of no easy return for a KRaft upgrade, so the checks below should pass before anyone runs the command. Each card covers one check, and together they confirm that the cluster runs the new binaries, behaves normally, and can be restored if the release proves faulty.
Verified rolling restart
Confirm that every broker has been restarted onto the new binaries, one at a time, and that no broker is still running the old version. Check that each broker has rejoined the cluster and that under-replicated partitions have cleared before continuing. Finalizing while a broker still runs old code means the rollout is incomplete, so treat any leftover older process as a blocker.
Controller and broker health
Check that the controller quorum has a leader, that all controllers are reachable, and that brokers report healthy metadata state after the rollout. Review logs for repeated leader elections, fenced brokers or metadata replay errors. A cluster that is unstable on the new binaries will not become stable because the feature level is raised, so confirm health before the change.
Client and Connect compatibility
Confirm that clients, Kafka Streams applications and Connect workers meet the Java versions the target release requires: Java 11 for clients and Streams, Java 17 for brokers and Connect. Inventory client libraries still in use and check them against the target release notes. Finalizing enables features that older clients may not expect, so an incompatible client found after the change is harder to roll back.
Backups and topic configuration exports
Export topic configurations, ACLs, client quotas and broker configuration files before the change, and store them outside the cluster. Confirm that the backup restores in a test environment. Metadata is held in the cluster itself, so a dated export gives the team a reference point if a configuration or feature question arises after finalizing.
Staging rehearsal of the finalize command
Run the same sequence in a staging cluster built from the same release, with comparable topics, ACLs and client traffic. Practice the bin/kafka-features.sh upgrade command with the exact release version you plan to use, and record the output and any warnings. The rehearsal should show whether workloads and automation behave as expected before production is changed.
Documented go or no-go owner
Name one person who makes the final go or no-go decision, and write down the criteria each check must meet. Record who approved the change, the time it was run and the rollback window that applies afterward. Without a named owner, a failed check can be overridden informally, which is the most common reason finalizing happens too early.
The finalization sequence with checks and rollback points
The sequence moves a KRaft cluster from the old metadata version to the target release in a fixed order, with a check at each step and a clear point where rolling back the binaries stops being an option.
Confirm every broker runs the target binaries
The platform team checks that every broker and controller in the cluster runs the new Kafka binaries after the rolling restart, and that no node is still on the previous release. Compare the running version reported by each node against the release you planned, and confirm that the upgrade was done one broker at a time as the upgrade notes describe. Rollback to the previous binaries is still possible at this point, because the metadata version has not changed. Done when every node reports the target version and the rollout log shows no node left behind.
Observe cluster behavior over the agreed period
Operators run the cluster on the new binaries for an observation period agreed in advance with the application owners. Watch under-replicated partitions, controller health, producer error rates, consumer lag and client connection errors against the baselines recorded before the upgrade. The length of the period is a decision for your team, so confirm the expectation with your change process rather than copying a figure. Rollback to the previous binaries is still possible during this step, although any problem found here may need a broker-by-broker reversal. Done when the agreed period ends with metrics inside the baseline and no open incidents tied to the upgrade.
Run the feature upgrade to the target release
Once the observation period is accepted, an operator runs bin/kafka-features.sh upgrade with --release-version set to the target release. This command writes the new metadata version into the cluster metadata log, which every broker then uses to enable the features of that release. Confirm the exact flags and the release-version value against the Kafka 4.3 upgrading notes before running it, and run the command from a host with access to the bootstrap servers. Done when the command returns without error and the controllers accept the change. From this point, rollback to the previous binaries is no longer simply a restart, so treat this as the commit point.
Describe the cluster to confirm the metadata version
Run bin/kafka-features.sh describe against the cluster and check that the finalized metadata version matches the target release. Record the output in the change record along with the feature levels reported for the cluster. Also verify that all brokers and controllers report the same feature state, since a node that did not pick up the change needs attention before anything else. Rollback to the previous binaries should not be attempted now without the downgrade procedure for the features you enabled, so the rollback decision is made with this output in hand. Done when describe shows the target metadata version and the change record is complete.
Verify clients and downstream systems after finalization
Application teams and platform operators confirm that producers, consumers, Kafka Connect workers and Kafka Streams applications work against the finalized cluster. Use the client compatibility inventory from before the upgrade and check the error logs and dashboards for the first full processing cycle after the change. Any client that depends on a feature gated by the new metadata version should be tested in this window. Done when each application owner signs off in the change record and no client errors remain above the agreed baseline.
Close the rollback window and record the decision
After the verification is accepted, the change owner formally closes the rollback window in the change record and notes which downgrade options remain for each feature. Confirm with the vendor documentation which features can be downgraded and which cannot before closing, because this list determines what a later rollback can undo. Update runbooks so that the next operator knows the metadata version now in use and where the rollback limits are documented. Done when the record states the finalized version, the signed-off checks and the remaining rollback limits.
What makes finalizing and rollback planning take longer
This table lists the factors that stretch the time between broker upgrades and the metadata version finalization, and the questions to answer before scheduling it. Read each row as a question to settle with your team before setting a rollback window. The main takeaway is that finalization is gated by the slowest-to-verify part of your estate, not by the Kafka command itself.
| Driver | Why it matters | What to ask |
|---|---|---|
| Client fleet size and unknown client versions | Finalizing enables the new feature level for the whole cluster, so any client that breaks only after finalization can force a rollback decision. Clients you have not inventoried are the most common source of delay. | Which client libraries and versions connect to each cluster, and who owns each one? Which of them have been tested against the target release? |
| Number of Kafka Connect workers and plugins | Connect workers run their own JVMs and plugin sets, and each plugin may depend on client behavior that changes between releases. Each worker group needs its own check before the cluster is finalized. | How many Connect clusters exist, which connector plugins and versions do they load, and who can test each connector against the upgraded brokers? |
| Observation period required by the business | Finalization is the point of no simple return for most features, so the window that stakeholders require before accepting the new level sets how long the rollback option stays open. | What observation period has the business approved, what signals end it early, and who signs off the decision to finalize? |
| Number of clusters and MirrorMaker 2 replication | Each cluster is finalized separately, and MirrorMaker 2 links can carry data and offsets between clusters running different versions. Sequencing across sites adds coordination and retesting. | How many clusters are involved, which MirrorMaker 2 flows connect them, and in what order will each cluster be finalized and verified? |
| Security configuration that must be re-verified | SASL mechanisms, TLS listeners and ACLs stored in the metadata log must keep working after the upgrade. Authorization changes are easy to miss when only throughput is checked. | Which SASL mechanisms, listeners and ACL sets are in use? Who will confirm that clients still authenticate and that ACL-protected resources remain accessible before finalizing? |
| Change freeze windows | Finalization should not overlap with other planned changes, because a problem found afterward is hard to attribute. Freeze calendars often limit the usable days for a rollback window. | Which freeze periods apply to the teams that own producers, consumers and Connect workers, and how do they affect the earliest and latest finalization dates? |
| Rollback tooling and runbook readiness | A downgrade path exists only for features the release documents as downgradable. Operators need a tested procedure and clear limits before they can commit to a window. | Has the team confirmed in the Kafka 4.3 upgrade notes which features can be downgraded, and has the rollback procedure been rehearsed in a non-production cluster? |
Questions teams ask before finalizing a Kafka cluster
How long does finalizing a Kafka metadata version take?
Finalizing itself is a single command, bin/kafka-features.sh upgrade --release-version, run after every broker has been upgraded and the cluster has been verified. The time you spend is driven by the verification window before that command, not by the command. Teams usually plan for the length of their monitoring period, the number of brokers and clients to check, and how many change windows they need. The upgrade notes for the target release describe the sequence, and your own soak period, measured against your traffic and alerting, will set the real duration. Ask any partner to state the verification steps and exit criteria in writing before quoting time.
Can our own team finalize a Kafka cluster without outside help?
Yes, if your team already runs Kafka in KRaft mode and can follow the upgrade notes for the release you are moving to. The finalization step is a documented command, so the technical barrier is low. The harder part is deciding when the cluster is ready, which requires client inventory, metrics review and a plan for what happens if something fails. Many teams use an outside partner for the planning, rehearsal and evidence, and keep the change itself in-house. Either way, the decision should be recorded and owned by a named person.
What does the work need from our team?
Expect to supply access to the cluster and its change process, a current inventory of brokers, controllers, Kafka Connect workers and client applications with their versions, and the people who own each client. You also need monitoring dashboards and alerts that show broker health, under-replicated partitions, request errors and consumer lag. Someone must be able to approve the finalization window and make the call to stop. Ask for these inputs to be listed in the written scope so both sides know what is required before work starts.
Can a finalized Kafka cluster go back to the old release?
Not in general. Once the metadata version is finalized, some features are enabled in the cluster metadata, and the upgrade notes describe which of those can be downgraded and which cannot. Rolling back broker binaries before finalization is a different case, and it is limited by the rolling upgrade path your target release supports. Plan the rollback window as the time between upgrading brokers and finalizing, and confirm the exact downgrade rules for your versions in the upgrade notes before you set that window.
What does finalizing cost, and how is a quote built?
DeployOpen does not publish a price list. A quote follows from a written scope that lists the clusters, environments, client inventory, verification steps, rehearsal runs and deliverables. The main cost drivers are the number of clusters and brokers, how many client applications must be checked, the depth of rehearsal and the amount of evidence you need afterward. Cost recurs only if you add ongoing support or testing. Ask for the quote to separate planning, execution and any follow-up work, so you can compare it with the effort of doing the same steps internally.
What should we do if a client breaks after finalizing?
Start by identifying which client, version and feature is involved, then check the upgrade notes for that release for known client compatibility limits. If the break is caused by a feature enabled at finalization, the rollback options are limited, so your response plan must be agreed before the change. Typical steps are to pause the affected rollout, isolate the client, restore service through a client-side fix or version change, and record the incident. Make sure the runbook names who decides, and test the response in a non-production cluster first.
Which Kafka release notes should we read before finalizing?
Read the upgrade notes for the target release first, because they describe the supported upgrade path, the broker-by-broker sequence and the finalization command. The Kafka time-based release plan explains which earlier releases support rolling upgrades to the latest version. Use these documents as the basis for your change plan, then add your own environment checks on top of them.
How DeployOpen scopes, rehearses and supports a Kafka metadata finalization
DeployOpen treats metadata version finalization as a gated change inside a written scope, not as a single command. We agree the target release, the verification evidence and the rollback window with you before any broker moves, then rehearse the sequence in a non-production environment that matches your cluster. Engagements are fixed-scope, and ongoing support is available as a retainer.
How we scope
- Confirm the current and target 4.x release, and the upgrade path your cluster supports.
- Inventory brokers, controllers, clients and Connect workers that must run the new version before finalizing.
- Agree in writing the finalization gate, the verification checks and the rollback window owner.
How we build
- Write the runbook for the broker-by-broker rollout and the kafka-features upgrade step.
- Rehearse finalization and the feature downgrade path in a staging cluster sized like production.
- Prepare monitoring checks for broker health, under-replicated partitions and client error rates.
How we hand over
- Deliver the runbook, rehearsal results and a record of the metadata version before and after.
- Document which features are irreversible after finalizing and the downgrade steps still available.
- Walk your operators through the go or no-go checklist and the rollback decision points.
How we support
- Support follow-up checks after finalization under the agreed scope or a retainer.
- Review release notes for each new 4.x version before planning the next upgrade window.
- Keep the rehearsal environment and runbook current as your cluster and release plan change.
