Which Kafka client compatibility gaps will break your applications on Kafka 4?
The gaps that break applications on Kafka 4 are most often in client versions, not only in the broker. Start by finding every client that talks to your self-hosted KRaft cluster, recording its library and version, and checking each pair against Kafka's compatibility reference before any broker moves. Clients from 2.1 to 2.8 are listed as partially compatible with Kafka 4.0, Kafka Streams and Kafka Connect from those versions have limited compatibility, and 3.x clients, Streams and Connect are listed as fully compatible. Read the Kafka 4.3 compatibility page for the exact ranges, because a client that works on your current brokers can still fail once the cluster runs Kafka 4.x.
The release plan decides which upgrade paths you can take. Rolling upgrades are supported from each release in the past year, which means the last three releases, to the latest version, according to the Kafka time-based release plan. If your clients or your cluster sit outside that window, the broker move is not a single step. Decide the intermediate versions first, then schedule client upgrades before the broker move so that no application is left running on a version the target release does not cover. Note any application that is pinned to an old library because its vendor has not released a newer build.
Build the inventory from the systems that already know who connects. Consumer group listings, topic ACL entries and the service accounts in your identity provider usually show more applications than the architecture diagram does. For each application, record the client library and version, the language runtime, and whether it is a plain producer or consumer, a Kafka Streams application or a Kafka Connect worker. Kafka 4.0 requires Java 17 for brokers and Connect, and at least Java 11 for clients and Streams applications, so runtime versions belong in the same record. Confirm the library mapping for any non-Java client, such as one built on librdkafka, with its maintainer and with your own tests rather than assuming it matches the Java client line.
What Kafka 4 changes for client compatibility, Streams and Connect
This table compares each client, Streams and Connect version family against Kafka 4, plus the runtime and operating-mode changes that apply to every client. Read each row as a question: does this component need an upgrade, a runtime change, or a test pass before the cutover. The main takeaway is that 2.1 to 2.8 clients and Streams or Connect applications carry the most work, while 3.x components mainly need the Java and runtime checks.
| Change | Impact | Action needed |
|---|---|---|
| 2.1 to 2.8 clients | Partially compatible with Kafka 4.0, so some requests or features may not behave as they did before the upgrade. | Upgrade the client library to a 3.x or later version, then test producers and consumers against a staging Kafka 4 cluster before cutover. |
| Streams and Connect from 2.1 to 2.8 | Limited compatibility with Kafka 4.0, so these applications and workers may fail or need changes to run correctly. | Upgrade the Streams or Connect runtime first, then test each application or connector before moving production traffic. |
| 3.x clients | Fully compatible with Kafka 4.0 according to the compatibility page. | Keep the client on its current version, confirm the Java runtime meets the minimum, and run a functional test before cutover. |
| 3.x Streams and Connect | Fully compatible with Kafka 4.0 according to the compatibility page. | Confirm the Java runtime, then test connector plugins and Streams topologies in staging before cutover. |
| Java 11 minimum for clients and Streams | Client and Streams applications running on Java 8 or earlier cannot start on Kafka 4 runtimes without changes. | Inventory the JVM version of every application, and upgrade any runtime below Java 11 before the client upgrade. |
| Java 17 for brokers and Connect | Brokers and Kafka Connect workers must run on Java 17, which affects host images and container base images. | Upgrade the broker and Connect worker JVMs to Java 17 as part of the cluster change, and check plugin compatibility with the new runtime. |
| KRaft-only operation with ZooKeeper removed | Clients that depend on ZooKeeper-era tooling or connection settings have no supported path on Kafka 4. | Confirm the cluster already runs KRaft, and replace any ZooKeeper connection strings in client and tooling configuration. |
Client compatibility by version range: 0.x to 2.0, 2.1 to 2.8 and 3.x
This is the oldest range in most inventories, and it is the one this page cannot settle for you. The Kafka compatibility page is the authority on whether clients from these releases work with Kafka 4.x, so check the status for each exact version range there rather than relying on a summary. Treat any client in this range as unverified until you have checked it against the page and tested it in a non-production environment.
- Look up the exact release: Record the precise client version each application uses, not just the major line, because the compatibility status is listed by version range.
- Check the broker protocol: Confirm which protocol versions the page says Kafka 4.x still accepts, since some older protocol versions may have been removed.
- Plan for upgrades, not patches: If a client in this range is unsupported or untested, the usual path is to upgrade the client library to a later supported version and retest.
- Note the language runtime: Clients and Kafka Streams applications need at least Java 11 since Kafka 4.0, so an old client on an old runtime may need both changes.
- Record the decision: Keep a dated note of the status you confirmed for each range so auditors and owners can see what was checked.
Before moving on, confirm the status for 0.x to 2.0 clients directly on the compatibility page for the Kafka version you plan to run.
The compatibility page lists clients from 2.1 to 2.8 as partially compatible with Kafka 4.0. In practice this means some features or behaviors may not work as they did before, and the page does not promise that every application in this range will fail. Partial compatibility is a reason to test and plan upgrades, not a verdict that the application is broken. Each team should identify which of its clients fall in this range and schedule a test run against the target Kafka version.
- Test the features you use: Run producer, consumer and admin paths your applications rely on, since partial compatibility usually depends on which features are exercised.
- Plan a client upgrade: Where testing shows gaps, move the client to a later version inside the 3.x range or the version the compatibility page recommends.
- Check Streams and Connect separately: The page lists Streams and Connect from those versions as having limited compatibility, so these workloads need their own test plan.
- Sequence the work: Coordinate client upgrades with the broker rollout so that no application is left on an untested combination longer than necessary.
- Keep a rollback path: Confirm that the older client build can still be redeployed if a test uncovers a problem the team cannot fix in time.
Before the upgrade window, confirm which applications in this range you have tested and which still need a test run.
The compatibility page lists 3.x clients, Streams and Connect as fully compatible with Kafka 4.x. This is the range where the compatibility risk is lowest, so it usually needs the lightest testing. Full compatibility still does not mean zero work. Your own configuration, security settings and custom code can still behave differently, so a short verification run is worth doing for each important application.
- Confirm the exact version: Check that each application uses a 3.x client, including Streams and Connect plugins that bundle their own Kafka libraries.
- Verify configuration keys: Review producer, consumer and security settings for properties that have been renamed or removed, and confirm them against the upgrade notes.
- Run a smoke test: Produce and consume representative messages in a staging environment that matches your production security setup.
- Check framework bundles: Frameworks and connectors may pin their own client libraries, so confirm the version that actually runs in production.
- Include Connect workers: Connect and Streams components are listed as fully compatible, but confirm the Java runtime, since brokers and Connect require Java 17 from Kafka 4.0.
Before you rely on a 3.x client, confirm its bundled library version and run a smoke test in an environment that mirrors production.
How to inventory client versions before the upgrade
Client versions rarely sit in one place. The fastest inventory combines what your repositories declare, what your clusters run, and what your owners say they operate. Each source below catches clients the others miss, and the gaps between them are where upgrade surprises usually come from.
Build manifests and dependency files
Start with the declared client library version in each service's build file, such as Maven POMs, Gradle files, requirements files, or package manifests. Search every repository for the Kafka client artifacts and any wrapper libraries that pull them in transitively. Record the version each service builds against, because that is the client code it ships. Note services that share a parent BOM, since one change there can move many services at once.
Container images and deployment specs
A service's build file does not always match the image it runs. Check the base images and the packaged application archives in your container registry, and the image tags referenced in Helm charts, Kubernetes manifests, and deployment pipelines. Look for old images still pinned in staging or disaster recovery environments. Confirm which tags are actually deployed in each environment rather than assuming the latest build is running.
Runtime and JVM versions
Kafka 4.0 requires at least Java 11 for clients and Kafka Streams applications, so every JVM-based client needs a version check. Record the Java runtime each service starts with, not only the one in its build file, because a runtime can be upgraded independently of the code. Flag any service still on an older runtime, since it needs a runtime change before the client library change can ship.
Streams and Connect deployments
Kafka Streams applications and Kafka Connect workers are often run by a different team than the services that produce and consume messages. Inventory each Streams application and Connect cluster separately, including the connector plugins they load. Connect workers and Streams apps need their own runtime checks, and the Streams and Connect compatibility rows differ from plain client rows. Mark any component whose owner you cannot name.
Client libraries embedded in frameworks
Frameworks, data tools, and vendor products often bundle a Kafka client inside their own artifact, so the version you care about may not appear in your code at all. Check the shaded or bundled JARs inside framework distributions, third-party connectors, and commercial agents that talk to Kafka. For non-JVM stacks, check the librdkafka or other native client version inside each application's dependencies. Ask each vendor which client version their current release bundles.
Owner register and broker visibility
Create a register with one row per client application, listing its owner, its environment, its client and runtime versions, and its upgrade window. The register is the record your change plan depends on, so review it with each team. For broker-side evidence, confirm what your cluster's request metrics and broker logs expose in your version, such as client identifiers or software version fields, rather than assuming a specific field is present. Where broker data is thin, use the register as the source of truth and close gaps with the owners.
Why Streams, Connect and librdkafka clients need their own check
A plain Java producer or consumer check does not clear Kafka Streams applications or Kafka Connect workers. Kafka's compatibility page lists 2.1 to 2.8 clients as partially compatible with Kafka 4.0, and it describes Streams and Connect from those versions as having limited compatibility. That means a Streams or Connect application on a 2.x line needs its own row in the inventory, with its own test, even when the underlying Kafka client jar looks acceptable. Streams and Connect also carry their own Java floor: the upgrade notes state that Kafka Streams applications need at least Java 11 and that Connect workers need Java 17, so the runtime of each application and worker must be checked separately from the broker and client libraries.
Clients built on librdkafka, and frameworks that embed a Kafka client, follow their own release cycles. Your inventory should record the library or framework name, the version in each service, the Kafka protocol features that version uses, and the team or vendor who owns the upgrade. Confirm each of these with the library or framework maintainers, since their release notes, not the Kafka project, state which protocol versions and removed features they depend on. A framework that bundles an old client may hide the version from application developers, so check the lock files, container images and dependency manifests rather than asking teams from memory.
List every Streams application, Connect worker, librdkafka-based client and framework-embedded client with its owner, its runtime, its Kafka client version and the maintainer's upgrade path. Assign each entry a named owner who will run the test against a Kafka 4.x test cluster. Components without an owner or without a confirmed compatibility answer should block the upgrade window until someone resolves them, because a single unowned consumer can stall a whole topic family after the brokers have moved.
The sequence for clearing client blockers before the broker upgrade
This sequence turns a client inventory into a documented go or no-go decision, so every application that cannot run against Kafka 4 is upgraded, replaced or consciously accepted before any broker moves. Each step closes with a check and keeps a fallback in place.
Freeze and sign off the client inventory
The platform team freezes the inventory produced from broker request logs and application manifests, then the application owners sign it off. Every client identifier, application, team and owner should appear once, with an explicit note for anything that connects from outside the network or only runs in batch. The step is done when each owner has signed the inventory and the platform team has confirmed that no client IP or client ID seen in the logs is unaccounted for. Keep the frozen copy under version control so later changes are visible.
Classify each client by version and runtime
For each inventory entry, the owner records the client library, its exact version, the language runtime and the framework that wraps it, such as Kafka Streams, Kafka Connect or a librdkafka-based binding. Compare each entry against Kafka's compatibility page, where 3.x clients, Streams and Connect are listed as fully compatible, 2.1 to 2.8 clients as partially compatible, and older clients need separate review. Mark each entry as compatible, needs upgrade, needs runtime change or needs replacement. The output is a classified list where no entry is left as unknown.
Upgrade runtimes to Java 11 or 17
Teams that run Kafka Streams applications or Java clients on runtimes below the required level move them first, because a client upgrade on an unsupported JVM will not help. The upgrade notes state that clients and Streams applications need at least Java 11, and brokers and Connect need Java 17. Owners rebuild the artifact against the new runtime and run the existing unit and integration tests. The step closes when each runtime change has a passing build and the previous runtime image is still available for rollback.
Upgrade or replace clients on the 3.x path
Owners move 2.x clients to a 3.x client library, or replace clients that cannot be upgraded with a supported alternative, and record the new version in the inventory. Connect and Streams components are upgraded in the same pass so their embedded Kafka libraries match the target. Where a library cannot change, the owner documents the risk and the required workaround. The step is complete when every entry in the inventory shows a version on the confirmed compatible path. Keep the previous build deployable until the broker upgrade is finalized, so a client rollback does not require a rebuild.
Test each application against a Kafka 4 test cluster
Each application runs its producer, consumer and transactional paths against a non-production Kafka 4 cluster that uses the same authentication, TLS settings and ACLs as production. Tests cover topic creation rules, consumer group behavior and any Streams topology or Connect worker that the application depends on. The owner records failures with the client version, the Kafka error and the fix. The step is done when each application passes its test suite on Kafka 4 and the results are attached to its inventory entry.
Canary the upgraded clients in production
Owners deploy the upgraded clients to a small share of production traffic or to a single consumer group, while the rest of the workload stays on the existing build. The platform team watches client error rates, consumer lag and rebalance activity against the baseline from the current cluster. The owner rolls back the canary to the previous build if errors or lag rise. The step closes when the canary has run through a full business cycle that the owner has agreed in advance, with the results recorded.
Hold the go or no-go decision before broker rollout
The platform owner, application owners and security lead review the signed inventory, the test results and the canary report in one meeting. Each application is marked go, go with a named exception, or no-go, and no-go entries block the broker rollout for their cluster. The decision and any accepted risk are recorded in writing with the approver's name. The step is complete when the broker upgrade plan lists only clients marked go, and each previous client build has a documented path to remain deployable until the broker upgrade is finalized.
What makes client remediation take longer
This table lists the factors that expand the work of fixing clients before a Kafka 4 upgrade and what to ask for in a quote. Read each row as a question to answer about your own estate: the more a driver applies, the more discovery, code change and testing time to plan for.
| Driver | Why it adds effort | What to ask for in a quote |
|---|---|---|
| Number of unowned clients | Clients without a named owner must be found, traced to a team and triaged before anyone can decide whether to upgrade, replace or retire them. | A discovery phase with a named inventory output and an owner assignment step, priced separately from code changes. |
| Streams applications with state | Stateful applications need careful state store handling, changelog checks and rebalance testing, so each one takes more verification than a stateless client. | Per-application effort split between stateless and stateful workloads, and the test scenarios covering restarts and rebalances. |
| Connect plugins and connectors | Plugins are often third-party builds with their own release cycles, so compatibility depends on vendor support as well as on the Connect version in use. | A list of each plugin with its version, source and vendor support status, plus the work to replace any plugin with no Kafka 4 path. |
| Java runtime upgrades across many services | Clients and Streams applications need at least Java 11, and brokers and Connect need Java 17, so every JVM service may need a runtime change, base image update and retest. | Estimate per service for the runtime change, including container images, build tooling and deployment pipeline updates. |
| Third-party or vendor-built clients that cannot be changed | When the source is not owned, the only options are a vendor upgrade, a configuration workaround or replacement, and each depends on the vendor's timeline. | Written confirmation from each vendor of supported Kafka versions, and a costed fallback plan for clients the vendor will not update. |
| librdkafka and framework dependencies | Non-Java clients built on librdkafka or on application frameworks inherit library versions, so a fix can depend on a library release and a framework upgrade at the same time. | Dependency map for each client showing library and framework versions, and the upgrade work for any client that needs a newer base library. |
| Test environment available | Without a Kafka 4 test cluster that mirrors production topics, ACLs and client traffic, each fix must be verified later in production, which lengthens rollback planning. | Whether the quote includes provisioning a non-production cluster, replaying representative traffic and running client regression tests. |
Testing client behavior and planning rollback when clients lag
A client test only tells you something about Kafka 4 if it runs against a Kafka 4 cluster configured the way production is configured. Build the test cluster with the same listeners, SSL or SASL mechanisms, and authorizer settings that your producers, consumers, Streams applications and Connect workers use today. If production relies on ACLs, reproduce them on the test cluster and confirm that the test principals are allowed exactly what they need. In KRaft clusters the StandardAuthorizer keeps ACLs in the metadata log, and any resource without an ACL is limited to super users unless allow.everyone.if.no.acl.found is set to true. A client that works on an open test cluster can fail on a secured one, so an unsecured test proves little about authentication or authorization. Record the client version, the Java runtime (clients and Streams applications need at least Java 11 on Kafka 4), and the exact configuration for each test so the results can be repeated after any change.
Some clients will not move in time. For those, the safest plan is to keep the cluster on a release that is still supported and to avoid stacking a broker upgrade on top of an unfinished client upgrade. The Kafka time-based release plan says rolling upgrades are supported from each release in the past year, that is the last three releases, to the latest version, so confirm your current release is inside that window before you schedule a broker move. Use the Kafka downloads page to check which releases are currently supported. Do not assume a client that lags can be kept indefinitely on a broker line that has left the supported set. Ask the client owners for a dated list of components that depend on older client libraries, and decide per component whether it will be upgraded, rebuilt against a newer library, or retired.
Before committing to any broker upgrade, read the finalizing and downgrade guidance for the target release. Rolling back a broker binary is not the same as reverting the metadata version. Once the metadata version is finalized, some features may not be downgradable, so a rollback that looked possible during the rolling upgrade may no longer be available. Write down the rollback limits for your exact source and target releases and confirm them with the release notes, not with this page. Make the decision on the written limits, and keep in mind that a client that has not been tested against the target release cannot be used as evidence that the cluster can safely move forward. Agree the go or no-go conditions with the client owners before the window opens.
Questions teams ask before checking client compatibility
How much effort does a client inventory and remediation take before a Kafka 4 upgrade?
The effort depends mostly on how many client applications exist, how many distinct client libraries and versions they use, and how much of that code the owning teams can change themselves. A clean inventory of a small platform can be a short exercise, while a large organization with many services built by different teams usually needs a structured survey, owner confirmation for every application, and a backlog of library upgrades. The work has two parts: finding every producer, consumer, Streams application and Connect worker, then deciding whether each one needs a library upgrade, a runtime upgrade to Java 11 or later, or a rebuild. Cost comes from the people who run the inventory, the engineers who upgrade client code, and the testing needed to confirm each change. Ask any partner to show how they will split inventory work, remediation work and retesting in a written scope before you agree a price.
How long does client compatibility work take?
The duration is driven by the number of clients, the number of owning teams, and how quickly those teams can release changes. Illustrative example: a platform with a handful of services owned by one team usually moves faster than one with dozens of services spread across many teams that each run their own release calendar. Teams should expect the inventory to finish before remediation can be planned, because nobody can size the backlog until every client and version is listed. Remediation then depends on each owner's release schedule and on test runs against a non-production cluster. Ask for a timeline that lists each client group, its owner and the test window it needs, rather than a single total.
What does the internal team need to provide for this work?
The internal team needs to provide four things: a named owner for every client application, access to the source repositories or dependency manifests that show library versions, a non-production Kafka environment where clients can be tested against Kafka 4, and agreed change windows for production client releases. Owners matter most, because a client without an owner cannot be upgraded or retired. Your team also decides which clients are in scope for the upgrade and which are retired, and approves the changes. Confirm with your platform team which Kafka version the test cluster runs and how long it can stay available for the test runs.
Can our own team do this, and when does a partner help?
Yes, most teams can run a client inventory themselves, especially when they already track dependencies and can read build files. Internal teams often struggle with three areas: finding clients that are not in a source repository, such as scripts, legacy batch jobs and third-party tools; mapping library versions to the compatibility guidance for each Kafka version; and running consistent test suites across many services. A partner helps most when the inventory crosses many teams, when the test environment needs to be built or hardened first, or when your own staff are needed for production work. Compare the cost of pulling staff from other projects against a fixed-scope engagement with a written deliverable list.
Can Kafka 4 run clients that are older than 2.1?
Clients older than the versions Kafka 4 supports fully may fail, so each one must be checked before cutover. Kafka's compatibility page describes 2.1 to 2.8 clients as partially compatible with Kafka 4.0, and it describes Streams and Connect from those versions as having limited compatibility. Clients from 3.x are listed as fully compatible. Older protocol versions were removed in the 4.0 line, so a client that depends on them will not connect. Read the compatibility page for the version-by-version detail, and test each client against a Kafka 4 test cluster before you rely on it. Plan a fallback for any client that cannot be upgraded in time.
How does DeployOpen scope a client compatibility engagement?
DeployOpen scopes this work as a fixed-scope engagement where the client inventory, the list of clients to upgrade, the test plan and the deliverables are written down before work starts. DeployOpen works inside your own infrastructure and does not issue audit reports or certifications. A typical scope covers the inventory method, a compatibility assessment for each client group, a remediation plan with owners, test evidence from the non-production environment, and a cutover checklist. The scope should name which clients are excluded, who signs off each change, and how a rollback is handled. Ask for a quote based on that written scope.
How DeployOpen scopes and delivers the client compatibility work
DeployOpen approaches client compatibility as a written, fixed-scope piece of work that produces an inventory, a compatibility decision for each client, and a remediation plan before any broker changes. The work runs inside your own infrastructure and ends with documentation your teams can keep using after the engagement closes.
How we scope
- Agree in writing which clusters, topics, client applications and frameworks fall inside the inventory
- Confirm which client libraries, Kafka Streams applications and Connect workers must be checked against the compatibility page
- Define the evidence to collect for each client, such as library version, runtime Java version and authentication method
- Set the output format and who signs off each compatibility decision before the scope is accepted
How we build
- Collect client and framework versions from application manifests, build files, container images and broker connection logs
- Map each client to the Kafka 4 compatibility position, flagging 2.1 to 2.8 clients and Streams or Connect versions with limited compatibility
- Test representative producers, consumers and Streams applications against a non-production Kafka 4 cluster you control
- Record failures, required library upgrades and runtime changes such as Java 11 or Java 17 requirements per component
How we hand over
- Deliver the client inventory as a versioned file with owners, versions, compatibility status and remediation actions
- Provide test scripts and configuration notes so your teams can repeat the checks for later Kafka 4.x releases
- Document the upgrade order for clients relative to brokers, with rollback considerations for each client group
- Walk through findings with application owners and record open items that remain with your teams
How we support
- Offer ongoing support and upgrade work as a retainer once the written scope is complete
- Re-run the client inventory when new Kafka releases or application versions are added to your estate
- Track vendor and framework changes that affect client compatibility and report them against your inventory
- Confirm retainer scope, response expectations and deliverables in writing before any ongoing work starts
