Mutual TLS client authentication in Apache Kafka

Mutual TLS client authentication in Apache Kafka: when teams need it

Mutual TLS (mTLS) authenticates Kafka clients with certificates when the broker is configured to require them, and the identity in that certificate then drives authorization. A team that must prove which application connects to its brokers, and feed that identity into ACLs, has a reason to use this path. The Kafka security overview lists SSL and SASL as the client authentication options, and it treats encryption as optional, so this page covers only the certificate identity path: a client presents a certificate, the broker verifies it against a trusted authority, and the resulting principal is what ACLs check.

Requested mode gives false comfort. Setting ssl.client.auth to requested asks clients for a certificate but still lets connections proceed without one, so an unauthenticated client can reach the listener and be treated as anonymous. Use required when certificate identity is the control you depend on. Confirm the setting on every secured listener, because a single permissive listener can bypass the controls on the others. Also confirm that the principal produced from the certificate subject matches what your ACLs expect, since a mismatch usually shows up as denied access rather than as an obvious misconfiguration.

Choose mTLS over SCRAM or Kerberos when your organization already runs a certificate authority it trusts for workload identity and can issue, rotate and revoke client certificates on a schedule. Plan the certificate lifecycle before rollout: who issues certificates, how long they last, how revocation reaches brokers, and how clients are renewed without downtime. Those operational costs recur for as long as the cluster runs, so ask your team or partner to estimate them in the written scope rather than treating mTLS as a one-time setup.

Client authentication modes and the settings that control them

This table compares the three values of ssl.client.auth (required, requested and none) and the related settings that decide who can connect and how a certificate becomes a Kafka principal. Read each row as a decision: the option, when it fits, and the risk it carries if it is the only control in place.

Setting or option	When it fits	Risk it carries
ssl.client.auth=required	Any listener that must accept only clients holding a certificate issued by your trusted CA, including inter-broker and production client listeners	Clients without a valid certificate fail the TLS handshake, so every client needs a certificate and rotation plan before the switch
ssl.client.auth=requested	Short migration windows where you are moving clients to certificates and need to observe which ones present them	Clients can connect without a certificate, so the broker still accepts unauthenticated sessions and this gives false comfort if left in place
ssl.client.auth=none	Listeners that authenticate with SASL instead of certificates, or encryption-only listeners on a restricted network	No client certificate check happens at the TLS layer, so identity depends entirely on the other mechanism enabled on that listener
ssl.principal.mapping.rules	When certificate distinguished names must map to short principal names that ACLs can reference cleanly	An overly broad rule can map many certificates to one principal, which weakens every ACL written against it
Truststore scope	When only certificates from your own internal CA, or a deliberately limited set of issuers, should be trusted	A shared or public-CA truststore lets any certificate from those issuers authenticate as a valid client
Default ACL behavior	Confirm your ACL plan before enabling authorization, since resources without ACLs are restricted to super users by default in KRaft	Setting allow.everyone.if.no.acl.found to true opens resources that have no ACL, which undoes mTLS-based least privilege
Revocation	Plan how a compromised client certificate is blocked, because a valid signed certificate is accepted until its ACLs or trust are changed	Kafka does not serve as your revocation authority by default, so rely on removing ACLs, rotating trust and reviewing your CA process

Setting up required client certificates on a KRaft cluster

This sequence moves a KRaft cluster from server-only TLS to required client certificates, so each client is identified by its certificate subject and can only do what its ACLs allow. Each step ends with a check, and the final test confirms that a client without a certificate is rejected while a valid client connects.

Confirm the certificate authority and its scope
Decide whether your organization already operates a certificate authority that can issue server and client certificates, or whether a private CA will be created for the Kafka cluster. The security team or PKI owner makes this decision and records who holds the CA signing key, how it is stored, and how revocation is published. Confirm that the CA can sign certificates with the extended key usages your clients and brokers need, and that its root is trusted by the Java truststores you plan to use. The step is complete when the CA certificate can be inspected and a test certificate chains back to it.
Issue broker and client certificates with fixed subjects
Issue one certificate per broker with subject alternative names that match the hostnames clients and brokers use, and one certificate per client identity with a distinguished name that will become its Kafka principal, such as a CN for each application or service account. Keep the naming convention documented, because every ACL and mapping rule depends on it. Each certificate should be stored in a keystore that only the owning process can read. The check is to print each certificate with keytool or openssl and confirm the subject, the SANs, and the issuer match the convention.
Create broker truststore and enable required client auth
Build a truststore on each broker that contains the CA certificate used to sign client certificates. Then set ssl.client.auth to required on the SSL or SASL_SSL listener, and point ssl.truststore.location and its password at that file. Do this in the broker configuration that your deployment tool manages, so every broker in the cluster gets the same setting. Rolling restart and listener changes are covered in the sibling topic on configuring TLS for Kafka listeners, so follow that sequence rather than restarting all brokers at once. The check is a broker log line showing the listener started without errors, and a TLS handshake that requests a client certificate.
Map certificate subjects to principals
By default Kafka derives the principal from the full distinguished name of the client certificate, which is often too long to read in ACLs. Set ssl.principal.mapping.rules so that the distinguished name is reduced to the field you chose in the naming convention, for example the CN. Test the rule before deploying it, because a rule that does not match will cause every client to fail authorization with an unexpected principal. Keep the rules in version control with the broker configuration. The check is a kafka-acls or broker log entry that shows the mapped principal, such as User:orders-api, rather than the full subject string.
Grant ACLs to each mapped principal
With the StandardAuthorizer, resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so confirm that the setting is false and that super users are limited to administrators. Create ACLs for each mapped principal that grant only the operations its application needs, such as Read on a specific topic and Read on its consumer group. Use prefixed ACLs only where your naming model supports them. The check is a listing of ACLs for each principal with kafka-acls, reviewed against the application's documented topic and group list.
Test with a client that has no certificate
Run a client configured for SSL without a keystore against the secured listener. The handshake must fail because the broker requires a client certificate, and the client must receive no metadata or data. Record the error message and the broker log entry for the rejected connection. This test checks the one control that differs from requested mode, so do not skip it or substitute a test that only checks the listener port is open. The step is complete when the connection is refused at the TLS layer, not after a successful connection.
Test with a valid client certificate
Run a client with a certificate issued under the naming convention and confirm it can produce to and consume from only the topics and groups its ACLs allow. Then run the same client against a topic it is not granted, and confirm authorization fails. Confirm the broker logs show the mapped principal for both requests. The step is complete when the permitted operations succeed and the denied operation fails with an authorization error, which together show that authentication and authorization work as designed.
Document issuance and revocation procedures
Write down how certificates are requested, approved, renewed and revoked, and who performs each action. Confirm with your PKI owner how revocation is published and how brokers will learn about it, because Kafka's TLS handshake does not query revocation by default and your team must verify what the platform checks in your environment. Record the expiry date of each certificate and assign an owner to renew it before it lapses. The step is complete when a revoked test certificate is confirmed as rejected under your revocation process, or when the gap is recorded as an accepted risk.

Choosing a certificate source by environment

Many organizations already run an internal certificate authority that their workstations and servers trust. Using it for Kafka clients keeps trust roots consistent and reuses an existing approval process. Issuance usually runs through a request form or an enrollment tool that the CA team controls, so the Kafka team receives a signed certificate and a key it generated or was given. The private key is normally created on the client host or in the application's secret store, not by the CA, and it should never be sent to the broker team. Revocation depends on the CA publishing certificate revocation lists or an online status responder that brokers can reach. Confirm with the CA owner whether those endpoints are available from every broker network segment, how long a revoked certificate stays effective after publication, and whether Kafka's configuration will be checked against that revocation data at all.

Issuance path: Confirm the request workflow, required approvals and turnaround with the CA team before the first client is onboarded.
Key ownership: Ask whether keys are generated on the client, which keeps them out of the CA's hands, or generated centrally.
Certificate lifetime: Confirm the maximum validity the CA issues for client certificates and who renews them.
Revocation: Confirm which revocation mechanism the CA publishes and whether broker hosts can fetch it.
Subject naming: Agree the distinguished name format early, because ACL principals are derived from it.

Check that the CA's policy permits client authentication usage on the certificates you request.

Where mTLS gives false comfort and how to avoid it

Client certificates only protect a cluster when the broker enforces them, maps them to the right identities and keeps unsecured paths closed. The cards below cover the gaps we see most often between a certificate policy on paper and what the cluster actually accepts, and the check that closes each one.

Requested mode accepts certificate-less clients

Setting ssl.client.auth to requested asks clients for a certificate but still lets connections through when none is presented. A client without a certificate then authenticates as anonymous or falls back to another path, so the control looks enforced while it is optional. Set the value to required on every secured listener, then test with a client that presents no certificate and confirm the connection is refused.

Subject patterns that map too broadly

Principal mapping rules in ssl.principal.mapping.rules turn certificate distinguished names into Kafka principals. A loose pattern, such as one matching any certificate issued by the CA, can give many services the same principal and therefore the same ACL grants. Write rules that match a specific CN or SAN pattern for each client, and review the resulting principals with the mapping rules in a test cluster before rollout.

Shared client keys across services

When several applications share one keystore and private key, the certificate identifies the key, not the workload. Any one service can then act as every other service, and revoking one compromised deployment means revoking all of them. Issue a separate certificate and principal per application or team, store each private key only where that workload runs, and check that no two services present an identical subject.

No revocation path inside Kafka

Kafka does not manage a certificate revocation list or check revocation status for client certificates on its own. Revoking a client depends on a process in the certificate authority, plus a way for brokers to stop trusting the certificate, which the team must design and test. Document who can revoke, how fast brokers pick up the change, and remove the matching ACLs so the principal has no residual access.

Plaintext listeners left open beside mTLS

A cluster can run a secured listener and an unsecured one at the same time, and clients will use whichever address they were given. An open PLAINTEXT listener bypasses mTLS and the ACLs built on it. Map listeners explicitly in listener.security.protocol.map, remove plaintext listeners once clients have moved, and confirm which ports are reachable from each network. Listener exposure is covered in the penetration testing topic.

Issuing, rotating and revoking client certificates

Every client certificate should trace back to a CA your team controls, and the lifecycle should be written down before the first application receives a keystore. Issuance means a defined process: who requests a certificate, which CA signs it, what the subject distinguished name encodes, and where the private key is generated. Kafka does not care how the certificate was issued, but it does care about the subject it sees, because that subject becomes the principal your ACLs grant or deny. Keep the subject format stable and documented, since a change to it can silently move every client to a different principal. Distribution is the step most often done by hand: keystores are copied into deployment pipelines, secret stores or configuration management. Decide which system holds the keystore, who can read it, and whether the private key ever leaves the host or pipeline that uses it.

Rotation should happen well before expiry, and the cost of short lifetimes is the part teams underestimate. Shorter certificate lifetimes limit the damage from a leaked key, but every rotation means a new keystore reaches every client, and the listener's truststore must still accept the new chain. Test rotation on a non-production cluster and confirm that your CA can issue overlapping certificates so clients can move without a window of failure. Ask the CA vendor or internal PKI team whether revocation is performed through a certificate revocation list (CRL) or the Online Certificate Status Protocol (OCSP), and whether your Kafka client stack checks either one. Do not assume Kafka brokers enforce revocation on their own. Confirm this against the Kafka version you run and your client libraries.

Retiring an application is not complete until its principal can no longer reach the cluster. A decommissioned client keeps its certificate until it expires, and its ACLs remain in the metadata log until someone removes them. Build a retirement step that revokes or stops trusting the certificate, deletes the ACLs that name its principal, and removes its keystore from secret stores and pipelines. Review this against your ACL inventory on a fixed schedule so that orphaned principals do not accumulate. Because a valid certificate is the proof of identity, a certificate that still chains to a trusted CA with no matching ACL is a risk, not a harmless leftover.

Limits to confirm in your environment

Mutual TLS client authentication depends on details that vary by Kafka release, client library and listener layout. The cards below name the limits to confirm in your own environment before you rely on certificate identity for access control.

Revocation behavior by version

Certificate revocation in Kafka is not something to assume from the TLS standard alone. Confirm exactly how your Kafka release handles revoked or expired client certificates, and whether a revoked certificate is rejected at connection time or only after an existing session ends. Test this against the release you run, using a revoked test certificate, and record the result before you write a revocation procedure.

Certificate chain compatibility by client

Each client language and library builds its trust store and handles intermediate certificates in its own way. Some clients need the full chain presented, some require specific key formats, and some handle hostname checks differently. Inventory every client your teams use, including Connect workers and tools, and confirm each one completes a mutual handshake with your issuing chain before you enforce client certificates.

Java version for brokers

Brokers, Kafka Connect and tools require Java 17 from Kafka 4.0 onward, while clients and Kafka Streams applications need at least Java 11. Check the JVM on every broker and Connect worker before changing listener settings, and check the Java level of older application clients that will present certificates. A mismatch found during the change window is harder to resolve than one found in planning.

Mixed SASL and mTLS listeners

A listener can accept SSL with client authentication, SASL, or both, and the security protocol map determines which listener serves which clients. Confirm which principal Kafka assigns when a client connects over a listener that accepts more than one method, and whether an unintended path can reach a privileged resource. Map each listener to an explicit set of clients and verify the principal each one receives.

Requested mode and optional clients

The ssl.client.auth setting has modes that differ in how strictly brokers require a client certificate. In the requested mode, a client without a certificate may still connect, which means ACLs can be bypassed if the principal falls back to an anonymous identity. Set the mode deliberately, then confirm with a client that presents no certificate that it is refused on the listeners meant to be protected.

Testing with your running release

Client certificate behavior, principal mapping and revocation handling can change between releases, and the supported set moves forward on a fixed schedule. Keep a test cluster on the same Kafka version, JVM and client libraries as production, and rerun the certificate test cases after every upgrade. Use the project release list to confirm your version is still supported before you plan the test cycle.

Mutual TLS in Kafka: questions buyers ask before hiring help

What drives the cost and effort of mutual TLS in Kafka?

The main drivers are how the certificate authority is designed, how many client applications need certificates, and how much of issuance, renewal and revocation is automated. A single internal CA with a short list of services is a small job. A multi-team estate with many languages, long-lived clients and no existing PKI takes considerably more design and testing. One-time work includes the CA hierarchy, the broker keystores and truststores, the principal mapping rules and the test plan. Recurring work includes issuing new client certificates, rotating them before expiry and reviewing who holds which identity. Ask any quote to break out CA design, client count, automation and rotation separately, so you can see which parts scale with your estate.

Can our own team set up mutual TLS for Kafka without a partner?

Yes, a team with PKI experience can do it, provided it has a CA it trusts, a written test plan and time for a staged rollout. The Kafka security documentation covers the broker configuration, and the authorization documentation explains how a certificate identity becomes a principal for ACLs. The part teams most often underestimate is client language support. Each client library, connector and tool must present a certificate, read a keystore in the format it expects and reconnect cleanly after rotation. Check those behaviors for every client you run before you commit to a deadline. An outside partner usually adds value on the CA design and the rollout sequence, not on the Kafka settings themselves.

How long does a mutual TLS project take?

No fixed figure applies because the duration depends on the number of listeners, clients, certificate authorities and change windows in your environment. A project covering one cluster and a few well-understood clients is much shorter than one spanning several clusters, Connect workers and legacy applications. Most engagements are fixed-scope, so the timeline is agreed in writing once the scope is known. Before that conversation, inventory your clients, their language and library versions, and the systems that cannot take a restart during a rollout. Those three facts shape the schedule more than anything else.

What does the work need from our organization?

It needs access to the certificate authority or the team that operates it, an inventory of every client that will connect over TLS, and agreed change windows for broker restarts and client redeployments. Security and platform owners also need to approve the principal naming scheme, since ACLs will be written against those names. You should also name an owner for certificate renewal and revocation. Without that owner, certificates expire silently, and the first sign of trouble is an outage. Ask any partner to list these inputs in the proposal so you can confirm you can provide them.

Does Kafka alone provide a complete certificate lifecycle?

No. Kafka reads keystores and truststores, enforces the client authentication setting and maps certificate subjects to principals. It does not issue certificates, track expiry across your estate, handle revocation distribution or rotate client certificates on a schedule. Those functions belong to your PKI, whether that is an internal CA, a managed service or a secrets platform. Kafka also needs a restart or dynamic configuration update to pick up some truststore changes, so the rotation process has to be designed around that behavior. Confirm the exact reload behavior for your Kafka version in the documentation before you write the runbook.

Why is requested client authentication not enough?

With ssl.client.auth set to requested, the broker asks for a client certificate but still accepts clients that do not present one. Any unauthenticated client can then connect, and ACLs have no reliable identity to evaluate for those connections. Required mode closes that gap, but only when the trust store contains exactly the CAs you intend to accept. Teams often run requested mode during migration and forget to change it. Treat that setting as a release gate and verify it during testing.

What evidence should we expect from a mutual TLS engagement?

Expect the CA design, the listener and client authentication settings, the principal mapping rules, the list of issued client identities and a record of the test results. Each client should be shown connecting with its certificate, failing without one and being denied access outside its ACLs. Expect a documented revocation and rotation runbook as well. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors and certification bodies issue any audit report or attestation. Confirm with your assessor which artifacts they will request before the build starts.

How DeployOpen scopes and delivers mTLS for Kafka

DeployOpen plans and implements Mutual TLS client authentication for self-hosted Apache Kafka inside your own infrastructure. We begin with a written scope that maps your client population, certificate authority and principal naming to ACLs, then build, test and document the result for your operations team.

How we scope

Inventory every producer, consumer, Connect worker and admin tool that connects to each listener.
Agree the certificate authority model, issuing process and certificate lifetimes with your security team.
Define the distinguished name to principal mapping rules and the ACL model each principal will receive.
Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.

How we build

Configure brokers with ssl.client.auth set to required on client-facing listeners, not requested.
Write and test ssl.principal.mapping.rules so each certificate subject resolves to the intended principal.
Set up the StandardAuthorizer deny-by-default posture and confirm no resource falls back to open access.
Test rejected connections, expired certificates and mismatched subjects against a non-production cluster first.

How we hand over

Deliver a runbook for issuing, renewing, revoking and rotating client certificates and truststores.
Document the principal mapping rules, ACL inventory and the reasoning behind each access decision.
Provide verification scripts that show which principals connect and which requests the broker denies.
Train your operators on revocation steps and on the limits of certificate revocation checking in Kafka.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after handover.
Retainers can cover re-review of ACLs and mapping rules when new teams or applications onboard.
Support can include certificate rotation rehearsals before expiry dates in your change calendar.
Ask any quote to state which items recur, such as rotation, reviews and upgrade testing.

Mutual TLS client authentication in Apache Kafka