Enabling TLS and SASL on a running Kafka cluster without downtime
You can enable TLS and SASL on a Kafka cluster that is already serving production traffic without a full outage, because Kafka security is optional and a broker can run secured and unsecured listeners at the same time. The staged path is to add a secured listener alongside the existing plaintext one, roll the brokers one at a time, move clients to the new listener, switch inter-broker traffic to the secured listener, and only then close the plaintext port. This work applies to clusters running in KRaft mode on the supported releases listed on the Apache Kafka downloads page: 4.3.1, 4.2.2 and 4.1.2. Confirm your running version against that list before you plan the change, since the rolling steps depend on the release you are upgrading from.
Start by inventorying every client, Connect worker, Streams application and tool that connects to the cluster, along with the bootstrap addresses each one uses. Each of these must be able to switch to the new listener, and the inventory is what decides how long the dual-listener period lasts. Plan the listener.security.protocol.map, listeners and advertised.listeners changes as one reviewed configuration set, and decide whether inter.broker.listener.name moves in the same window or in a later one. Confirm the certificate chain, SASL mechanism and principal mapping in a non-production cluster first, because a mistake in a listener definition is caught most safely there.
Keep the plaintext listener open until the metrics show that no client still connects through it. Broker request metrics and connection logs by listener show who is still on plaintext, and those are the signals to act on before closing the port. Closing the plaintext listener is the step that cannot be undone quietly, so the rollback plan should state which configuration returns the old listener, which clients would be affected and who approves the closure. Ask your team or partner to write these checks into the change record so the cutover is verifiable rather than assumed.
What TLS, SASL and ACLs each change for clients and brokers
Turning on security on a live cluster touches four separate things: how traffic is encrypted, how clients prove who they are, what those identities may do, and what default applies to resources nobody has granted access to. Each one changes client configuration and broker behavior in a different way, so plan and verify them in that order.
Encryption in transit with TLS
Kafka encrypts traffic between clients, brokers and tools with SSL/TLS, but security is optional, so a cluster can run unsecured or mix secured and unsecured listeners. Each secured listener needs a broker certificate chain that clients trust. Clients must switch their bootstrap address to the secured port and set the security protocol, plus a truststore if the internal certificate authority is not already trusted. Check hostname verification and the certificate names in every advertised listener before you move clients.
Client authentication with SASL
Kafka authenticates clients, brokers and tools with SSL or SASL. The documented SASL mechanisms are GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. Each mechanism asks something different of the client: a Kerberos ticket and keytab, a username and password, or an OAuth token from your identity provider. Pick one mechanism per listener, confirm that every client library in use supports it, and plan credential distribution before the first client switches.
Authorization with the StandardAuthorizer
In KRaft clusters the StandardAuthorizer stores ACLs in the cluster metadata log, so ACL changes replicate through the controllers rather than through a separate store. Authorization is pluggable, and some organizations route it to an external service instead. Confirm which authorizer your target configuration names and that it is set before clients authenticate, so that authenticated principals are not silently blocked or silently allowed.
Default deny for unmatched resources
With the StandardAuthorizer, resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. This default matters most during migration: a client that works on plaintext may fail once authenticated if no ACL grants it access. Before you close plaintext, list every topic, group and cluster operation each client uses, grant them explicitly, and test that the rest of the cluster stays closed.
Broker and inter-broker settings
Brokers need their own identity on every secured listener, including the inter-broker listener, which is set separately from client listeners. Enabling security on the inter-broker path is a broker-side change that goes through rolling restarts, one broker at a time. Confirm the listener and security protocol maps, the inter-broker protocol setting and the controller listener before you start, because a mismatch leaves brokers unable to replicate or elect leaders.
Client configuration changes
Every client needs its own security settings: protocol, SASL mechanism, credentials or a JAAS configuration, and a truststore where the broker certificate is not public. Changes apply per application, so inventory producers, consumers, Kafka Streams applications and Connect workers before you begin. Track which clients have moved, and keep the plaintext listener open until the last one has been confirmed on the secured port.
Which listener and authentication options fit which situation
This table compares four options for the new secured listener on a running cluster: SSL only, SASL_SSL with SCRAM, SASL_SSL with GSSAPI and SASL_SSL with OAUTHBEARER. Read each row against your identity sources and client fleet, since the main takeaway is that the identity system you already run usually decides the choice, and every option still needs TLS underneath.
| Option | When it fits | What clients must change | Main trade-off |
|---|---|---|---|
| SSL only | Clients can hold certificates from your internal CA and you can issue and revoke them per application | Add truststore and keystore settings to every client; certificate distribution becomes part of each release | Strong transport encryption, but identity depends on certificate lifecycle work that must be run and audited |
| SASL_SSL with SCRAM-SHA-256 or SCRAM-SHA-512 | Many applications or teams need separate usernames and passwords and you have no Kerberos or OAuth identity service to reuse | Add SASL mechanism, username and password settings, and a truststore for TLS; credentials are created in cluster metadata | Simple to operate, but credentials must be created, rotated and stored securely by your team, and TLS is required alongside it |
| SASL_SSL with GSSAPI (Kerberos) | Your estate already runs Active Directory or FreeIPA and clients can obtain tickets from it | Install Kerberos client configuration, keytabs or ticket caching, JAAS settings and service principal names on brokers and clients | Central identity with no shared passwords, but clock sync, DNS and ticket handling add operational dependencies that must be tested |
| SASL_SSL with OAUTHBEARER | You run an OAuth 2.0 or OIDC provider and want short-lived tokens tied to your identity model | Obtain and refresh bearer tokens in each client and configure token validation on brokers; confirm the validation settings for your release | Good fit for federated identity, but token validation and the callback setup are more involved, and recent OAUTHBEARER validation issues make patch tracking essential |
| Mechanism support in your release | Applies to every option above | Confirm that each listed mechanism is supported in the exact Kafka 4.x release you run and in every client library version in use | Support can differ between releases and client libraries, so verify before choosing rather than assuming |
| Plaintext listener during transition | Needed only while old clients still connect and must move to the secured listener | Keep it open until the client inventory shows no remaining plaintext connections, then remove it | Keeping it open after migration leaves unauthenticated access in place; set a dated removal step |
| Inter-broker listener | Brokers must authenticate to each other on the secured path before plaintext is closed | Brokers need their own keystores and the inter-broker security settings, applied through rolling restarts | Mistakes here can stop brokers from communicating, so test on a non-production cluster first |
Setup sequence for a rolling security rollout
This sequence moves a running Kafka cluster from plaintext to secured listeners in an order where every client and broker can still connect at each stage. Each step ends with a check, so a failure is caught while the old listener still serves traffic.
Inventory clients, listeners and dependencies
Before changing anything, list every broker listener, every client application, Connect worker, Kafka tool and monitoring agent that connects to the cluster, and the listener each one uses. Record the owning team for each entry and the cutover window they can accept. Confirm which clients run Kafka client libraries you can update without a code change, because those determine how long the plaintext listener must stay open. Done when every connection source appears in one inventory that the platform team and application owners have reviewed.
Issue certificates and credentials
Create the certificate authority or obtain certificates from your existing internal CA, then issue broker server certificates with subject alternative names that match each broker hostname or address clients use. Prepare the keystores and truststores, and generate the SCRAM or Kerberos credentials your inventory calls for. Store private keys and passwords in your secrets system, not in configuration files that get copied between hosts. Done when a test keystore loads on one broker host and the chain validates against the truststore from a client host.
Add a secured listener beside plaintext
Add a new listener on a separate port with a SASL_SSL or SSL security protocol, and advertise it alongside the existing plaintext listener in the listener and advertised listener settings. Keep the plaintext listener active so existing clients keep working. Roll this change one broker at a time and restart each broker only after the previous one has rejoined the cluster. Done when every broker lists both listeners, the secured port accepts a TLS handshake, and under-replicated partition counts return to their pre-change baseline.
Perform a rolling restart with health checks
Restart brokers one at a time, waiting after each restart until the broker is back in the cluster, its partitions are in sync and controller activity is normal. Check the broker logs for listener or certificate errors before moving to the next broker. Stop the rollout and revert the change on the broker if health checks fail. Done when all brokers run the new listener configuration and the cluster shows no unexpected leader elections or offline partitions.
Create principals and ACLs before cutover
Create SCRAM users or Kerberos service and client principals, then write ACLs that grant each one the same access it has today. Confirm the authorizer is in place and check whether
allow.everyone.if.no.acl.foundis set, because a resource with no ACLs is restricted to super users unless that setting is true. Test each ACL set with a non-production client before any production client uses it. Done when a test client authenticated through the secured listener can produce and consume on its expected topics and is denied on topics it should not reach.Move clients in batches while watching metrics
Update client configuration to the secured listener in small batches, grouped by owning team, and restart or reload each batch. Watch request error rates, authentication failures, consumer lag and connection counts for each batch before starting the next. Keep a rollback path that points the batch back to the plaintext listener. Done when the inventory shows every client on the secured listener and the plaintext listener has no active connections for at least one full business cycle that your team has agreed on.
Switch inter-broker traffic to the secured listener
Change the inter-broker listener name and security protocol so brokers replicate and coordinate over the secured listener, and confirm the inter-broker settings match the certificates and credentials you issued. Roll this change across brokers one at a time with the same health checks as before. Confirm that controller traffic follows the secured path for your KRaft layout. Done when broker logs show inter-broker connections on the secured listener and replication is fully in sync.
Verify with a test client and remove plaintext
Run a test client that connects only through the secured listener and exercises produce, consume, admin and ACL-denied paths. Check that a plaintext connection attempt is refused once the plaintext listener is removed in a staged test. Then remove the plaintext listener from every broker with a rolling restart, and rerun the test client afterward. Done when the test client succeeds, plaintext connections are refused, and your security evidence records the final listener and authorizer configuration.
How the rollout differs by environment
On self-managed hosts, the broker configuration file is the main place where the change happens. You add a new listener with its own name and port, map its name to a security protocol in the listener security protocol map, and keep the existing plaintext listener running until clients have moved. Each broker is restarted in turn by your own tooling, so the restart order, health checks between brokers and the wait for under-replicated partitions to recover all sit in your runbook. Certificates are usually copied to each host by configuration management, and the keystore and truststore paths must match on every broker.
- Listener changes: Edit
listeners,advertised.listenersandlistener.security.protocol.mapin eachserver.propertiesfile, then restart brokers one at a time. - Restart mechanics: Your team controls the sequence, so wait for in-sync replicas to recover before moving to the next broker.
- Certificate distribution: Configuration management or a secrets tool places keystores and truststores, so confirm file ownership and permissions on every host.
- SASL credentials: Create SCRAM or Kerberos principals before the listener that requires them is opened.
- Rollback path: Keep the old plaintext listener defined until the final step so a broker can be reverted without a cluster-wide change.
Check the exact property names, the listener syntax and the rolling restart guidance for your Kafka release in the Kafka security overview before you edit any file.
With an operator, you describe listeners in the Kafka custom resource rather than in broker files, and the operator performs the rolling restart. The operator generally manages certificate creation and mounting for its own listeners, so the work shifts toward defining listeners correctly, deciding how external access is exposed and confirming that the operator reconciles changes in the order you expect. Your job is to read what the operator does during a listener change and to avoid manual edits to resources it owns.
- Listener changes: Add an internal TLS or SASL listener to the custom resource while the plaintext listener remains, then let the operator reconcile it.
- Restart mechanics: Confirm with the operator documentation how it sequences broker pod restarts and whether it waits for partition health between pods.
- Certificate distribution: Check whether the operator issues certificates itself or reads secrets you supply, and how it handles renewal and trust stores.
- Client rollout: Clients inside the cluster may use service names, so update bootstrap addresses and check hostname verification against the generated certificates.
- Pod disruption budgets: Confirm the disruption budget and maintenance window settings so that restarts do not stack up during a change.
Confirm these behaviors in the operator's own documentation for your version, because restart sequencing and certificate ownership differ between operators.
Across sites, listener changes must be coordinated so that brokers in one site can still reach brokers in another while each site moves forward. The inter-broker listener and the replication path between sites carry the most risk, because a misconfigured listener can split a cluster or stall replication. Certificates often come from different internal certificate authorities or trust domains, so the truststore on every broker must trust every site's issuing authority before any site switches to secured inter-broker traffic.
- Listener changes: Stage the secured inter-broker listener site by site, with the plaintext path kept until every site confirms health.
- Restart mechanics: Plan restarts per site and confirm that a single site's maintenance does not remove the quorum needed for metadata.
- Certificate distribution: Confirm that each site's truststore contains the other sites' issuing authorities, and that rotation schedules do not expire in different windows.
- Cross-site replication: If you use MirrorMaker 2 for geo-replication, check how its clients and connectors authenticate to each cluster before changing those clusters' listeners.
- Network paths: Verify firewall rules and advertised addresses for each site separately.
Confirm the multi-site and geo-replication guidance in the Kafka geo-replication documentation and your own network design before starting the first site.
Moving clients over without breaking consumers
The first task is an inventory, because a plaintext listener can only close once nothing depends on it. List every producer, consumer, Kafka Connect worker, Kafka Streams application, admin script and monitoring tool that connects to the cluster, along with the bootstrap address, the listener port and the client library version each one uses. Include the quiet members of the list: scheduled jobs, one-off scripts, dashboards and vendor agents that only talk to Kafka at night or during a quarterly process. For each entry, record the owning team and the security mechanism it will move to, whether that is mutual TLS, SCRAM or Kerberos. Clients must also run a library version that supports the chosen mechanism, so check each one against the release notes of its client library before scheduling its change. Confirm the broker-side mechanism names and the client-side configuration keys with the official Kafka security documentation for your release, not from memory.
Keep the plaintext listener open until the inventory is complete and every client has moved. Clients often fail in ways that look like a broken cluster, and a plaintext fallback lets a team roll back a single application without an outage for everyone else. Confirm that traffic has left the plaintext port by checking broker-side connection and request metrics per listener, filtered by client identity, over a window long enough to include weekly and month-end jobs. Cross-check the result against the inventory, and treat any unexplained plaintext client as an open item. Consumers deserve particular care: a consumer that moved to a new listener but still has stale configuration can keep committing offsets from a process that nobody monitors. Record the sign-off for each team in writing before you remove the plaintext listener in the final rolling restart.
Security and operational risks during the cutover
Moving a running cluster to TLS and SASL creates a window where clients, brokers and tooling are on different security settings. The risks below sit in that window, and each one has a control you can check before, during and after the change.
Plaintext listener left open
A cluster that finishes its migration with the original PLAINTEXT listener still bound keeps accepting unauthenticated traffic on the same port, so the new security settings protect only part of the attack surface. Confirm that listeners and advertised.listeners on every broker contain only secured protocols, and that listener.security.protocol.map no longer maps any name to PLAINTEXT. Then test each advertised port from outside the broker network to confirm nothing answers without a handshake.
Certificate expiry during rollout
A rolling restart can take longer than expected, and a certificate that expires partway through leaves some brokers unable to authenticate to their peers while others still work. Check the validity dates of every broker and client certificate before the first restart, and confirm the truststores contain the CA chain the peers will present. Record which certificate each broker is serving so you can confirm the rollout finished on the intended set.
ACL gaps blocking clients
Once authentication is enforced, a client that was allowed in plaintext can be refused if no ACL grants it access to its topics or groups. In KRaft clusters, resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, which is a setting to review rather than a default to rely on. Before enforcing, map each existing client principal to the topics, consumer groups and cluster operations it uses, and verify those grants in a test cluster.
Credentials stored in unsafe places
During the cutover, SASL usernames and passwords tend to appear in client config files, shell history, deployment manifests and tickets. Store SCRAM credentials and keystore passwords in a secrets manager or restricted files readable only by the service account, and confirm that no credential is written to broker logs or change records. Ask who can read the broker configuration files, since they hold the keystore and SASL settings.
Rolling restarts dropping partitions
Each broker restart temporarily removes its replicas from service, and a restart started while other partitions are already under-replicated can leave some partitions with no in-sync copy. Confirm that topics run with a replication factor and minimum in-sync replicas that tolerate one broker down, and check the under-replicated partition count before restarting each broker. Pause the rollout whenever that count does not return to its baseline, and keep the plaintext listener open until every broker has finished.
Questions buyers ask before enabling security on a live cluster
Can we turn on TLS and SASL on a running Kafka cluster without downtime?
In most cases yes, because Kafka lets a cluster run secured and plaintext listeners side by side while brokers restart one at a time. The usual sequence is to add a secured listener, roll brokers so they advertise both listeners, move clients to the secured port in stages, enable inter-broker security, and close the plaintext listener last. Downtime risk comes from mistakes in the listener map, certificate hostnames and client configuration, not from the sequence itself. Confirm the exact steps against the security overview for your Kafka version before scheduling any change, and rehearse the sequence in a non-production cluster that matches your topology.
What drives the effort and cost of enabling security on an existing cluster?
The main drivers are the number of brokers, the number of clients and applications that connect to them, how many teams own those clients, and how identity is integrated. A small cluster with a handful of well-known clients is a different job from a shared platform with dozens of teams, each running its own producers, consumers and connectors. Identity work, such as connecting SCRAM credentials to an existing directory or Kerberos to Active Directory or FreeIPA, adds design and testing time. Ask any provider for a quote that breaks the work into inventory, design, change execution, client migration, testing and handover, and that states which items are one-time and which need ongoing support.
What does our internal team need to provide for this work?
Your team needs to supply a current inventory of brokers, listeners, clients, Connect workers and tools that connect to the cluster, plus owners who can confirm each client's configuration and change window. You will also need access to the change process, the certificate authority or PKI that issues broker and client certificates, the identity source for SASL, and a test environment that mirrors production as closely as practical. Expect your security and platform teams to approve the ACL and certificate design. Ask the provider to list these inputs in the written scope so gaps show up before the change window, not during it.
Can our own staff enable TLS and SASL without outside help?
A team that already runs Kafka in production, understands listeners, keystores and truststores, and can coordinate client owners can often do the change itself. The risk rises when the cluster is business critical, when many unknown clients connect, or when the team has not yet rehearsed rolling restarts. A practical test is whether your team can explain, for each listener, which clients use it, how they authenticate and how you would reverse the change. If the answer is unclear, a fixed-scope engagement can cover inventory, design and rehearsal while your staff keep operational ownership.
How long does enabling security on a live cluster take?
The duration depends on the same factors as cost: cluster size, client count, number of owning teams, identity integration and how many change windows your organization allows. Client migration usually takes the longest, because each owner has to test and deploy changes on their own schedule. DeployOpen does not publish a timeline. A figure becomes meaningful only after a written scope lists the systems, clients, identity sources and deliverables, and the timeline follows from that scope. Ask for the schedule to show separate phases for rehearsal, each broker roll, each client group and the final closure of plaintext listeners.
Is rollback possible if a TLS or SASL change goes wrong?
Rollback is generally possible at each stage because the plaintext listener stays open until clients have moved and inter-broker security is verified. Before each broker roll, record the current configuration and certificates, and keep the plaintext listener defined until the final closure step. If a broker fails to start or clients cannot authenticate, you can revert that broker's listener configuration and restart it, then investigate. Closing the plaintext listener is the step that removes the fallback, so it should happen only after a full verification period. Confirm the exact rollback commands and configuration keys for your version with the security overview and test them during rehearsal.
Which Kafka security features should we enable first on an existing cluster?
Start with the controls that protect the most data with the least client disruption. In practice this means secured listeners with TLS for client and inter-broker traffic, then a SASL mechanism that fits your identity model. The documented SASL options are GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER, and SCRAM is often the simplest first step for teams without Kerberos. Authorization with ACLs should follow authentication, because enabling ACLs before clients have principals can block traffic. Confirm which mechanisms your clients and libraries support before choosing one.
Limits to confirm in your environment
Each card below is a constraint that can stop a rolling security change partway through. Confirm them against your own cluster, client inventory and certificate process before scheduling the first broker restart, because several are much harder to reverse once brokers are running with mixed listeners.
Exact release and mechanisms
Confirm the Kafka release your brokers run and check the security overview for the SASL mechanisms that release documents, which are GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. The supported lines on the downloads page are 4.3.1, 4.2.2 and 4.1.2, so an older broker may need an upgrade before you can use a mechanism you planned around. Write down the exact version now, since the rolling steps depend on it.
Java 17 on every broker
The upgrade notes state that since Kafka 4.0, brokers, Kafka Connect and tools require Java 17. Check the JVM on each broker host, including any host used for administration tools, before you restart it. A broker still running an older JVM will fail to start on the new configuration, which turns a planned rolling restart into an unplanned outage on that node.
ZooKeeper-based clusters
Confirm whether any cluster still uses ZooKeeper. The upgrade notes say ZooKeeper has been removed in Kafka 4.0 and that ZooKeeper-based clusters must migrate to KRaft before upgrading to 4.x. Do not plan listener changes on a ZooKeeper cluster as if they were a KRaft change, because the migration is a separate project with its own sequence and rollback considerations.
Client library compatibility
Inventory every producer, consumer, Kafka Streams application and Connect worker, and record the client library and JVM each uses. Clients must support the security protocol and SASL mechanism you choose, and Kafka Streams applications and clients need at least Java 11 since Kafka 4.0. Clients that cannot be upgraded will block the move to SASL_SSL, so the list determines how long the plaintext listener must stay open.
Default ACL behavior
In KRaft clusters, the StandardAuthorizer stores ACLs in the metadata log, and resources without ACLs are limited to super users unless allow.everyone.if.no.acl.found is set to true. Confirm the value on each cluster before enabling authorization. If it is set to true, unmapped clients keep working after you add ACLs for the others, which hides gaps until a later review finds them.
Certificate authority policy
Confirm with your security team which certificate authority issues broker and client certificates, the maximum validity period, how renewal is authorized, and whether hostnames must match the advertised listener names. Also confirm how the truststore will be distributed to clients. Changing the authority after brokers are rolled forces a second rolling restart, so agree the chain and rotation process before the first secured listener is created.
How DeployOpen scopes and delivers this security rollout
DeployOpen plans and implements self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a live Kafka cluster, we start from a written scope that sequences secured listeners, client migration and plaintext closure so that producers and consumers keep working throughout.
How we scope
- Inventory every listener, client application, Connect worker and tool that connects to the running brokers today.
- Agree in writing which listeners move to TLS or SASL, in what order, and which plaintext ports will eventually close.
- Confirm with your team the maintenance windows, rollback criteria and owners for each client group before work starts.
How we build
- Add secured listeners alongside existing plaintext ones and roll them through the brokers one at a time with health checks between restarts.
- Test client connections against the new listeners in a staging cluster that mirrors your broker and listener layout.
- Move clients in batches, tracking which applications still use plaintext, then enable inter-broker security once every broker supports it.
How we hand over
- Deliver the broker and client configuration, certificate and credential procedures, and the runbook for rolling restarts.
- Document the verification steps used to confirm each listener, the rollback path and the final plaintext shutdown record.
- Walk your operations and application teams through the changes so they can repeat them for new clusters and clients.
How we support
- Fixed-scope engagements end at the agreed handover, with the scope, deliverables and timeline set out before work begins.
- Ongoing support, upgrades and continuous security testing for the cluster are available as a retainer.
- Ask for the support terms, response expectations and handover acceptance criteria in your written quote.
