When SASL/SCRAM is the right login for KRaft Kafka clients
Choose SASL/SCRAM when your clients cannot use Kerberos or client certificates and need a username and password to log in to the brokers. Many applications, connectors and third-party tools only offer a password-style login, and a Kafka cluster that serves them has to accept it without weakening the rest of the platform. Kafka supports SCRAM-SHA-256 and SCRAM-SHA-512 as SASL mechanisms alongside GSSAPI (Kerberos), PLAIN and OAUTHBEARER, according to the Kafka security overview. SCRAM is a challenge-response mechanism, so the password itself never crosses the wire, and the broker stores salted, hashed credentials rather than plain passwords. In KRaft clusters those credentials live in the cluster metadata log, which means they replicate to every controller and broker through the same mechanism as other cluster metadata.
SCRAM does not make a cluster secure on its own. The mechanism authenticates a principal, but it does not encrypt traffic, and a password that is replayed over plaintext listeners is exposed to anyone on the network path. Pair SCRAM with TLS on every listener that carries credentials, and confirm that your clients verify broker hostnames. Authentication is also only the first control. After a client logs in, what it can read, write or create is decided by ACLs, so a SCRAM principal with no ACLs should have no useful access. Plan the credential lifecycle as well: who creates each user, how passwords are issued and rotated, and how a compromised client is removed quickly.
This page covers how to set up SCRAM in a KRaft cluster, whether to create credentials at format time or later with kafka-configs, and how to operate and rotate them safely. Before you start, confirm the exact client library versions and their SCRAM support with your application owners, and check the security settings for your Kafka release in the upgrade and security documentation for that version.
Choosing between SCRAM, PLAIN, GSSAPI and OAUTHBEARER
This table compares the SASL mechanisms Kafka documents by where the secret lives, whether it crosses the wire without a challenge, who manages the identity and when each one fits. Read it as a first filter, then confirm client library support and your identity policy before choosing.
| Mechanism | Where the secret lives | Who manages the identity | When it fits |
|---|---|---|---|
| SCRAM-SHA-256 or SCRAM-SHA-512 | Salted, hashed credentials stored in the KRaft metadata log; the broker never holds a plaintext password | Kafka administrators, using kafka-configs or format-time credentials | Self-contained clusters with per-application users and no existing directory. Pick the variant your client libraries and security policy both support. |
| PLAIN | Cleartext username and password sent on every authentication, checked against broker configuration or an external store | Whoever maintains the JAAS configuration or the external password backend | Only with TLS enforced on every listener and a short-term or test setup. It does not protect credentials on the wire without TLS. |
| GSSAPI (Kerberos) | Keytabs for brokers and clients, with tickets issued by a KDC such as Active Directory or FreeIPA | Central directory team, through the KDC | Organizations that already run Active Directory or FreeIPA and want one identity source and ticket-based access. |
| OAUTHBEARER | Short-lived bearer tokens issued by an external authorization server | Your OAuth or OIDC identity provider | Environments standardizing on OAuth or OIDC for workloads. Token validation and the identity provider setup need careful review. |
| Choosing SCRAM-SHA-256 versus SCRAM-SHA-512 | Same challenge-response design; the hash function differs | Not applicable | Match the mechanism your client libraries implement and the algorithm your password policy names. Confirm both sides support it before you pick one. |
Where SCRAM credentials live and how they get there
SCRAM authentication depends on four parts that must agree: the credential record stored in the cluster metadata, the step that creates it, the later tooling that changes it, and the broker listener that checks it. Each part has a different owner, so each needs its own check before clients connect.
Credential record in the metadata log
In KRaft mode, SCRAM credentials are stored as cluster metadata rather than in a separate database or file on each broker. Every broker reads the same record, so a credential created once is available cluster-wide. The operator must confirm that the metadata quorum is healthy before relying on a new credential, because a credential written during an unhealthy period may not be visible where you expect it. Check the record after creation by authenticating from a client, not only by reading the command output.
Bootstrap creation at format time
Kafka supports creating SCRAM credentials while the storage directory is formatted for the first time, so the first controller and brokers can start with authentication already in place. This is the path for new clusters, and it runs once per cluster bootstrap. The operator owns the bootstrap script and must confirm that the credential set, the mechanism and the cluster identity used at format time are the ones intended for production, since changing them later is a separate operation.
Later changes with kafka-configs
After the cluster runs, the kafka-configs tool creates, updates and deletes SCRAM credentials for users. This is the routine path for onboarding applications and rotating passwords. The operator must confirm who may run the tool, how the admin connection itself is authenticated, and that the command output and shell history do not expose passwords to other users on the host. Record each change against a ticket so credential history stays auditable.
Listener and JAAS consumption
A broker listener must be configured for a SASL mechanism that matches the credentials stored in the metadata log, and each client must present a JAAS configuration with the same mechanism. A mismatch between listener, mechanism and stored credential is the most common reason SCRAM logins fail, and it can look like a password problem. The operator must check that each listener advertises only the intended mechanisms and that client JAAS files are readable only by the service account that runs the application.
How to set up SCRAM on a KRaft cluster in order
This sequence takes a KRaft cluster from unsecured listeners to a TLS-protected SASL/SCRAM listener with an administrative user, a client login that works, and an authorizer that denies anything not explicitly granted. Each step closes with a check you can run before moving on.
Confirm release, Java and upgrade path
Confirm the Kafka release you will run against the project's downloads page and the upgrade notes for that version, and check that brokers and tools run on Java 17, which the notes require for Kafka 4.0 and later. If the cluster is still on ZooKeeper, migrate to KRaft first, because Kafka 4.0 supports only KRaft. The step is done when the target version, Java runtime and any migration prerequisites are written into the change record and the broker hosts report the expected Java version.
Enable a TLS-protected SASL listener
Add a dedicated listener that uses the SASL_SSL security protocol, with a keystore on each broker and a truststore that clients will use. Keep the SCRAM mechanism for this listener only, and leave the plaintext listener in place until the later cutover. Confirm that the certificate subject alternative names match the broker hostnames clients will use, and that the listener accepts a TLS handshake with a standard client tool such as openssl s_client before any SASL configuration is tested.
Create the administrative SCRAM user
Create the first SCRAM credential either during storage format, which writes it into the metadata log before the cluster starts, or later with the kafka-configs tool against a running cluster. Use SCRAM-SHA-256 or SCRAM-SHA-512 as the mechanism, and give the administrative user a long random password stored in your secrets manager, not in a configuration file. The step is done when kafka-configs lists the user's SCRAM credentials for the chosen mechanism and the password is absent from shell history and version control.
Configure client JAAS and truststores
Give each client a JAAS configuration that sets the SCRAM mechanism, the username and the password, and point the client's security settings at a truststore that contains the broker CA. Deliver these values through your secrets tooling and keep the truststore password out of the repository. The step is done when a producer or admin client connects to the SASL_SSL listener using the new credentials and the broker logs show an authenticated session under the expected principal.
Enable the StandardAuthorizer with deny by default
Set the StandardAuthorizer in the broker configuration and keep allow.everyone.if.no.acl.found at its default of false, so resources without ACLs are limited to super users. Write the super user list deliberately, since those principals bypass ACLs. The step is done when a non-super SCRAM user that has no grants is refused a produce or fetch on a test topic, and the refusal appears as an authorization failure in the broker logs.
Verify login and ACLs with test clients
Create a test topic and a narrowly scoped ACL for a test application user, then run one produce and one consume with that user. Confirm that the same user is denied on a different topic and on a consumer group it was not granted. Record the results so they can be repeated after each change. The step is done when the allowed operations succeed, the denied operations fail for the expected reason, and the results are saved as evidence.
Rotate credentials and retire plaintext access
Add a second SCRAM credential for each client, move clients to it, then delete the old credential with kafka-configs so the old password stops working. After every client is confirmed on the secured listener, disable the plaintext listener through a rolling change. The step is done when no client connections appear on the plaintext port in broker metrics or logs, and a credential rotation test shows the old password is refused while the new one is accepted.
Why TLS must be required before SCRAM goes live
SCRAM protects the password exchange, not the session. SCRAM-SHA-256 and SCRAM-SHA-512 let a client prove it knows the password without sending the password itself across the network. That is a real gain, but it covers only the authentication handshake. Once the client is in, the records it produces, the records it consumes, and the offsets it commits travel over the same connection in whatever form the listener uses. The Kafka security overview states that SSL/TLS is what encrypts traffic between clients, brokers and tools, so the encryption you rely on comes from the listener configuration, not from the SCRAM mechanism. Treat TLS as a precondition for enabling SCRAM on any listener that carries real data, and set the listener's security protocol to SASL_SSL rather than SASL_PLAINTEXT for every client-facing and inter-broker endpoint.
Mixed listeners are allowed, which is why a leftover plaintext port undoes the work. The security overview confirms that security is optional, so a cluster can run unsecured or mix secured and unsecured listeners. Kafka will not stop you from keeping a PLAINTEXT listener alongside a SASL_SSL one during a migration, and a client that connects to the plaintext port will skip both TLS and SCRAM entirely. Authorization does not close this gap on its own, because a session that never authenticates has no identity for ACLs to evaluate. Before go-live, confirm in your own environment that every plaintext port is closed in listeners, that advertised.listeners lists only secured endpoints, and that no load balancer, DNS record or client bootstrap list still points at the old port. Check the advertised addresses from a host outside the broker network, because a broker that advertises a plaintext endpoint will send clients there after the first metadata request.
Security risks in SCRAM deployments and how to contain them
SCRAM gives you per-user credentials, but the setup around it decides whether those credentials protect anything. The risks below tend to appear together, so review them as one set: listeners, identities, secrets, authorization and the places credentials end up.
Plaintext listener left open
A SASL_SSL listener does not help if a PLAINTEXT listener still accepts connections on the same broker. Clients can fall back to it, and traffic carrying SCRAM exchanges or data is readable on the network. Check every entry in listeners and advertised.listeners, confirm which security.protocol each one maps to, and restrict or remove plaintext listeners once all clients have moved. Confirm the network rules that reach broker ports from outside the trusted zone.
Shared credentials across teams
When several applications log in with one SCRAM user, you cannot tell which system produced a given request, and revoking access means breaking every consumer at once. Create one SCRAM user per application or team, then grant each user its own prefixed ACLs. Ask who owns each user, how it is named, and whether the owner can rotate it without touching other systems.
Weak or unmanaged passwords
SCRAM protects passwords in transit and at rest on the broker, but it cannot make a weak password strong. Generate long random values for each user, avoid human-chosen strings, and record a rotation date for every credential. Use SCRAM-SHA-512 where your clients support it, and confirm the mechanism list on each client matches what the broker enables.
Super users used by applications
In KRaft clusters, the StandardAuthorizer restricts resources without ACLs to super users unless allow.everyone.if.no.acl.found is set to true. Any principal listed in super.users bypasses ACL checks entirely. If an application authenticates as a super user, a leaked password grants full control of the cluster. Keep super.users to a few operator accounts, grant applications only the ACLs they need, and review the list during each access review.
Credentials leaking into configs and logs
SCRAM passwords often end up in client properties files, container environment variables, Helm values, or shell history when kafka-configs is run with inline values. Logs and ticket systems can capture them too. Move client secrets into a secret store, mount them at runtime, and keep them out of version control. Confirm that log levels and tooling do not echo JAAS configuration, and check that the kafka-configs commands used for credential changes are not kept in shared shell history.
Rotating SCRAM credentials without an outage
Rotating a SCRAM credential means adding a second credential for the same principal, moving the application to it, confirming logins, and only then removing the old one. Kafka stores SCRAM credentials in the cluster metadata log, so the change is made through the cluster rather than by editing broker files, and each step can be checked before the next starts. The pattern works only if the client can hold two credentials for a window, which means the application or connector must be able to reload its password without a restart or be restarted in a planned way. Clients that read the password once at startup and cache it need to be identified before rotation is scheduled.
Write the sequence down before the first run. Add the new SCRAM-SHA-512 (or SCRAM-SHA-256) credential with kafka-configs.sh using --alter --add-config and a new password, and confirm that a test login with the new secret succeeds from the client's network segment. Then deploy the new secret to the application, watch authentication failures and connection counts on the brokers, and check that consumer groups and producers reconnect as expected. Only after the new login has been seen working for the full rollout should the old credential be removed with --delete-config. Keep the old secret available until that point so that rollback means re-deploying the previous secret to the application, not recreating credentials under pressure.
Confirm the exact commands, config keys and behavior against the Kafka version your cluster runs, because the release you deploy determines how credentials and their listener settings are handled. Run the full procedure, including rollback, in a non-production cluster that uses the same authentication setup first, and record how long the application took to pick up the new secret. In production, schedule each rotation, name an owner, and log the principal, the time of each step, the secret's storage location and the removal date for the old credential. Those records are the evidence that a rotation happened and that no unplanned credential is left active.
Limits to confirm in your environment before you commit
SCRAM authentication depends on a handful of constraints that sit outside Kafka itself: the release line, the Java runtime, the client libraries, the authorization model and the handling of credentials. Confirm each one against your own estate before you commit to a design, because a gap in any of them changes the build.
Which supported release will you run
Start from the releases the project still supports, not from whatever version a package repository offers. Ask whether your target is on the current supported list and whether your cluster can reach it through a supported upgrade path. If you are already on a KRaft cluster, confirm its current version, since the upgrade notes give a specific path for older KRaft clusters. Record the chosen release in your design so that later patch decisions have a fixed baseline.
Which Java runtime your brokers and clients need
Brokers, Kafka Connect and tools require Java 17 from Kafka 4.0 onward, while clients and Kafka Streams applications need at least Java 11. Check the JVM on every broker host, every Connect worker and every application that authenticates with SCRAM. Older application servers or fat-jar build pipelines are a common place where the runtime lags behind, and a mismatch there blocks the upgrade even when the brokers are ready.
Whether your clients support the SCRAM mechanism
Kafka offers SCRAM-SHA-256 and SCRAM-SHA-512 as SASL mechanisms, but each client library must implement the one you choose. List every language, driver and connector your teams use, including older or vendor-packaged clients, and confirm which mechanisms each supports. If a client only supports PLAIN, decide whether to keep a separate listener for it or to replace the client before you enable SCRAM.
Whether an external authorization service is required
Kafka authorization is pluggable, and the built-in StandardAuthorizer stores ACLs in the KRaft metadata log. Decide whether your access model can live inside Kafka or must be enforced by an external service that your security team already runs. Also confirm the behavior for resources without ACLs, since they are restricted to super users unless you deliberately change that setting. Make this decision before creating users, because it determines where access is granted and reviewed.
How credentials will be stored and audited
SCRAM credentials are written to the cluster metadata, so the team that can change them effectively controls access. Confirm who may create, rotate and delete users, how those actions are logged, and where the initial passwords are generated and kept. Check that your secrets store and audit pipeline can capture these events. Ask your auditors which evidence they will request for credential lifecycle, because that answer decides how you set up the process before go-live.
Questions teams ask about SCRAM setup, cost and effort
What drives the cost of setting up SCRAM authentication on Kafka?
The cost follows the written scope, and the main drivers are the number of clients, whether TLS already exists on every listener, how credentials reach applications and how much testing is required. Enabling SCRAM itself is a configuration change, but each client application must be found, updated and verified, and that work is what grows with the estate. Ask any provider to price these as separate line items: listener and TLS work, credential creation and storage, client migration by application, identity integration if credentials come from a secrets platform, and testing. Apache Kafka is licensed under the Apache License 2.0 with no paid edition from the project, so the cost is services and infrastructure, not software licensing. Ask for one-time and recurring items to be listed separately, including any retainer for rotation and upgrades.
How long does a SCRAM rollout take and what makes it longer?
Duration depends mainly on how many applications authenticate to the cluster and how many of them can change their client configuration without a code release. A small, well-documented cluster with a few clients is a short project, while a shared platform with many teams adds coordination, change windows and per-application verification. TLS readiness is another driver: SCRAM credentials should travel over encrypted listeners, so any listener still in plaintext adds work first. Identity integration, such as pulling credentials from a secrets manager, adds design and testing time. Ask the provider for a milestone plan that names the client inventory, the cutover order and the rollback steps, and treat any duration given without that inventory as an assumption to test.
What does our internal team need to provide for a SCRAM project?
Your team needs to supply knowledge of the environment and time from owners who can make decisions. The most useful inputs are an inventory of clients, the teams that own them, their current authentication settings and their release schedules. You will also need access to the cluster's configuration, the certificate authority or PKI that issues broker certificates, and the secrets platform where credentials will be stored. Someone must own the decision on who may create and rotate credentials and how the ACLs for each principal are designed. Confirm these with your own environment before the work begins, because the answers change the scope more than the Kafka steps do.
Can our own team set up SCRAM on Kafka without outside help?
Yes, a team with Kafka operations experience can enable SCRAM, because the mechanism is documented and the setup uses standard tooling. The harder part is usually the surrounding work: moving every client without an outage, confirming that no application is still on plaintext or PLAIN, and building a repeatable rotation process. A useful test is whether your team has already rolled a secured listener onto a running cluster and whether it can name every client that connects. If those capabilities are thin, a fixed-scope engagement can cover the design, the rollout and the handover so that your staff keep the knowledge. Decide this by comparing your skills and capacity against the client inventory, not by the mechanism alone.
How should we rotate SCRAM credentials without breaking clients?
Rotate by adding a new credential for the principal, moving clients to it, and then removing the old one once no connection uses it. Kafka stores SCRAM credentials as configuration on the cluster, and the kafka-configs tool can add a SCRAM-SHA-256 or SCRAM-SHA-512 credential for a user, which lets you stage a new password before retiring the old one. Keep rotation tied to the secrets platform so that applications read the current value at start or through a reload mechanism your team has tested. Schedule rotation as a recurring operational task with an owner, an inventory and a rollback plan. Confirm each client library's reconnect behavior before choosing the method, because a client that caches the old password will fail when the old credential is removed.
Is SCRAM enough, or do we also need mTLS or Kerberos?
SCRAM can be sufficient for many internal workloads when it runs over TLS, uses strong credentials and is paired with a deny-by-default ACL model. It identifies clients by username and password, so it does not prove the identity of a workload the way a client certificate or a Kerberos ticket can. Mutual TLS suits environments that already issue certificates to services, and Kerberos suits organizations running Active Directory or FreeIPA. Choose based on your identity system, your audit requirements and how many principals you must manage. Many estates use SCRAM for applications that cannot hold certificates and reserve stronger identity for critical paths. Ask your security team which control evidence they need before deciding.
What should a quote for SCRAM setup include?
A useful quote lists the written scope, the deliverables and the acceptance checks before any price appears. Look for the following items:
- The client inventory and the cutover order, with each application named
- The TLS work on every listener that will carry SCRAM credentials
- Credential creation, storage and rotation design, including the tooling used
- ACL changes tied to each principal, and the testing that proves denied access fails
- Handover documentation, runbooks and any training for your operators
- Assumptions, exclusions and the change control for scope growth
DeployOpen plans, implements, hardens and documents self-hosted Kafka inside your infrastructure, and most engagements are fixed-scope with agreed deliverables and timelines in writing. Ask for recurring items, such as rotation support or upgrades, to be quoted separately.
How DeployOpen scopes and delivers SCRAM authentication
DeployOpen plans, implements, hardens, documents and supports self-hosted Kafka inside your own infrastructure, including SCRAM credentials, TLS requirements and credential rotation on KRaft clusters. Engagements are fixed-scope, with the scope, deliverables and timeline agreed in writing before work starts.
How we scope
- Inventory every client, broker, Connect worker and tool that will authenticate with SCRAM and the mechanism each one supports.
- Agree which SCRAM variant (SHA-256 or SHA-512) applies, and confirm TLS is required on every listener that carries credentials.
- Define the credential lifecycle in writing: who creates, stores, rotates and revokes SCRAM users, and where the secrets live.
How we build
- Configure the KRaft bootstrap for initial SCRAM credentials and set up kafka-configs procedures for later users in your environment.
- Wire listener, SASL mechanism and JAAS settings for brokers and clients, then pair them with ACLs managed through the StandardAuthorizer.
- Test failed logins, plaintext rejection, rotation without downtime and client reconnection before any production cutover.
How we hand over
- Deliver the written configuration, credential runbook, rotation procedure and ACL mapping for each application or team identity.
- Document the test cases used, their results and the steps to repeat them after upgrades or certificate changes.
- Walk your platform, security and application owners through the operating model so they can add users without rework.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped separately from the original engagement.
- Track Kafka 4.x release and security updates, and plan how SCRAM settings and client libraries are updated alongside them.
- Confirm with your team which credential rotation and audit evidence you need each review cycle, and agree how it is produced.
