Kafka Kerberos authentication with SASL/GSSAPI

When Kafka Kerberos authentication fits an Active Directory or FreeIPA estate

Kerberos authentication through SASL/GSSAPI fits an organization that already signs people and services in through Active Directory or FreeIPA and wants Kafka to draw on that same identity source. GSSAPI is one of the SASL mechanisms Kafka documents, alongside PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER, according to the Kafka security overview. It lets brokers and clients prove identity with tickets from a Kerberos key distribution center, so passwords are not sent to Kafka and credentials can be governed centrally by the directory team. It is usually the better fit than SCRAM or PLAIN when joiners, leavers and service account rotation already run through the directory, and when platform teams want one place to disable access.

Security in Kafka is optional, so none of this applies until you enable it deliberately. A cluster can run plaintext listeners, mix secured and unsecured listeners, or turn on SASL without the controls around it. Choosing GSSAPI also means you accept Kerberos operational concerns: time synchronization between hosts, reachable KDCs, keytab handling and principal naming. Confirm with your directory team whether the realm is an Active Directory domain, a FreeIPA realm or a cross-forest trust, because that changes which principals exist and how Linux brokers reach them.

The Kerberos realm is only one part of the design. Authorization still needs ACLs, and on KRaft clusters the StandardAuthorizer stores them in the metadata log, with resources without ACLs limited to super users unless allow.everyone.if.no.acl.found is set to true. You also need TLS on the listeners that carry tickets, a principal-to-user mapping that produces names your ACLs can match, and a plan for how non-human workloads such as stream processors authenticate. Before committing, confirm the GSSAPI behavior for your Kafka release against the Kafka 4.3 authorization documentation and test the full path in a non-production cluster.

Choosing the Kerberos realm: Active Directory, FreeIPA or a cross-realm trust

Brokers join an existing Active Directory domain when the directory team already runs AD DS for Windows users and wants Kafka service principals in the same Kerberos realm. Microsoft's comparison of Active Directory and Entra ID notes that Linux machines need third-party tools or must be configured to authenticate against Active Directory as a Kerberos realm, because Group Policy and domain join work natively only for Windows devices. This keeps the directory change small, but the Linux configuration is entirely your platform team's work.

Directory team scope: the directory team creates a service account for each broker host and agrees the service principal naming, since AD controls account lifecycle and password policy.
Linux enrollment: the platform team configures each broker as a Kerberos client of the AD realm, including krb5.conf and keytab handling, unless a third-party tool is used instead.
Replication and DNS: AD DS replicates directory data to every domain controller in a domain, so broker authentication depends on the domain controllers and DNS names that each site resolves.
Licensing: Windows Server 2025 is licensed per core with a minimum of 16 cores per server, and Client Access Licenses apply to every user or device that accesses the server. Confirm what already applies to the domain controllers you run.
Keytab rotation: a password change on a service account invalidates its keytab, so agree who refreshes keytabs and how often.

Before choosing this path, confirm with the directory team who owns the service accounts, which domain controllers each broker site reaches and whether any Linux management tool is already in use.

FreeIPA is an integrated identity system for Linux and UNIX networks. It combines 389 Directory Server, an MIT Kerberos KDC, the Dogtag certificate system and SSSD on clients, so Kafka brokers can receive Kerberos service principals from a realm that the Linux team operates end to end. Per the FreeIPA About page, this removes the need to bridge a Windows directory for Linux services, but it also makes the IPA servers part of your Kafka availability design.

Directory team scope: none for the broker realm itself, since the Linux team owns the IPA servers, users, groups and host records.
Replica layout: the FreeIPA deployment recommendations advise at least two to three replicas in each datacenter, no more than four replication agreements per replica and dedicated servers without other applications.
Client resolution: clients should list at least two DNS servers in resolv.conf, so broker host lookups survive a single server failure.
Certificate and backup duties: the first IPA server handles certificate renewal and CRL publication by default, and ipa-backup should be part of the recovery plan for that server.
Cost model: FreeIPA is GPL-3.0-or-later and has no paid edition of its own, so cost is the infrastructure, the staff time to run it and any support you contract.

Before choosing this path, confirm that your Linux team can run the IPA replica topology, back it up and patch it on the schedule your Kafka clusters need.

The right realm depends on where your identities already live and which team will carry the operating load for the next several years. An Active Directory realm suits organizations whose directory team already owns Windows identity and can accept Linux work on the broker hosts. A FreeIPA realm suits Linux-heavy estates where the Linux team wants full control of Kerberos, host policy and certificates. A trust suits environments where both directories exist and AD users genuinely need Kafka access. Each choice shifts work between teams, so the decision is an operating model choice as much as a technical one.

Ownership of identities: list who creates principals, who resets passwords and who approves a new client, then match that list to a realm owner.
Operating load: ask who patches the KDC and domain controllers, who runs backups and who answers on-call when ticket issuance fails.
Client types: note which clients need Kerberos, such as Kafka tools, Connect workers or other Linux services, and whether any Windows clients connect.
Licensing and support: ask for the licensing terms that apply to your Windows Server estate and the support terms for any FreeIPA or Kafka component you depend on.
Migration path: if you later move identities between directories, note that ipa migrate-ds excludes Kerberos attributes that FreeIPA regenerates, and sudo and HBAC rules need manual migration.

Before committing, ask both teams to write down the realm owner, the replica or domain controller plan and the keytab refresh process, then request a written scope that covers those items.

How to set up Kafka Kerberos with service principals and keytabs

This sequence takes a Kafka KRaft cluster from DNS and time checks to a broker listener that accepts Kerberos tickets, with a verification at each step so failures surface where they start. The result is a working GSSAPI listener that clients reach with their own principals.

Confirm DNS and time synchronization
Before any principal is created, confirm that every broker, client and the KDC resolve each other by fully qualified hostname, in both forward and reverse directions, and that clocks are synchronized across all of them. Kerberos rejects tickets when clock skew exceeds the realm policy, so this is the most common cause of failures later. Your network or directory team owns DNS and time sources, and you should confirm them in writing before work starts. Verification: run a forward and reverse lookup for each broker's fully qualified name from a client host, and check that each host reports the same time source status.
Create broker service principals and export keytabs
In Active Directory or FreeIPA, create one service principal per broker in the form kafka/broker-host@REALM, using the broker's fully qualified hostname. Export each principal to a keytab file that only the Kafka service account can read, and keep the keytab out of configuration repositories. In Active Directory, the principal is usually linked to a service account and the keytab is generated with ktpass; in FreeIPA, the service entry is created with ipa service-add and the keytab is retrieved on the host. Verification: on each broker, run klist -k against the keytab and confirm the expected principal and key versions appear, then run kinit -k -t with that keytab and confirm a ticket is granted.
Configure the broker JAAS file and SASL mechanisms
Create a JAAS file for each broker that points the Krb5LoginModule at the broker's keytab and principal, with useKeyTab and storeKey enabled. Pass it to the broker through the JVM option java.security.auth.login.config. In server.properties, set sasl.enabled.mechanisms to GSSAPI and add a listener with a SASL_SSL or SASL_PLAINTEXT protocol, depending on your transport decision, plus sasl.mechanism.inter.broker.protocol set to GSSAPI if brokers authenticate to each other with Kerberos. Verification: start one broker and check its log for a successful login under the Kerberos login module and for the listener binding, with no login exceptions.
Set the Kerberos service name on brokers
Set sasl.kerberos.service.name to the service portion of the broker principal, which is kafka in the examples above. Clients and brokers must agree on this value, because the client builds the service principal it requests from it and the broker and client hostnames. Keep the value identical across all brokers and document it with the listener configuration. Verification: from a client host, run kvno kafka/broker-host@REALM against each broker and confirm the KDC returns a service ticket, which proves the principal exists and the name resolves correctly.
Configure client JAAS and krb5.conf
On each client, create a krb5.conf that names the realm, the KDC and the domain-to-realm mapping, then create a client JAAS file that uses Krb5LoginModule with either a keytab for unattended services or the ticket cache for interactive users. Set security.protocol and sasl.kerberos.service.name in the client configuration to match the brokers. Verification: run kinit as the client user, then klist to confirm a TGT is cached, and run a short producer or consumer test against the Kerberos listener using the console tools shipped with Kafka.
Map Kerberos principals to Kafka principals
Kafka sees the authenticated Kerberos principal, such as app1@REALM, and by default the full string is used for ACL matching. Use sasl.kerberos.principal.to.local.rules on the broker to strip the realm or transform the name into the short form your ACLs expect. Write the rules narrowly, because an overly broad rule can map a principal from another realm onto a trusted local name. Verification: connect with a test principal, then list the ACLs with kafka-acls.sh and confirm the principal string used in the rule matches what the authorizer records in the denial or grant log.
Test failure cases and troubleshoot ticket problems
Check the common failure modes deliberately before handing the cluster over. Expired tickets, a clock offset, a keytab with a stale key version after a password reset, a service name mismatch, and a DNS alias that resolves to a different host name all produce distinct errors in the broker and client logs. Run each one in a non-production environment and record the error text so operators can recognize it. Verification: each deliberate fault produces the expected error, and after correcting it the same test principal authenticates without a restart of unrelated components.

Mapping Kerberos principals to Kafka principals for ACLs

Kafka does not use the Kerberos principal as it arrives. When a client or broker authenticates over SASL/GSSAPI, Kafka takes the full principal, such as alice@EXAMPLE.COM, and passes it through the rules in sasl.kerberos.principal.to.local.rules to produce the short name that becomes the principal used in ACLs. A rule that matches [1:$1@$0] and (.*)@EXAMPLE\.COM style patterns rewrites the realm-qualified name into something like alice. The ACL you write must match the output of that transformation, not the name in the directory. Confirm the exact rule syntax against the security documentation for the Kafka release you run, because the rule grammar is the first place where a working cluster and a silently broken one tend to differ.

Test the rules against every realm that can reach the cluster, including cross-realm or trusted-forest principals that arrive from Active Directory or a FreeIPA trust. Run a known principal from each realm through the mapping, record the resulting short name, and compare it with the ACLs you intend to grant. A realm that is missing from the rules, a casing difference between the realm and the rule, or a service principal that maps to an unexpected name will produce an authorization denial, not an authentication failure. The client connects, but produces or fetches are refused because the mapped principal has no matching grant.

This matters because the StandardAuthorizer stores ACLs in the KRaft metadata log and denies access to resources that have no matching ACL. Unless allow.everyone.if.no.acl.found is set to true, which is rarely appropriate for a regulated environment, a mapping mistake fails closed. Before you enable Kerberos on production listeners, build a short test matrix of principal, mapped name, operation and expected result, and run it from a test client in each realm. Keep the mapping rules under version control with the ACL definitions so that a directory change or realm rename is reviewed as a change to access, not as an incidental configuration edit.

Kerberos configuration options and when each fits

This table compares the main Kafka and Kerberos settings used for GSSAPI authentication and shows when each one applies and what breaks if it is wrong. Read each row as a checklist item: confirm the value against your realm, KDC and broker host names before any broker restarts. The main takeaway is that most failed Kerberos deployments trace to a mismatch between a principal name, a hostname or a realm rather than to the Kafka settings themselves.

Setting	When it fits	What breaks if it is wrong
Broker JAAS KafkaServer section and keytab	Every broker that accepts GSSAPI clients, with a keytab holding that broker's service principal	Brokers fail to start the GSSAPI listener or log login failures, and clients receive authentication errors
sasl.enabled.mechanisms	Lists GSSAPI on the listeners that clients use, alongside any other mechanisms you intend to run	Clients offering GSSAPI are rejected if it is missing, and a listener may accept mechanisms you did not plan to expose
sasl.mechanism.inter.broker.protocol	Set to GSSAPI when brokers authenticate to each other with Kerberos principals	Brokers cannot form the cluster or replicate if the inter-broker principal or keytab does not match the setting
sasl.kerberos.service.name	Matches the primary part of the broker service principal, usually kafka, as recorded in your KDC	Clients request a service ticket for a name the broker does not hold, so the handshake fails with a server not found style error
sasl.kerberos.principal.to.local.rules	Maps Kerberos principals to short names used in ACLs, for example when a user principal includes a realm you want to strip	Principals keep their full form, so ACLs written against short names do not match and access is denied
Client sasl.jaas.config with useKeyTab	Automated services and batch jobs that cannot hold an interactive ticket, using a keytab and principal	A wrong keytab path or principal produces login failures, and a keytab with stale keys fails after a password change
Client sasl.jaas.config with useTicketCache	Interactive users or workstations that already hold a ticket from kinit or domain login	Expired or missing tickets stop the client, and long-running producers or consumers may fail when the ticket lapses
krb5.conf realm and KDC settings	Every client and broker host, pointing at the realm and KDC or domain controllers that issue tickets	Clock skew, wrong KDC hosts or DNS-based realm lookups that fail produce ticket errors that look like Kafka faults

Common ticket and keytab problems and how to diagnose them

Most Kafka Kerberos outages trace back to a small set of failures: clocks, keytabs, host names, principal mapping, ticket lifetimes and the inter-broker path. Each card names the first check to run, so the team confirms the failing layer before editing JAAS files or regenerating keys.

Clock skew between hosts and the KDC

Kerberos rejects tickets when the clocks on a broker, a client and the KDC drift beyond the allowed tolerance, and the error often looks like a generic authentication failure. Kafka hosts, clients and domain controllers or FreeIPA servers should sync to the same time source. Check the clock offset on every broker and client before examining the keytab. First step: compare the time on each broker with the KDC.

Wrong or stale keytab after rotation

A keytab holds the long-term keys for the broker service principal, and it stops working once the key version in the KDC changes, for example after a password reset or a key rotation in Active Directory or FreeIPA. Brokers that still load an old keytab fail while others succeed, which makes the problem look intermittent. First step: list the key version numbers in the keytab and compare them with the KDC.

Canonical name mismatch in the service principal

Kerberos builds the expected service principal from the host name the client resolved, so a reverse DNS alias or CNAME can send clients to a principal that the broker keytab does not contain. The broker's sasl.kerberos.service.name and the advertised listener host must match the principal exactly. First step: confirm that forward and reverse DNS for each advertised host return the name used in the principal.

Realm mapping that yields unexpected short names

The sasl.kerberos.principal.to.local.name rules convert a full principal such as a user in a realm into the short name used for ACL checks. A rule that is too broad can map a principal from another realm to a local user with more access, while a narrow rule leaves the principal unmapped and every ACL check fails. First step: run the rule against sample principals and compare the output with the ACL principals in the cluster.

Expired ticket caches in long-running clients

Clients that obtain a ticket once at startup can fail hours later when the ticket lifetime ends and no renewal runs, especially when the Java process uses a ticket cache rather than a keytab. Producers and consumers that stay idle for long periods show the same symptom. Use useKeyTab with storeKey in the client JAAS entry so the client can obtain new tickets. First step: check the ticket expiry with `klist` on the client host and confirm that the JAAS entry renews automatically.

Inter-broker authentication fails while clients work

Client listeners can succeed while broker-to-broker traffic fails, because the inter-broker listener uses its own security protocol, principal and keytab settings. A broker that fails to authenticate to its peers can leave partitions under-replicated without an obvious client error. Review security.inter.broker.protocol and the broker's JAAS section on each node. First step: read the broker logs for the inter-broker listener name and the principal it attempted to use.

Security and operational considerations for a Kerberos-backed cluster

Kerberos moves identity checks onto the KDC, keytabs and principal mapping, so the security of the cluster depends on protecting those pieces as much as the brokers. The cards below cover what to protect, what to monitor and what Kerberos does not cover on its own.

Protect and rotate keytabs

A keytab holds the long-term key for a broker's service principal, so anyone who reads it can authenticate as that broker. Store each kafka.keytab file readable only by the broker's service account, keep it out of configuration repositories and copy it to hosts only through a controlled channel. Plan rotation as a scheduled task that issues new keys, distributes keytabs and restarts brokers in a rolling order, and confirm with your KDC administrator how old keys are retired.

The KDC as a runtime dependency

Every new client connection needs a service ticket from the KDC, so a KDC outage or slow replica can stop producers and consumers from connecting even when the brokers are healthy. Check that clients reach at least two KDC or domain controller addresses, that time synchronization is enforced across brokers, clients and the KDC, and that ticket lifetimes and renewal behavior match how long your applications stay connected.

TLS still required for confidentiality

GSSAPI authenticates principals, but the Kafka security model keeps encryption as a separate setting, and a SASL_PLAINTEXT listener sends record data unencrypted after authentication. Pair the SASL_SSL security protocol with listeners that carry production traffic, and confirm that inter-broker and controller listeners follow the same rule. Treat any plaintext listener left open for migration as an exposure to remove.

Service account hygiene

Each broker principal should be a dedicated service identity with no interactive logon, no membership in unrelated groups and no permissions beyond what the broker needs. Client principals should be separate per application, so a single compromised keytab does not expose every topic. Review which principals are super users, because resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, and that setting should stay off in production.

Monitor authentication failures

Kerberos failures often appear as generic connection errors on the client, so the broker logs and the KDC logs are the places to find clock skew, unknown principals, expired keytabs and mismatched service names. Alert on spikes in failed SASL handshakes, route broker authentication logs to your central logging system and correlate them with KDC events. Confirm which log categories and levels your Kafka version emits before you write alert rules.

Kerberos or SCRAM: how to decide for your Kafka clients

Choose Kerberos (SASL/GSSAPI) when your users and hosts already live in a realm you run, and choose SCRAM (SASL/SCRAM-SHA-256 or SCRAM-SHA-512) when the clients are services that need credentials without a key distribution center. Kafka supports both mechanisms, so the question is less about which is more secure and more about which identity system will own the lifecycle of every client principal. If your organization already runs Active Directory or FreeIPA and has staff who manage keytabs, renewals and ticket lifetimes, Kerberos reuses that investment. If your Kafka clients are mostly application services that would otherwise need a KDC, a keytab per host and a renewal process they do not have today, SCRAM is usually the smaller operational footprint. Confirm which identity source your environment treats as authoritative before you pick, because that source will hold the clients either way.

The choice also changes operations, so it should not be made on security grounds alone. Kerberos clients depend on time synchronization, DNS resolution of broker hostnames and a working KDC, and a failure in any of these shows up as a client authentication error that can look like a Kafka problem. SCRAM credentials live in the cluster metadata, so rotating a password is a Kafka administration task, but every client still has to be updated and restarted with the new secret. Both mechanisms are normally paired with TLS, so you will manage certificates in either case. Mixed listeners are possible: a broker can expose one listener for Kerberos clients and another for SCRAM clients, which lets you migrate teams gradually. Plan that mix deliberately, document which principals map to which ACLs on each listener, and test each listener with the client libraries your teams actually use before you retire any path.

Questions buyers ask about Kafka Kerberos setup

What drives the cost of a Kafka Kerberos setup?

The main drivers are how ready your realm is, how many brokers and clients need Kerberos, how many client languages and libraries you run, and whether your existing ACLs must be redesigned. A realm that already issues service principals cleanly and has documented keytab handling costs less to connect than one where the directory is being prepared at the same time. Each broker, each client group and each language adds configuration and test work. Ask any provider for a quote that lists these items separately, and ask which parts are one-time (design, build, first rollout) and which recur (credential rotation, upgrades, ticket troubleshooting). A fixed-scope quote follows from a written scope agreed before work starts.

How long does a Kafka Kerberos project take?

Duration depends mostly on how quickly your directory team can respond, not on Kafka itself. Service principals, keytabs, DNS entries and time synchronization all come from your Active Directory or FreeIPA administrators, and each request can queue behind other work. Clients also need to be moved over one group at a time, which means waiting for application owners to test. Ask for a schedule that names who must act at each step and how long each approval takes in your organization. No duration is given here because none is supplied for this work; your own directory change process is the best input for a realistic plan.

What do we need from our Active Directory or FreeIPA administrators?

The work needs named administrators who can do four things: create service principals for each broker and any shared service accounts, generate and rotate keytabs, confirm that broker hostnames resolve consistently in forward and reverse DNS, and keep clocks synchronized across brokers and clients. In Active Directory, the team also decides how user principals map to short names, which affects principal mapping rules in Kafka. In FreeIPA, the team controls the KDC and can add host principals directly. Agree early who holds keytab files, how they are stored and who approves changes. Without these decisions, most of the delay appears later as ticket errors.

Can our internal team set up Kafka Kerberos without outside help?

Yes, a team with Kafka operations experience and directory administration support can do it, because the configuration is documented by the Apache Kafka project. The difficult parts are less about syntax and more about getting principals, DNS, time sync and principal mapping right across every broker and client, then testing each client language. Teams that have not configured GSSAPI before often underestimate the number of clients and the time needed to verify them. A useful test is whether one engineer can explain how a client ticket becomes a Kafka principal and how an ACL then applies to it. If not, an outside review of the design before rollout can reduce risk.

What does a Kerberos ticket problem look like on Kafka, and how do we find the cause?

The most common symptoms are clients that cannot authenticate, brokers that refuse connections after a restart, and clients that authenticate but get denied on topics they should reach. The cause is often a keytab that does not match the service principal, a hostname that differs from the principal name, clock skew beyond what Kerberos tolerates, or a principal that maps to an unexpected short name. Enable Kerberos debug logging on a test client, check the broker's sasl.kerberos.principal setting against the keytab contents, and compare the principal the broker logs with the one your ACLs grant. Fix mapping problems in the principal-to-name rule before changing ACLs.

Does Kerberos replace our ACL design work?

No. Kerberos proves who a client is, but Kafka still decides what that identity can do. With the StandardAuthorizer in KRaft mode, ACLs are stored in the cluster metadata log, and resources without ACLs are limited to super users unless allow.everyone.if.no.acl.found is set to true. Once Kerberos is in place, the principal format changes from what your old users or clients used, so existing ACLs may match nothing. Plan the ACL redesign alongside authentication, and test each team's access before cutting over, so that a mapping change does not lock out production consumers.

Should we choose Active Directory or FreeIPA as the Kerberos realm for Kafka?

Choose the realm your organization already operates and trusts for identity, because the Kafka side changes little between the two. Active Directory suits Windows-centric environments, where users and groups already live. Microsoft notes that Linux machines must be configured to authenticate against Active Directory as a Kerberos realm or use third-party tools. FreeIPA suits Linux-centric environments and includes an MIT Kerberos KDC, and it can establish a cross-forest trust with an Active Directory domain so AD users reach Linux resources. Compare the licensing and operating costs for your own environment, since Windows Server licensing is per core with client access licenses, while FreeIPA is community software.

How DeployOpen scopes and delivers Kerberos authentication for Kafka

DeployOpen plans, builds and documents SASL/GSSAPI authentication for self-hosted Kafka clusters inside your own infrastructure, connected to the Active Directory or FreeIPA realm you already run. Each engagement is fixed-scope, with principals, keytabs, client configuration and ACL mapping agreed in writing before work starts.

How we scope

Confirm whether the identity source is Active Directory, FreeIPA with a trust, or a standalone MIT Kerberos realm
Inventory every broker, client application and tool that must authenticate, and which principal each one uses
Agree the written scope covering keytab handling, principal naming, DNS and time synchronization checks, and test environments
Record which Kafka release and Java runtime the cluster runs, since these determine supported client and broker settings

How we build

Create broker service principals and keytabs with restricted file permissions, then validate each one against the KDC
Write broker listener configuration and the client JAAS files for each application, with keytab paths kept out of shared images
Define sasl.kerberos.principal.to.local.rules so mapped short names match the ACL principals your team designed
Pair SASL/GSSAPI with TLS on every listener that carries credentials, and test both the success and denial paths

How we hand over

Deliver a runbook covering keytab rotation, ticket lifetime checks and the commands used to verify each principal
Document the mapping rules with worked examples so administrators can predict the principal an ACL will see
Provide a test record showing authentication, authorization denials and failure behavior under expired tickets
Train your operators on the troubleshooting steps for clock skew, DNS mismatches and missing service principal names

How we support

Engagements are fixed-scope, and ongoing support, upgrades and continuous testing are available as retainers
Retainer work can cover keytab and principal rotation planned around the Kafka and KDC patch schedules
Review ACL and principal mappings after new teams or applications are onboarded to the cluster
Confirm with your Kafka release notes and your identity provider vendor before each upgrade that touches SASL settings