Apache Kafka in PCI DSS cardholder data environments

Where Apache Kafka fits in a PCI DSS cardholder data environment

A team that wants card-related events on Kafka should first decide what counts as cardholder data and whether the cluster falls inside PCI DSS scope. Kafka helps with three areas: encryption of PAN in transit under Requirement 4.2.1, audit logging under Requirement 10.2.1, and parts of Requirement 7.2 through ACLs and authentication. Kafka does not make a cluster compliant on its own, and the safer design keeps primary account numbers out of topics entirely by tokenizing them upstream, so the streaming platform carries tokens rather than PAN. Before any build starts, confirm which systems store, process or transmit cardholder data, and which ones could affect the security of that environment, because those systems are the ones an assessor will examine.

PCI DSS v4.0.1 is the only active version of the standard, following the retirement of v4.0 on 31 December 2024, so design work should be mapped to v4.0.1 rather than to older versions. The PCI Security Standards Council maintains the standard but does not enforce it. The payment brands decide which entities must validate and run any enforcement programs, so the first question to put to your acquirer or payment brand is whether your organization must validate and at what level. Confirm that answer in writing before you scope the Kafka platform.

DeployOpen plans, builds and documents the Kafka platform, prepares the configuration and evidence an assessor will request, and tests the controls. DeployOpen does not issue a Report on Compliance or an Attestation of Compliance. A Qualified Security Assessor completes a Report on Compliance, and an organization eligible for a Self-Assessment Questionnaire completes it with an Attestation of Compliance. Certificates from any vendor or consultant are not accepted as validation. Agree early with your QSA or assessor which Kafka components, brokers, Connect workers and consumers they will treat as in scope, since that decision drives most of the platform design that follows.

Mapping PCI DSS requirements to Kafka capabilities and configuration

This table compares each PCI DSS requirement in scope for a Kafka cluster with what Kafka provides, the configuration that makes the control real, and what the surrounding platform must add. Kafka security is optional, so every row depends on settings you enforce and verify, and the last column is where most assessment gaps appear.

Requirement	What Kafka provides	Configuration needed	What the surrounding platform must add
4.2.1: Strong cryptography for PAN in transit	SSL/TLS listeners for client, broker and tool traffic. SASL mechanisms (GSSAPI, SCRAM-SHA-256/512, OAUTHBEARER) for authentication over those channels.	No PLAINTEXT listeners. Disable unsecured inter-broker and controller listeners. Accept only trusted certificates and enforce expiry and revocation checks.	Certificate authority process, rotation and revocation handling, cipher and protocol policy, and evidence that no client can fall back to an unsecured path.
3.5.1: Stored PAN rendered unreadable	Kafka does not render PAN unreadable in topics or metadata. In KRaft mode, sensitive data in metadata records is stored unencrypted.	Keep PAN out of topics where possible. Where PAN must persist in logs, plan disk or volume encryption for broker data directories.	Tokenization or truncation upstream, key-management processes for any strong cryptography used, and encryption of storage and backups that hold topic data.
10.2.1: Audit logs enabled and active	Broker and client activity logs through standard logging. ACL decisions are recorded by the authorizer when audit output is configured.	Enable authorizer and request logging. Confirm log level captures authentication failures and authorization denials. Do not log message payloads.	Central log collection with retention, tamper protection and time synchronization. Log sources for every broker, Connect worker and admin tool.
10.2.1.1 to 10.2.1.7: Event content and coverage	Log entries cover authentication, authorization and administrative actions when the relevant loggers are enabled.	Confirm each event type in 10.2.1.1 to 10.2.1.7 maps to an enabled logger. Test by generating each event and reviewing the output.	Mapping of each sub-requirement to a log source, alerting on the events an assessor will ask about, and documented review of those events.
7.2: Access control model and least privilege	StandardAuthorizer stores ACLs in the cluster metadata log. Resources without ACLs are limited to super users by default.	Keep allow.everyone.if.no.acl.found set to false. Grant per-topic, per-group and per-principal ACLs. Limit super users to named administrators.	Role model that maps to ACLs, approval workflow for privilege grants, and periodic access reviews for human and application accounts.
7.2.4 to 7.2.5.1: Reviews of user and application accounts	ACL listings can be exported through admin tools for review. Principals are tied to the authentication mechanism in use.	Use distinct service principals per application. Remove unused principals and ACLs on a defined cycle.	Scheduled review records, ticketed removals, and a process for application and system account privileges and their reviews.
7.2.6: Access to stored cardholder data queries	Kafka does not provide query access to stored data. Consumer access is governed by topic ACLs.	Restrict read ACLs on topics holding card data. Separate consumers that need PAN from those that need only tokens.	Access control for downstream stores fed by Connect or stream processors, since those copies are in scope too.

Keeping PAN out of topics with tokenization upstream

The cleanest way to shrink the cardholder data environment around a Kafka platform is to make sure primary account numbers never enter a topic. Tokenize or truncate the PAN before the event reaches the broker, in the service that first receives card data, so that the producers, brokers, Connect workers and consumers handle only values that cannot be used to charge a card. This decision belongs to the card data architecture, not to the streaming team alone, so the people who own the payment flow should sign off on which fields are allowed on which topics. Requirement 3.5.1 accepts several ways to render stored PAN unreadable: one-way hashes of the entire PAN based on strong cryptography, truncation, index tokens, or strong cryptography with key-management processes. Each option has different reversibility, reporting and key-management consequences, so confirm with your Qualified Security Assessor which option your design relies on before you build the pipeline around it.

Kafka does not render PAN unreadable by itself. Records are stored in plain form on the broker disk, and KRaft metadata is stored unencrypted, so encryption at rest must come from the disk, volume or application layer. If a PAN does reach a topic, the brokers, every Connect worker that reads or writes the topic, every stream processor, and every downstream sink, including backups and replicas made by MirrorMaker 2, fall within the scope of the assessment. Illustratively, a single unmasked field in a payment event can pull a analytics cluster that was designed as out of scope into the assessment. Ask your engineers to run a field-level inventory of every topic and to reject schemas that carry PAN fields, and enforce that rule in the schema registry or in producer code review. Where a field is needed for matching, decide whether a token generated by the card vault suffices.

For guidance on whether sending PAN over messaging channels is permitted at all, read the PCI SSC FAQ on sending PAN over email or messaging before you finalize the design. Treat that FAQ as the next read, then validate your conclusion against the official assessment template with your assessor. Confirm with your vendor and your own environment whether any detokenization service, card vault or Connect connector in your chosen design touches cleartext PAN, because that component will carry the strongest scope and access control requirements.

Encryption in transit and at rest for Kafka under Requirement 4.2 and 3.5

Requirement 4.2.1 expects strong cryptography and security protocols for PAN sent over open, public networks, and Kafka's SSL/TLS support is the mechanism you will rely on for client, broker and tool traffic. Security in Kafka is optional, so a cluster can run with plaintext listeners alongside secured ones. Any listener that can carry cardholder data must be configured for TLS, and the configuration should be reviewed for listeners that were added for convenience during testing.

Every listener: Inventory each broker listener, controller listener, inter-broker listener, Kafka Connect REST endpoint and tool connection, and confirm that none accepts plaintext for traffic that carries PAN.
Certificate trust: Accept only certificates issued by trusted internal or public certificate authorities, and restrict truststores to the certificates you intend to trust.
Certificate validity: Monitor expiry and revocation status, and document how revoked certificates are detected, since the requirement expects them to be confirmed valid and not expired or revoked.
No fallback: Disable legacy protocol versions and weak cipher suites in broker and client settings so that a handshake cannot negotiate down to an insecure configuration.
Mutual TLS and SASL: Pair TLS with authenticated client identities so the connection is both encrypted and attributable.

Confirm the protocol and cipher settings against the current PCI SSC guidance and your assessor's expectations before you fix them in configuration, because those expectations can differ from one assessment to the next.

Configuration sequence for a PCI DSS scoped Kafka cluster

This sequence takes a Kafka cluster from scope definition to a configured, monitored and scoped deployment, so each PCI DSS control area has a concrete setting or evidence artifact before go-live. It is a build order for engineers, not a statement that any step makes the environment compliant.

Define the CDE boundary and data flows
Before any broker is installed, the team maps every producer, consumer, Kafka Connect worker, schema component and monitoring tool that touches topics, and marks which of them handle cardholder data or could affect the cardholder data environment. The output is a written data flow diagram and a system inventory that names each topic expected to carry PAN, sensitive authentication data or related identifiers. Ask your QSA or assessor to review this boundary early, because every later setting is judged against it. The step is done when the inventory has an owner for each system and each flow.
Enforce tokenization upstream of Kafka
Design the tokenization or truncation point in the systems that first receive card data, so that events published to Kafka carry tokens or masked values instead of PAN. Confirm which approach your design uses for stored PAN under Requirement 3.5.1, such as index tokens or strong cryptography with key management, and document where detokenization happens and who can call it. Test a sample of produced messages against a PAN detection scan before any topic is opened to producers. The output is a tested producer contract and a scan result that shows no clear PAN in the test topics.
Deploy KRaft on a supported 4.x release
Install Kafka in KRaft mode on one of the supported releases listed on the downloads page: 4.3.1, 4.2.2 or 4.1.2. Confirm the release against the downloads page on the day of installation, since patch releases change, and record the exact version in your configuration management. Brokers and tools need Java 17 for Kafka 4.x. Verify the cluster with a metadata quorum check and a test topic before adding any configuration beyond the defaults. The output is a running KRaft quorum with a recorded version and JVM baseline.
Enable TLS on every listener
Configure SSL listeners for client, inter-broker and controller traffic, and remove any PLAINTEXT listener from the broker and controller configuration. Kafka security is optional by default, so the removal must be explicit and checked, not assumed. Set the protocol versions and cipher suites your security team approves, and make sure clients reject unexpected certificates. Test a client connection with a plaintext attempt and confirm it is refused. The output is a configuration file with no plaintext listener and a connection test log showing the refusal.
Configure SASL or mTLS identities for each workload
Give each application, Connect worker, tool and administrator its own identity, using either mutual TLS certificates or one of the documented SASL mechanisms, such as SCRAM-SHA-512 or OAUTHBEARER, depending on what your identity platform supports. Avoid shared accounts so that ACLs and audit events can be traced to a single principal. Store credentials in your secrets manager and set certificate lifetimes and rotation steps in writing. The output is an identity register listing each principal, its mechanism and its owner, with a successful authentication test per principal.
Set StandardAuthorizer with default-deny ACLs
Enable the StandardAuthorizer, which stores ACLs in the KRaft metadata log, and confirm that allow.everyone.if.no.acl.found is not set to true, so resources without an explicit ACL are restricted to super users. Keep the super user list minimal and documented. Grant each principal only the operations on the topics and consumer groups it needs, then test a denied produce or consume for a principal outside its grant. The output is an ACL export per topic and group, plus denial test results that an assessor can reproduce.
Encrypt broker disks and volumes
Kafka stores sensitive values in KRaft metadata records without encryption at the application level, so protection at rest comes from the platform. Choose disk, volume or filesystem encryption on every broker and controller data directory, with keys held in a key management service you control. Confirm that snapshots and backups of these volumes inherit the same protection. Record the key custody model and the rotation procedure. The output is an encryption configuration per host and a restore test that shows the data is unreadable without the key.
Route audit logs to a central store
Send Kafka server logs, authorizer decisions and authentication events to a central log platform that sits inside the same scoped environment, with access restricted to the security and operations roles. Map the captured events to the Requirement 10.2.1 sub-requirements your assessor expects, and confirm that log retention and time synchronization meet your policy. Test that a denied ACL event appears in the central store with the principal name. The output is a logging configuration, a dashboard or saved query for authorization denials, and a sample event trail for review.
Plan MirrorMaker 2 only inside the scoped environment
Plan cross-cluster replication only between clusters that are both in scope, and only over TLS with the same identity and ACL controls as the source cluster. MirrorMaker 2 replicates topics, configuration, consumer group offsets and ACLs, so a replica outside the boundary would move cardholder data outside the scope you defined. Document the target location, the network path and the data the replicated topics carry. Ask your assessor whether any replica changes the boundary. The output is an approved replication design that names every cluster and topic in the flow.

Evidence an assessor expects from the Kafka platform

An assessor will ask for proof, not descriptions. The cards below name the artifacts a self-hosted Kafka platform should produce for requirements 4.2, 3.5, 7.2 and 10.2.1, and how each artifact stays under change control so it reflects the cluster as it runs today.

TLS configuration and certificate inventory

Keep a versioned record of every listener that carries cardholder data, the protocol and cipher settings applied to it, and the certificates behind it, including issuer, subject, and expiry date. Export listener configuration from each broker and compare it against the approved baseline on a schedule. Confirm that no plaintext or unsecured listener is exposed to a network path that carries PAN, because Kafka security is optional and must be enforced by configuration. Track expiry and revocation status for every certificate so renewals are scheduled before they lapse.

ACL exports and access reviews

In KRaft mode the StandardAuthorizer keeps ACLs in the metadata log, so export the ACL list from the cluster on a fixed cadence and store each export as a dated artifact. Verify that resources without ACLs are denied unless allow.everyone.if.no.acl.found has been deliberately set to true, and record that setting in the baseline. Use the exports as the input for user access reviews under Requirement 7.2, with each grant traced to an approver and a business owner.

Authentication configuration per client identity

Document which SSL or SASL mechanism each client, broker and tool uses, such as SCRAM-SHA-512 or OAUTHBEARER, and bind each identity to one owner and one purpose. Keep the listener-to-mechanism mapping in version control so a reviewer can see when a shared or generic account was introduced or removed. Where service accounts exist, record how their credentials are issued, rotated and revoked, since these accounts are in scope for the application and system account reviews.

Audit logs covering the 10.2.1.x events

Retain broker and tool logs that show authentication attempts, authorization decisions, administrative changes to topics, ACLs and configuration, and access to cardholder topics. Map each log source to the specific sub-requirement it satisfies in 10.2.1.1 through 10.2.1.7, and confirm that logging stays enabled and that log integrity is protected. Kafka does not produce all of this evidence on its own, so document the forwarding pipeline and its retention settings as part of the same record.

Disk encryption records

In KRaft mode, sensitive values in metadata records are stored unencrypted, so protection at rest depends on the disk, volume or application layer. Record which volumes hold log directories and metadata directories, the encryption method used, the key custodian, and the rotation schedule. Keep the volume build records and any key-management configuration under change control so an assessor can trace encryption from the hardware to each broker.

Change records for upgrades and releases

Record each Kafka version change with the release notes reviewed, the upgrade path followed, the test results and the approver. Only supported releases receive bugfix updates, so tie each upgrade to a current version, and check the upgrade notes before moving between major lines. Store change tickets, configuration diffs and rollback plans together so a reviewer can reconstruct the cluster state at any date.

Questions teams ask before running Kafka in cardholder data scope

Can Kafka be kept out of PCI DSS scope?

Kafka can sit outside scope only if no cardholder data, meaning no PAN or sensitive authentication data, ever reaches its topics, brokers, Connect workers, logs or backups, and if nothing in the cluster could affect the security of the cardholder data environment. PCI DSS scope is set by data flows and connected systems, not by the product name. The practical test is whether tokenization happens upstream, whether topic payloads and debug logs are checked for PAN, and whether the network paths to the cluster are segmented from the cardholder environment. Document that reasoning in the scope statement and have your assessor confirm it, because the assessor, not the platform team, decides whether scope boundaries hold.

What does a Kafka PCI DSS effort cost, and what drives it?

Kafka itself is open source under the Apache License 2.0, so the cost is the labor and infrastructure around it. Expect the cost to come from four drivers: how many data flows carry card data, how many clusters you run and whether they are separated by environment or region, how much of the surrounding platform (identity, secrets, logging, backups, key management) must be built or evidenced, and which assessment route applies. One-time costs include scoping, architecture, configuration, testing and evidence collection. Recurring costs include access reviews, certificate renewal, patching, log review and the annual assessment itself. When you request a quote, ask for the written scope, the list of data flows and systems in scope, the deliverables, and which recurring activities are included.

How long does scoping and configuring Kafka for cardholder data take?

Duration depends on how many topics, producers, consumers and connectors touch card data, how mature your identity and certificate processes already are, and how much evidence your assessor needs. No general timeline applies, so ask any implementation partner for a schedule tied to a written scope and to named milestones. The phases to confirm are data-flow mapping, tokenization validation, cluster configuration for encryption and access control, logging and review setup, and a dry run of the evidence package. Ask whether the schedule assumes your assessor's availability and whether it covers the first assessment cycle or only the build.

Can our own team run this without outside help?

Yes, if your team already has the skills to configure TLS, SASL, ACLs and log pipelines and can document them for an assessor. Teams often struggle less with Kafka settings than with the surrounding evidence: proving that PAN never enters topics, showing reviewed access lists, and keeping certificate and key records current. Check whether you have engineers who can write the authorization model, someone who owns the data-flow inventory, and a process for reviewing access on a schedule. An outside partner can take on fixed-scope parts such as configuration, testing or evidence preparation, and DeployOpen prepares systems and evidence for assessments but does not issue the assessment itself.

What must our team provide before the work starts?

Prepare the inputs an assessor and an implementer will both ask for first: an architecture diagram showing brokers, Connect workers, clients, MirrorMaker links and network zones; a data-flow inventory marking where PAN could appear and where tokenization occurs; an access list covering people, service accounts and the roles they hold; and the certificate authority arrangement, including who issues and rotates broker and client certificates and where private keys are stored. Also provide your current logging destinations, backup locations, and the identity provider used for human access. Missing any of these usually extends the scoping phase more than the Kafka configuration itself.

Do we need a QSA, or can we complete an SAQ?

The payment brands decide which entities must validate and how, and the PCI Security Standards Council does not enforce compliance itself. Assessments take one of two documented forms: a Report on Compliance completed by a Qualified Security Assessor, or a Self-Assessment Questionnaire, each with an Attestation of Compliance. Ask your acquirer or the relevant payment brand which validation level and form applies to your organization, since that answer depends on transaction volume and how card data flows. A Kafka platform that carries PAN generally expands scope, so confirm the route before designing the architecture rather than after.

How does the customized approach differ from the defined approach?

The defined approach follows each requirement as written, using the testing procedures the standard lists. The customized approach, available in PCI DSS v4.x to entities with robust security and risk management, meets the stated objective of a requirement through a different control design, and it requires documented analysis and evidence that the objective is met. For Kafka, an example could be a different method of protecting PAN in messages, but any customized design is a decision your assessor must accept, so agree on the approach and evidence early. The customized approach is not a shortcut, and it is not the same as a compensating control.

What Kafka and DeployOpen cannot cover for you

Kafka secures the transport and the access path to topics, but a PCI DSS assessment covers the whole cardholder data environment around it. The cards below mark the controls that sit outside the Kafka cluster, which your team owns, and the formal assessment that only an independent assessor can issue.

Tokenization and the card vault

Kafka should carry tokens, not primary account numbers. The system that issues tokens, stores the mapping and controls detokenization sits upstream and must be designed, scoped and operated by your team or a payment provider. Before go-live, confirm which services can call the vault, that producers never receive raw PAN, and that schemas reject card-number patterns. A topic that carries PAN brings the whole cluster into scope.

Key management processes and HSM use

Kafka does not encrypt sensitive data in KRaft metadata records, so disk, volume or application encryption must supply that protection. Keys used for TLS, disk encryption and any application-level encryption need generation, storage, rotation, split custody and retirement procedures. A hardware security module or managed key service may hold those keys, but the key-management process and its evidence remain your responsibility. Ask your assessor which key controls apply to your design.

Log retention and review program

Kafka brokers and clients produce logs, but Kafka does not decide what to keep, for how long, who reviews them, or how reviews are recorded. Your team must send authentication, authorization and administrative events from brokers, Connect workers and clients to a central store, define retention, and run a documented review process. Confirm that the events you capture match the audit log requirements you are assessed against, and that log integrity is protected.

Network segmentation and firewalls

Brokers, controllers, Connect workers, schema services and client networks form part of the cardholder data environment if they carry or could affect card data. Segmentation must isolate these components from out-of-scope networks, restrict broker listener ports to approved source ranges, and document every allowed flow. Kafka's listener configuration does not replace firewall rules. Your network team owns the rule sets and the periodic segmentation tests that show the boundary holds.

Vulnerability and patch management

The Kafka project publishes supported releases, but the operating system, the Java runtime, container base images and supporting services are patched by your team. Track the Kafka release line you run against the supported list, apply bugfix releases, and keep JVM and OS patch records as evidence. Scan hosts and images on a schedule, and retain proof that critical findings were fixed or risk-accepted. Confirm your patch window and approval workflow before the cluster enters the cardholder data environment.

The formal assessment itself

Only a Qualified Security Assessor, through a Report on Compliance, or your entity, through the applicable Self-Assessment Questionnaire, can conclude that requirements are met. Each route ends with an Attestation of Compliance. DeployOpen prepares systems, configurations, scope diagrams and evidence for that assessment, but does not issue ROCs, AOCs or certifications. Ask your acquirer or payment brand which validation level applies, and agree the evidence list with your assessor before testing begins.

Keeping the design current as PCI DSS and Kafka change

PCI DSS v4.0.1 is the only active version of the standard, and the PCI SSC had announced no release date for a successor as of October 2026. Treat the requirement mapping for this Kafka design as tied to v4.0.1 for now, and schedule a review when the next iteration is published. The PCI SSC ran a request for comments on v4.0.1 from 3 June to 20 July 2026 to start that work, so the review trigger is a published draft or final version, not the comment period itself. Until then, confirm the current version and any announced effective dates directly on the PCI Data Security Standard page before each assessment cycle, and record the version your Report on Compliance or SAQ was written against.

Kafka moves faster than PCI DSS does, and that pace drives most of the maintenance work. The Kafka project aims for three releases a year, and the supported lines listed on the downloads page are 4.3.1, 4.2.2 and 4.1.2. Rolling upgrades are supported from each release in the past year to the latest version. Each upgrade can change TLS defaults, authorizer behavior, client settings or logging output, so plan it against the mapping for Requirements 4.2.1, 7.2, 10.2.1 and 3.5.1 rather than treating it as a routine patch. Re-run your TLS protocol and cipher checks, confirm that allow.everyone.if.no.acl.found is still false, and re-collect audit log samples afterward so the evidence for the assessment period stays valid.

If any cluster still runs on ZooKeeper, that migration comes first. Apache Kafka 4.0 supports only KRaft mode, and ZooKeeper-based clusters must migrate to KRaft before upgrading to 4.x. The Kafka upgrade notes also recommend that KRaft clusters older than 3.3.x move to 3.9.x before going to 4.x. Those steps change the controller quorum, the metadata store and the authorizer configuration, so they need their own change record, test run and evidence of the post-migration ACL state. Also confirm the Java runtime requirement for brokers, Connect and tools, which is Java 17 since Kafka 4.0, and check the Kafka upgrade notes for the exact target release before scheduling the change.

How DeployOpen scopes and delivers PCI DSS ready Kafka configuration

DeployOpen plans, builds and documents self-hosted Apache Kafka inside infrastructure you control, with the cardholder data environment boundary agreed before any configuration starts. The work covers Kafka settings, the surrounding identity, logging and key management, and the evidence your assessor will ask to see.

How we scope

Confirm which topics, producers, consumers and connectors carry PAN, and which only carry tokens or masked data.
Map the cardholder data environment boundary, including brokers, Connect workers, MirrorMaker 2 links and admin tools.
Agree the assessment context in writing, including which PCI DSS version and SAQ or ROC approach applies, confirmed with your acquirer or assessor.

How we build

Configure TLS on every client, broker and tool listener, and disable unsecured listeners in cardholder-data paths.
Set SASL or mutual TLS identities per service, with ACLs in the KRaft metadata log and no reliance on open default access.
Design encryption at rest for disks or volumes, since KRaft metadata is stored unencrypted, and test audit log capture for access events.

How we hand over

Deliver a written configuration baseline, network diagram and data flow map that match the agreed scope.
Provide a control-to-evidence matrix showing which configuration files, logs and access reviews support each relevant requirement.
Document key rotation, certificate renewal, access review and log retention procedures for your operations team to run.

How we support

Offer ongoing support, upgrades and continuous testing as a retainer after handover, with the scope written in advance.
Plan Kafka upgrades around the supported 4.x release line and retest TLS, authorization and logging after each change.
Re-check the cardholder data environment boundary when new topics, connectors or sites are added, and update the evidence set to match.