How Apache Kafka supports ISO 27001 controls on infrastructure you run
A CISO or platform lead who must show ISO/IEC 27001:2022 controls for a Kafka cluster carrying business or regulated data needs to know one thing first: Kafka helps with Annex A controls 8.24 Use of cryptography, 8.15 Logging, 8.16 Monitoring activities and 8.13 Information backup, but it does not cover them alone. Each control is met by a combination of Kafka settings, the operating platform around the cluster, and written procedures that someone runs and records. This page covers those four controls only. It does not describe a full information security management system, risk assessment, statement of applicability or the other 89 Annex A controls, which your organization still has to address.
Annex A is normative and lists 93 controls in four themes taken from ISO/IEC 27002:2022: organizational, people, physical and technological. The four controls discussed here sit in the technological theme, clause 8. ISO does not certify organizations. Accredited certification bodies do that work, and they audit against your own documented ISMS and the controls you have selected. DeployOpen plans, configures and documents self-hosted Kafka in your infrastructure and prepares the systems, settings and evidence that an assessor will ask for. DeployOpen does not issue audit reports, attestations or certificates, and the readiness work described here does not replace an independent assessment.
Before scoping the work, confirm three points with your certification body and internal audit: which Annex A controls appear in your statement of applicability, whether your clusters fall inside the certified scope, and which edition of the standard your certificate references. Transition deadlines under IAF MD 26:2023 have passed for ISO/IEC 27001:2013 certificates, so your evidence should map to the 2022 control set. The sections that follow take each of the four controls in turn, starting with what Kafka's security overview says about encryption in transit and what it leaves to you.
Mapping controls 8.24, 8.15, 8.16 and 8.13 to Kafka capabilities and configuration
This table pairs each Annex A control in scope with the documented Kafka capability, the configuration it needs, and what the surrounding platform must still provide. Read each row as a starting point for the control design, not as evidence that the control is met, because an assessor will test the whole environment. The main takeaway is that Kafka provides transport encryption, authentication, authorization and cross-cluster replication, but encryption at rest, log retention, monitoring and tested restores come from the platform around it.
| Control | Requirement in brief | Documented Kafka capability | Configuration and platform gaps |
|---|---|---|---|
| A.8.24 Use of cryptography | Define and implement rules for cryptography, including key management | SSL/TLS for client, broker and tool traffic; SASL mechanisms GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER | Security is optional, so disable plaintext listeners and require TLS on every listener. Storage encryption and key lifecycle must come from disk, volume or application encryption, because KRaft stores sensitive metadata unencrypted. |
| A.8.15 Logging | Produce, store, protect and analyze logs of activities, exceptions and faults | Broker and client activity can be logged; ACL decisions are enforced by the StandardAuthorizer in the metadata log | Confirm which authorizer and request logs your version emits, then ship them to a protected central store. Keep ACL change records and review denied requests, since Kafka does not provide that workflow by itself. |
| A.8.16 Monitoring activities | Monitor networks, systems and applications for anomalous behavior and act on it | Exposes operational metrics and authorization decisions that can feed external monitoring systems | Alert on authentication failures, unexpected principals, ACL changes and replication lag. Confirm the metric names for your release in the Kafka documentation before writing alert rules. |
| A.8.13 Information backup | Maintain backup copies of information and systems and test them regularly | MirrorMaker 2 replicates topics with configuration, consumer group offsets and ACLs to another cluster | Replication is not a backup by itself, since deletes and corruption can replicate too. Add point-in-time copies of topic data and KRaft metadata, and test restores on a schedule defined in your backup policy. |
Which Kafka settings serve each control
Control 8.24 asks you to define and implement rules for cryptography, including key management. Kafka supplies the mechanisms (SSL/TLS for client, broker and tool traffic, and SASL for authentication), but security is optional in the software, so a cluster can run with plaintext listeners unless you disable them. A secure configuration means every listener that accepts clients, brokers or tools uses TLS, inter-broker traffic is encrypted, and no plaintext listener is advertised to applications.
- Mandatory TLS on every listener: Define listeners with SSL or SASL_SSL only, and confirm that no PLAINTEXT listener remains in
listenersoradvertised.listenerson any broker or controller. - Certificate and key handling: Set a documented issuance, rotation and revocation process for broker and client certificates, with private keys kept in a secrets store and never committed to configuration repositories.
- Authentication alongside encryption: Pair TLS with SASL mechanisms such as SCRAM-SHA-512 or OAUTHBEARER so that encrypted connections also identify the caller.
- Metadata at rest: In KRaft mode, sensitive values in metadata records are stored unencrypted, so encrypt the disks or volumes that hold the metadata and log directories, or apply application-level encryption to sensitive payloads.
- Key custody records: Keep an inventory of keystores, truststores and key owners, which is the evidence an assessor will ask for.
Before the assessment, confirm with your security team which algorithms and key lengths your policy requires, and test that a client with a plaintext configuration is refused.
Control 8.15 asks that logs of activities, exceptions, faults and other relevant events are produced, stored, protected and analyzed. Kafka writes broker and controller application logs and records authorization decisions, but the set of events you need, where they are shipped, and how long they are kept are platform decisions. Start by listing which events matter to you: authentication failures, ACL denials, topic and ACL changes, and broker or controller restarts.
- Authorization decisions: Enable logging for the authorizer so that denied and permitted access attempts are recorded with the principal and resource.
- Administrative changes: Capture changes to topics, configurations, ACLs and quotas, which the cluster metadata reflects.
- Central collection: Ship broker and controller logs to a log platform outside the Kafka hosts, with write access restricted to the logging pipeline.
- Protection of log stores: Limit who can alter or delete stored logs and record who reads them.
- Retention and review: Set retention periods in your policy and document who reviews the events and how often.
Confirm with your log platform team which events reach the central store, then run a test action such as a failed login and check that the entry appears there.
Control 8.16 covers monitoring activities, so that anomalous behavior is detected and assessed. Kafka exposes metrics through JMX, and the signals that matter for security and availability include under-replicated partitions, offline partitions, broker availability, authentication failures and unexpected consumer group changes. Metrics alone do not satisfy the control, so define thresholds, owners and the response for each alert.
- Availability signals: Alert on offline partitions, under-replicated partitions and controller changes, since these show both faults and possible tampering.
- Access signals: Alert on repeated authentication failures and authorization denials for sensitive topics.
- Configuration drift: Track changes to listeners, ACLs and topic configuration against an approved baseline.
- Alert handling: Route alerts to a monitored queue with an on-call owner and keep the record of each investigation.
- Collection path: Export JMX metrics to your monitoring system and confirm that the monitoring system itself is covered by backup and access controls.
Before the audit, check that a deliberate test, such as stopping a broker in a non-production cluster, produces the alert and the recorded response.
Control 8.13 asks that backup copies of information, software and systems are maintained and regularly tested. Kafka replication keeps several copies of each partition across brokers, and MirrorMaker 2 can replicate topics, consumer group offsets and ACLs to another cluster. Replication protects against broker loss, but it is not a backup: a bad deletion or a corrupted message is copied to every replica. Treat replication as one input and keep separate, restorable backups of configuration, certificates and metadata.
- Replication settings: Set replication factors and minimum in-sync replicas in line with your recovery needs, and document the choice.
- Cross-site replication: Use MirrorMaker 2 to a second cluster for site-level failure, and check which topics, offsets and ACLs it replicates in your version.
- Configuration and certificates: Back up broker configuration, keystores, truststores and ACL definitions outside the cluster.
- Restore tests: Restore a topic and a consumer group offset in a test environment on a schedule, and record the results.
- Recovery targets: Agree recovery time and recovery point objectives with the business owners before choosing the replication design.
Confirm the restore procedure with your operations team and treat a successful restore test, not the existence of a replica, as the evidence the control requires.
Evidence an auditor will ask for from a Kafka platform
An auditor assessing Kafka against Annex A will ask for evidence that each control operates, not only that settings exist. The cards below list the artifacts for cryptography, access, logging, monitoring and backup, and each one must be kept current and versioned by your operations team.
Cryptography configuration evidence
Keep the broker and client listener definitions as versioned files, along with a certificate inventory that lists each listener, issuer, validity period and owner. Kafka's security is optional, so an auditor will look for any plaintext listener and for the rule that governs which listeners may exist. Record how keys are generated, rotated and revoked, because control A.8.24 expects defined key management rules, not only working TLS. Confirm each listener is secured before you export the configuration.
ACL exports and authorizer setting
Export the ACL set from the cluster on a fixed schedule and store each export with the date and the approver of any change. Capture the authorizer configuration alongside it, including whether the StandardAuthorizer is active and whether allow.everyone.if.no.acl.found is set. An auditor will test that resources without an explicit grant are denied, so the export and the setting must agree.
Retained broker and authorizer logs
Keep broker server logs and authorizer logs for the retention period your policy defines, and ship them to storage that operators cannot silently edit. Auditors will sample events, such as a denied access attempt or a configuration change, and ask you to show the record end to end. Confirm where logs are collected, who can read them and how access to the log store is itself logged.
Monitoring alert rules and review records
Export the alert rules that watch broker health, under-replicated partitions, authentication failures and certificate expiry. Pair them with dated review records showing who looked at alerts, what they found and how each finding was closed. Control A.8.16 is judged on whether monitoring is acted on, so an alert rule with no review history will not satisfy an auditor on its own.
Replication settings and restore tests
Keep the replication configuration for each cluster, including replication factor and minimum in-sync replicas, along with the MirrorMaker 2 settings that copy topics, offsets and ACLs to a second site. Restore records matter most: store dated test results showing that topics and consumer offsets were recovered and checked. Control A.8.13 expects backups to be tested regularly, so a replication setting without a restore test record is incomplete evidence.
Why an ISO 27001 certificate comes from a certification body, not from the platform
No platform, Kafka included, can issue an ISO/IEC 27001 certificate. Certification is granted by an accredited certification body, and those bodies operate to ISO/IEC 17021-1:2015 together with ISO/IEC 27006-1:2024, which IAF MD 29:2024 required them to apply to all clients by 31 March 2026. For your Kafka work, this means the evidence you produce for controls 8.24, 8.15, 8.16 and 8.13 is an input to an auditor's judgment, not a certificate in itself. Confirm with your chosen certification body which version of these requirements applies to your audit plan, and ask how it will sample your Kafka configuration, access records and restore tests.
The certification cycle runs three years, and ISO/IEC 17021-1 requires at least one audit, either surveillance or recertification, in every calendar year of that cycle. A Kafka platform therefore needs to keep its evidence current throughout the cycle, not assemble it once before the first audit. Ask the certification body how it will schedule these visits and which records it expects on each one. Also note the transition under IAF MD 26:2023: all certifications to the 2013 edition had to expire or be withdrawn by 31 October 2025, so any Kafka documentation still mapped to the old clause and control numbering should be updated to the 2022 Annex A.
The 2024 climate-related amendment (ISO/IEC 27001:2022/Amd 1:2024) changed clauses 4.1 and 4.2 rather than the Annex A controls. Clause 4.1 now asks the organization to determine whether climate change is a relevant issue, and clause 4.2 notes that interested parties can have climate-related requirements. Your context and interested-party records should reflect your own determination. On the accreditation side, Global ACI has taken over the roles of the International Accreditation Forum (IAF) and ILAC since 1 January 2026, while ISO still points to the IAF CertSearch database for checking accredited certificates, so use that database to verify any certification claim made by a vendor or partner.
What the platform does not cover and who fills each gap
Kafka covers part of the technical evidence for ISO/IEC 27001:2022 Annex A controls 8.24, 8.15, 8.16 and 8.13, but several requirements sit outside the broker. Each card names the gap, who usually owns it, and what to confirm before you rely on Kafka for that control.
Key lifecycle and rotation
Kafka uses TLS certificates, SASL credentials and, if you encrypt data at rest, keys that the broker does not manage for you. Your key management process (issuance, storage, rotation, revocation and destruction) sits with your security or platform team and any secrets or PKI service you run. Confirm which keys Kafka consumes directly, how a rotated certificate is loaded without a full cluster restart, and who approves each rotation.
Log shipping and tamper protection
Broker and authorizer logs are written to local files by default, and nothing in Kafka makes them tamper-evident. Platform engineering usually owns the forwarding pipeline, retention and write-once storage for those logs. Confirm which events (authentication failures, ACL changes, topic configuration changes) reach your central log store, how long they are kept, and who can alter or delete them.
Anomaly monitoring and alerting
Kafka exposes broker and client metrics, but metrics alone do not tell you that an unfamiliar principal is producing to a sensitive topic at night. A security operations team typically owns detection rules, correlation across identity and network logs, and on-call response. Confirm which authentication and authorization events feed the detection system, which alert thresholds were agreed, and who receives and acts on each alert.
Backup of configuration and metadata
Replication protects topic data across brokers, but it does not protect the cluster configuration, ACL definitions, client credentials or KRaft metadata. Platform operations owns the backup schedule and the restore runbook. Confirm which configuration artifacts are exported and stored outside the cluster, and schedule restore tests that rebuild a cluster from them and record the results, since A.8.13 asks for backups that are regularly tested.
Unencrypted sensitive metadata in KRaft
In KRaft mode, Kafka stores sensitive values in metadata records without encrypting them, so anyone who can read the metadata log or its disks can read those values. Protection at rest therefore comes from disk, volume or application-level encryption, and access to the storage layer falls to your infrastructure team. Confirm which values in your configuration are sensitive, which volumes hold metadata, and how access to those hosts is restricted and logged.
The ISMS itself
Kafka settings do not create an information security management system. Policy, risk assessment, the Statement of Applicability, internal audit and management review belong to the organization, usually under the CISO and the management team. Confirm that each Kafka control is traced to a documented risk treatment decision, that its owner is named, and that the internal audit programme samples its evidence before an external certification body does.
Configuring a Kafka platform for ISO 27001 evidence
This sequence sets up a self-hosted Kafka cluster so that its cryptography, logging, monitoring and backup settings can be shown to an ISMS auditor. Each step ends with an artifact that goes into the evidence register, and the steps that need your own change approvals are marked.
Confirm the supported release and KRaft mode
Platform engineering selects a release from the supported lines on the Kafka downloads page: 4.3.1, 4.2.2 or 4.1.2. Confirm the cluster runs in KRaft mode, since Kafka 4.0 supports only KRaft and ZooKeeper-based clusters must migrate before upgrading to 4.x. Record the exact version, the Java runtime (Java 17 for brokers and Connect) and the upgrade notes reviewed. Done when the version and mode are written into the configuration baseline and the change record shows your approval for the chosen release. Your change board approves the version choice.
Define cryptography rules and key management policy
The security or compliance lead writes the topic-specific rules that A.8.24 expects: which protocols and cipher suites are permitted for client, broker and tool traffic, who owns the keystores and truststores, how certificates are issued, rotated and revoked, and where keys are stored. Note that KRaft stores sensitive metadata records unencrypted, so the policy must name the disk, volume or application encryption that covers data at rest. Done when the approved policy document exists, has an owner and a review date, and the cluster's certificate inventory matches it.
Enable TLS on every listener and retire plaintext
The platform team configures SSL or SASL_SSL on each broker, controller and inter-broker listener, using the certificates defined in the policy. Kafka security is optional, so remove any PLAINTEXT listeners and confirm no client still connects without encryption by checking broker logs for unsecured connections during a monitored window. Tools such as the CLI and Connect workers must also use TLS. Done when a scan of the listener configuration shows only encrypted endpoints and the test report from a client connection attempt on a plaintext port fails. Changing live listeners needs a planned maintenance approval.
Select the SASL mechanism and authorizer
Choose the authentication mechanism from the documented options: GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 or OAUTHBEARER, based on your identity provider. Enable the StandardAuthorizer, which stores ACLs in the cluster metadata log, and grant each principal only the topic, group and cluster permissions it needs. Set allow.everyone.if.no.acl.found to false so resources without an ACL are limited to super users, or record in the design why you are confirming its effect with your own tests. Done when an access review export lists every principal with its ACLs and the test shows an unlisted principal is denied.
Route broker logs and metrics to protected storage
The operations team sends broker server logs, controller logs, authorizer audit output and the JMX metrics for authentication failures and ACL denials to a central log platform. Set retention periods based on the logging requirement in your own policy, and restrict write and delete rights on the log store to a separate administrator group. Protect the log stream in transit using the TLS configuration from earlier. Done when a test event such as a failed login appears in the central store with its timestamp and broker identity, and the access list for that store is attached to the evidence register.
Set alerts on security and availability signals
Operations defines alert rules for the monitoring control in A.8.16, covering under-replicated partitions, offline partitions, controller changes, authentication failures, certificate expiry and disk usage. Route alerts to a named on-call rota and document the response steps for each. Trigger one controlled test alert, such as a broken replica in a non-production cluster, and confirm it reaches the rota. Done when the alert rule definitions, the rota and the test alert record are stored together. The exact thresholds should be set against your own baseline, not copied from defaults.
Configure disk or volume encryption
Infrastructure engineers enable encryption at the layer your policy names: full-disk or volume encryption on broker and controller nodes, or application-level encryption for payloads where the design requires it. Confirm the key custody for disk encryption sits outside the Kafka hosts and that the keys can be recovered during a restore. Verify on one node that the data directory is encrypted and that the boot process requires the key source. Done when the encryption configuration, key custody record and verification output are in the evidence register.
Set up MirrorMaker 2 and run restore tests
Configure MirrorMaker 2 to replicate topics, consumer group offsets and ACLs to a second cluster in a location you control, following the geo-replication guidance for the chosen release. Then run a scheduled restore test: stop a test consumer, fail over to the replica, confirm the data and offsets are present and record the measured recovery time. Repeat the test on the schedule defined in your backup policy for A.8.13. Done when the restore test report, the MirrorMaker 2 configuration and the ACL replication results are filed. Failover and any production replication changes need your change approval.
Assemble the evidence register
The compliance lead builds one register that maps each control to the artifact that proves it: the policy for A.8.24, the listener and authorizer configuration for access and encryption, the log store and access list for A.8.15, the alert rules and test alert for A.8.16, and the restore test report for A.8.13. Each entry names the owner, the source system and the review date. Walk the register with the internal audit function before the certification body arrives, because DeployOpen prepares evidence but auditors and certification bodies decide conformity. Done when every artifact is linked and an internal reviewer has signed off the register.
What buyers ask before engaging a partner for Kafka and ISO 27001
Does running Apache Kafka make us ISO 27001 compliant?
No. Kafka does not make an organization compliant with ISO/IEC 27001:2022, and no software product does. Kafka supports specific technological controls, such as encryption in transit through SSL/TLS, access control through SASL and ACLs, and replication for recovery. Your management system, risk assessment, policies, and the evidence that these controls operate still have to come from your organization. Kafka is one input into the controls you declare in your Statement of Applicability.
What drives the effort and cost of preparing Kafka evidence?
The main drivers are the number of clusters, the inventory of listeners and clients, how keys and certificates are managed today, what log tooling already exists, and whether a second site is part of the design. Each cluster needs its listener configuration, authorization model and backup approach documented and tested. Mixed plaintext and TLS listeners, many client applications with different credentials, and undocumented key rotation all add work. A written scope should list each cluster, each listener and the owner of each control so the quote reflects the actual estate.
How long does the work take and what sets the duration?
The duration depends on the size and state of the estate, not on a fixed template, so this page does not give a figure. The factors that lengthen the work are the number of clusters and environments, the amount of undocumented configuration, the maturity of your key management and logging, and how many approval steps your change process requires. Ask any partner to show the phases, the deliverables in each phase and the points where your team must review or approve, and ask which of those dates depend on your access and change windows.
What does our team need to provide during the engagement?
Plan for named owners in security, platform operations, identity and the data teams that produce and consume topics. The work needs read access to cluster configuration, a test environment that mirrors production, change windows for broker restarts and listener changes, and a contact who can approve access to key stores and backup repositories. Ownership of each control after handover should also be agreed in writing, since an auditor will ask who performs each recurring check and who keeps the evidence.
Can our own team prepare the Kafka controls without a partner?
Yes, many teams do this internally, especially where they already run Kafka and operate an information security management system. The usual gaps are not in Kafka itself but in the surrounding work: a documented risk treatment for each control, key management procedures, centralized logging and alerting, and restore tests with recorded results. Compare your team's capacity for those tasks against the work in the scope. A partner can take on part of the build, document the configuration and prepare the evidence, while your team keeps ownership of decisions and the management system.
What does a fixed-scope quote include for a Kafka compliance engagement?
A fixed-scope quote should name the clusters, environments, listeners and client groups in scope, the Annex A controls being addressed, and the deliverables for each one, such as configuration baselines, authorization designs, key management procedures, logging and monitoring setup, and backup and restore test records. It should state the assumptions about your environment, the responsibilities on each side, the acceptance criteria and the change process. Ask which items are one-time work and which recur, such as periodic access reviews or restore tests, and whether ongoing support is priced separately as a retainer.
Can DeployOpen obtain ISO 27001 certification for us?
No. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Certification is granted by an accredited certification body after an audit of your management system. ISO does not certify organizations itself. Before you start, confirm with your chosen certification body which evidence they will sample, how they will treat the controls you exclude, and how they check the Kafka configuration in your environment.
Which Kafka settings do auditors ask about first?
Auditors usually start with whether encryption in transit is enforced on every listener that carries sensitive data, because Kafka security is optional and plaintext listeners can remain enabled. They then ask how access is granted, which means the authentication mechanism and the authorization model, and how log and monitoring records show who changed what. Confirm which listeners, users and super users exist in your clusters, and be ready to show the configuration files, the ACL export and the recorded results of your most recent restore test.
How DeployOpen scopes and delivers Kafka compliance configuration and evidence
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, including Apache Kafka. For ISO 27001 work, we configure the cluster and the surrounding platform for controls 8.24, 8.15, 8.16 and 8.13, then document the evidence your team and its certification body will need.
How we scope
- Agree in writing which Kafka clusters, topics and environments fall inside the Annex A controls in scope.
- Map each control to the Kafka settings, platform components and owners that support it.
- Confirm which items remain with your team, such as the ISMS policy, risk assessment and internal audit.
How we build
- Configure TLS listeners, SASL authentication, ACLs and KRaft settings to match the agreed control map.
- Connect logging, metrics and alerting to your existing log and monitoring stack in your environment.
- Test restores of backups and MirrorMaker 2 replication against documented recovery steps before sign-off.
How we hand over
- Deliver configuration baselines, runbooks and a control-to-evidence register written for your auditors.
- Document data-at-rest encryption, key management and access review procedures with named owners.
- Walk your operations and security teams through the evidence each control produces and where it is stored.
How we support
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Ask for a written quote that lists the controls, clusters, environments and deliverables it covers.
