How Apache Kafka supports HIPAA ePHI data streams
Apache Kafka can help meet several HIPAA Security Rule technical safeguards, but running it does not make a deployment compliant. The Security Rule applies to covered entities (health plans, health care clearinghouses and covered health care providers) and to their business associates, for electronic protected health information (ePHI) they create, receive, maintain or transmit (45 CFR 164.302). Kafka is one component in that picture. Its TLS encryption between clients, brokers and tools supports the transmission security standard in 45 CFR 164.312(e)(1), and its SASL and ACL controls support the access control standard in 164.312(a)(1). Kafka's KRaft authorizer stores ACLs in the metadata log, and resources without ACLs are limited to super users unless allow.everyone.if.no.acl.found is set to true, so the default posture is a useful starting point. Confirm that setting on every cluster before any ePHI topic is created.
Kafka does not cover everything the rule asks for. Its own documentation notes that security is optional, so unsecured listeners can exist and must be disabled. In KRaft mode, sensitive data in metadata records is stored unencrypted, so encryption of ePHI at rest (164.312(a)(2)(iv)) must come from disk, volume or application-level encryption and key management outside Kafka. Audit controls under 164.312(b) also need logs from brokers, Connect workers, identity providers and the hosting platform, collected and reviewed under procedures your organization owns. Business associate contracts under 164.314 and the administrative and physical safeguards remain your responsibility. The rule also includes an addressable specification for encryption, so document the reasoning for each choice.
There is no HIPAA certification. HHS states that no Security Rule standard requires certification and does not recognize private certifications, and the rule is enforced through complaints, compliance reviews and audits by the HHS Office for Civil Rights. DeployOpen configures and hardens the Kafka platform, prepares its controls, and assembles the evidence your assessors will ask for, but it does not issue attestations or audit reports. Before scoping work, confirm with your privacy and security officers which systems, topics and connectors touch ePHI, and which NIST SP 800-66 Rev. 2 control areas your risk analysis already covers.
Mapping HIPAA technical safeguards to Kafka settings
This table maps the four HIPAA technical safeguards that most often touch a Kafka cluster to the Kafka capability documented for each and the configuration the hosting platform must add. Read each row as requirement, what Kafka gives you, and what you still have to configure or provide yourself. Encryption at rest is the main gap: Kafka does not protect KRaft metadata records at rest, so disk or application-level encryption must come from your platform.
| Requirement | Kafka capability as documented | Configuration needed |
|---|---|---|
| Access control, 45 CFR 164.312(a)(1), unique user identification | SASL mechanisms GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER identify clients; StandardAuthorizer stores ACLs in the metadata log | Give each client and tool its own identity, use SASL over TLS, and confirm allow.everyone.if.no.acl.found is unset |
| Access control, unique user identification for shared service accounts | Authentication identifies a principal, but Kafka does not decide who a person is | Map each workload to a named principal, avoid shared credentials, and record the owner in your identity system |
| Encryption and decryption, 45 CFR 164.312(a)(2)(iv) | Kafka does not encrypt KRaft metadata records at rest; SSL/TLS covers traffic only | Encrypt broker volumes and backups at the storage layer, or apply application-level encryption to ePHI fields before producing |
| Audit controls, 45 CFR 164.312(b) | Kafka documents authorization decisions but its security overview does not describe a built-in audit trail of user activity | Ship broker, authorizer and client logs to a central store, retain them, and review access to ePHI topics |
| Transmission security, 45 CFR 164.312(e)(1) | SSL/TLS listeners encrypt traffic between clients, brokers and tools; security is optional and unsecured listeners can run | Disable plaintext listeners, enforce TLS on inter-broker and controller traffic, and cover MirrorMaker 2 links |
| Automatic logoff, addressable specification | Kafka clients hold sessions and credentials that do not expire from the broker's view by default | Set credential and token lifetimes in your identity provider and document how idle client sessions end |
| Authorization default, access restricted to granted rights | Resources with no ACL are limited to super users unless allow.everyone.if.no.acl.found is true | Keep the default, grant ACLs per topic and group, and limit super users to a documented break-glass set |
How Kafka TLS and SASL protect ePHI in transit
Kafka's transport security is opt-in, so the first control for ePHI in transit is to make every listener that can carry ePHI a secured one. The Kafka security overview documents SSL/TLS for traffic between clients, brokers and tools, but a cluster can run unsecured or mix secured and unsecured listeners. Review each entry in listeners, advertised.listeners and listener.security.protocol.map and confirm that no plaintext listener is reachable from a network segment that carries ePHI. For 45 CFR 164.312(e)(1), the transmission safeguard is about guarding against unauthorized access in transit, so an unencrypted listener left open for convenience is a finding even if no one uses it today. Encryption also needs to match the access model: the Security Rule's access control standard in 164.312(a)(1) depends on knowing who or what is connecting.
Authentication is the second half. Kafka supports SASL mechanisms including GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER, and it can use mutual TLS with client certificates. Pick one model per listener and document it, because the choice drives who issues credentials, how they rotate and what the audit trail names. SASL/PLAIN sends credentials that must be protected by TLS, so it should never run without TLS. Confirm the same rule for every internal path: inter-broker and controller listeners, Kafka Connect workers and their connections to brokers, and the MirrorMaker 2 replication links between sites. Each of these is a client of some kind and each needs its own encryption and identity, not a shared admin credential.
Certificate lifecycle and key custody belong to the platform team, and they are where most ePHI-in-transit controls quietly fail. Someone must own certificate issuance, the trust store contents, rotation before expiry, revocation for departed staff and services, and the location of private keys. Ask the team whether keys are generated and held in an HSM, a secrets manager or a restricted keystore, and who can read them. Test a rotation in a non-production cluster before the first production expiry date arrives. Kafka only enforces what the listeners and trust settings tell it to, so the evidence an assessor will want is the broker configuration as deployed, the certificate inventory and the rotation record, all kept under change control.
Encrypting ePHI at rest in Kafka storage
Kafka encrypts traffic on the wire through SSL/TLS, but it does not encrypt data at rest in its own storage. Under 45 CFR 164.312(a)(2)(iv), the encryption and decryption mechanism for ePHI is an addressable specification, so the decision and its documentation belong in your risk analysis. The cards below cover each place ePHI rests around a KRaft cluster and what to confirm for each.
Disk or volume encryption on broker logs
Encrypt the volumes or filesystems that hold each broker's log directories, so topic segments are unreadable if a disk, snapshot or decommissioned host leaves your control. Kafka does not do this itself, so the protection depends on the storage layer. Confirm with your storage team which key manager holds the volume keys, who can rotate them, and whether broker replacement re-encrypts data or only re-attaches it.
KRaft metadata storage encryption
The Kafka documentation states that sensitive data in KRaft metadata records is stored unencrypted, so the metadata log and snapshots need the same disk or volume protection as the log directories. Treat any configuration or credential material that reaches metadata as sensitive. Confirm that the controller volumes are included in your encryption scope and that your backup tooling captures them under the same policy.
Application-level payload encryption
Encrypt sensitive fields in the producer before they reach the topic, using keys held in a key management service or hardware security module outside the Kafka cluster. This protects ePHI even when an operator with broker access or a copied log segment is compromised. The trade-offs are key rotation, schema changes for encrypted fields and the loss of broker-side searching on those fields. Confirm which fields are encrypted and who holds decrypt rights.
Backup and snapshot encryption
Backups of topic data, controller snapshots and volume snapshots often leave the primary storage boundary and are copied to other sites or media. Each copy needs encryption with keys that are managed and access-logged separately from the production cluster. Confirm the backup target, the key custody model, retention periods and how restores are authorized and recorded.
Encryption at downstream Connect sinks
Kafka Connect sinks write the same records to databases, object stores and search indexes, where they are often copied again. Each sink must meet the at-rest requirements of its own system, and the connector configuration must not store database passwords in plain text. Confirm the encryption settings of every sink that receives ePHI and whether the sink retains deleted or replayed records.
Access control for ePHI topics and the minimum-necessary principle
Access to ePHI topics in Kafka is controlled by ACLs, and in KRaft clusters the StandardAuthorizer stores those ACLs in the cluster metadata log. The setting that matters most for this section is allow.everyone.if.no.acl.found. Under the default behavior, a resource without an ACL is restricted to super users, which is the safer starting point for a health data platform. Leave this key unset or false for ePHI topics, and confirm in your own test cluster that an unauthorized principal is refused on a newly created topic before any clinical feed is connected. Under 45 CFR 164.312(a)(1), only persons or software programs granted access rights under 164.308(a)(4) may reach ePHI, so the ACL design has to start from the access authorization decisions your organization has already documented.
Those decisions define the ACLs. Map each topic or consumer group to a role, such as an interface engine service account that produces admission events, a care-coordination application that consumes only the patients in its care team, or an analytics job that receives de-identified data on a separate topic. A consumer that is granted read on a whole topic receives every patient's record in that topic, so the need-to-know boundary usually has to be enforced by splitting topics, applying separate principals per service and keeping consumer groups narrow. Kafka ACLs do not filter records by patient or by care team. Where that granularity is required, it lives in the producer, in a dedicated topic per team, or in an external policy layer, and the design needs to say which.
Kafka authorization is pluggable and can integrate with external authorization services, but the capabilities of any such integration vary. Before you rely on one, confirm with its vendor or maintainers whether it supports the attribute-based or role-based decisions your access matrix needs, whether it can be audited per decision, and how it behaves when the external service is unavailable. Also confirm that the identities it trusts are the same ones your identity provider issues, and that removing a workforce member's access in the identity system removes their Kafka access in a time your policy allows. The Kafka authorization and ACL documentation is the starting point for the built-in behavior described above.
Configuration sequence for a HIPAA-scoped Kafka cluster
This sequence turns a general Kafka deployment into one where every ePHI flow is identified, encrypted in transit and at rest, authenticated, authorized by explicit ACLs, logged and documented against the Security Rule. Each step closes with a check the team can run and keep as evidence.
Confirm covered entity or business associate role
Compliance and legal leads, with the platform owner, first decide whether the organization is a covered entity (health plan, health care clearinghouse or covered health care provider) or a business associate under the Security Rule. They then list every topic, connector and consumer that creates, receives, maintains or transmits ePHI, including downstream copies made by sinks. The output is a signed data flow inventory naming each topic and its classification. Confirm the role against 45 CFR 164.302 before the cluster design is frozen, because it sets which contracts and documentation are required.
Deploy KRaft on a supported release
The platform team deploys a KRaft cluster on Apache Kafka 4.3.1, 4.2.2 or 4.1.2, the supported releases on the downloads page, and runs brokers on Java 17. Any ZooKeeper-based cluster must migrate to KRaft before moving to 4.x, and clusters older than 3.3.x should first move to 3.9.x. The check is a documented version inventory for every broker and tool, plus a record of the Java runtime on each host.
Enable TLS on ePHI listeners only
The platform team configures SSL/TLS on every listener that carries ePHI, for client, inter-broker and tool traffic, and removes unsecured listeners from those paths. Certificates are issued from the organization's internal CA with rotation owned by a named team. The check is a connection test showing that a plaintext client is refused on each ePHI listener, with the result saved as transmission security evidence under 45 CFR 164.312(e)(1).
Configure SASL tied to the identity source
The identity team and the platform team select a SASL mechanism that matches the directory, such as GSSAPI for Kerberos, or OAUTHBEARER for an OIDC provider, and avoid shared static credentials for people. Service principals are created one per application so that each identity is traceable. The check is that every client authenticates as a distinct principal and that the mapping from directory group to Kafka principal is written down and reviewed.
Set the authorizer and keep default-allow off
The platform team enables the StandardAuthorizer, which stores ACLs in the cluster metadata log, and leaves allow.everyone.if.no.acl.found unset so that resources without ACLs stay restricted to super users. Super user membership is limited to named administrators. The check is a test where an unlisted principal is denied on a sample ePHI topic and the ACL export is stored with the access model.
Encrypt data at rest at the storage layer
Because KRaft stores sensitive data in metadata records unencrypted, the infrastructure team encrypts the volumes or disks that hold log directories and the metadata directory, using keys managed in the organization's key management system. Application-level encryption can be added for specific fields where needed. The check is a documented encryption configuration per broker volume, with key ownership and rotation recorded, which addresses the at-rest specification under 45 CFR 164.312(a)(2)(iv).
Route broker and application logs to a monitored store
The security operations team ships broker server logs, authorization logs and the logs of Connect and client applications that touch ePHI to a central store with restricted access and retention set by policy. Alerts cover denied authorizations and changes to ACLs or listeners. The check is a test event (for example, a deliberate denied produce) appearing in the store and triggering an alert, which supports the audit control requirement in 45 CFR 164.312(b).
Document the access model and business associate flow-down
Compliance leads write the access model, mapping each role and principal to the ePHI topics it may read or write, and link it to the procedures for granting and removing access under 45 CFR 164.312(a)(1). Where a vendor or hosting provider handles ePHI, the business associate contract must require Security Rule compliance, flow the same terms to subcontractors, and require incident reporting under 45 CFR 164.314(a)(2)(i). The check is a signed access model and a contract register listing each party with ePHI access.
Evidence auditors and OCR will ask for from the Kafka platform
Kafka produces evidence only where someone has configured and kept it. The cards below cover the records a reviewer is likely to request for transmission security, access control, encryption, audit controls, backups and documented risk analysis. For each one, keep the record current as a routine output of the platform, not a snapshot assembled before an assessment.
Listener and TLS configuration exports
Export the broker listener map, the advertised listeners and the TLS settings for every client, inter-broker and controller listener, and store them under version control. Kafka's security is optional, so an unsecured listener can remain active without anyone noticing. Check that no listener carrying ePHI accepts plaintext connections and that certificate expiry dates are tracked. This record supports 164.312(e)(1).
Authentication and authorizer configuration
Retain the authentication mechanism for each listener, such as SSL, SCRAM or OAUTHBEARER, along with the authorizer class and its settings. Confirm that allow.everyone.if.no.acl.found is not set to true, since that setting changes the default for resources without ACLs. This evidence shows that only identified clients and tools reach systems holding ePHI, which relates to 164.312(a)(1).
ACL listings and change history
In KRaft mode, the StandardAuthorizer keeps ACLs in the cluster metadata log, so the change history is available from the platform itself. Produce a current ACL listing per topic and consumer group, and retain a log of who changed each entry and why. Reviewers will compare these records with the workforce access decisions that the organization documents. Ask your auditor which format they will accept.
Encryption-at-rest configuration
Kafka's KRaft metadata records hold sensitive data unencrypted, so encryption at rest depends on the disk, volume or application layer beneath it. Record the encryption method, key custody, key rotation schedule and the nodes covered for every broker and controller volume. Include the encryption status of backup copies. This documents the addressable encryption specification in 164.312(a)(2)(iv) and shows the reviewer what the platform, not Kafka, protects.
Access review and audit log records
Keep periodic access review sign-offs for Kafka administrators, service accounts and application principals, along with the broker and authorizer audit logs that show activity against ePHI topics. Define how long logs are retained and who reviews them, and keep evidence that the review happened. These records address 164.312(b), which asks for mechanisms that record and examine activity. Confirm the retention period with your compliance lead and your counsel.
Backup, restore and risk analysis records
Keep dated restore test results for topic data and cluster metadata, with the recovery steps and the people who ran them. Pair these with the risk analysis and documented policies required under 164.316, which must name Kafka and its data flows. Under 164.306(d), any addressable specification you decline needs a documented reason and an equivalent measure. No HIPAA certification exists, so these records are what an OCR investigation or auditor would examine.
What does it cost to make a Kafka ePHI pipeline HIPAA-ready?
What does it cost to make a self-hosted Kafka pipeline HIPAA-ready?
The cost depends on how many ePHI flows exist, how much of the surrounding platform must be built, and who performs each part. Kafka itself is open source under the Apache License 2.0, so there is no license fee from the project, but you still pay for the infrastructure, the people who build and run it, and the risk work. Recurring costs include compute, storage for encrypted volumes, backups, monitoring, patching and staff time for access reviews. One-time costs include design, identity integration, encryption setup, logging pipelines and the risk analysis input. Ask for the quote to split these into one-time and recurring lines, with each line tied to a named deliverable.
Who pays for each part of HIPAA readiness on a Kafka platform?
Your organization pays for its own infrastructure, cloud or data center capacity, licenses for any commercial tools it adds, and internal staff time. Legal counsel and privacy staff own business associate agreements, breach notification procedures and the decisions about what counts as reasonable and appropriate. A implementation partner is paid for the work in the written scope, such as platform configuration, TLS and authentication setup, authorization design, log export and documentation of evidence. Ask the quote to state explicitly which items are excluded, especially legal review and the enterprise risk analysis, because those are not technical deliverables.
How long does HIPAA-ready Kafka work take?
Duration depends mainly on the number of ePHI flows, the number of producers and consumers that touch them, and how much identity integration is needed. A single cluster with a few flows and an existing identity provider is far simpler than a multi-site platform with dozens of topics and connectors feeding downstream stores. Identity integration, especially connecting Kafka's SASL mechanisms to your directory, often sets the pace because access decisions depend on it. Ask for a timeline in the written scope that lists the flows in scope, the identity sources, and the points where your team must review or approve before the next step.
Can an internal team make a Kafka pipeline HIPAA-ready on its own?
An internal team can do it if it has engineers who know Kafka security, a clear view of its ePHI data flows, and time for testing and documentation. The Security Rule does not require outside help, and many organizations build these controls in-house. The usual gaps are not in Kafka configuration but in the surrounding work: mapping every topic and connector that carries ePHI, deciding how addressable specifications are handled, and producing evidence that holds up during an OCR review. Assess honestly whether those skills and hours are available before choosing between in-house work and a partner.
What does the work need from our team before a quote can be written?
A written scope needs inputs from your side. Start with a data flow inventory that names each topic, producer, consumer, connector and downstream store that carries ePHI. Provide the identity sources that will authenticate people and services, and the groups that should receive access. Supply the results of your most recent HIPAA risk analysis under 45 CFR 164.308(a)(1), or the inputs your privacy and security leads will use to create it. Also confirm where data is hosted, which business associates touch it, and what logging and retention your audit team already requires.
Is there a HIPAA certification for a Kafka platform?
No. HHS states in its 2003 FAQ that no Security Rule standard requires an entity to certify compliance, and that HHS does not endorse or recognize private certifications. Such certifications also do not prevent HHS from later finding a violation. The Office for Civil Rights enforces the rule through complaints, compliance reviews and audits, so the evidence you keep matters more than any badge. Be cautious with vendors who offer a HIPAA certificate for a platform or service, and ask them to explain what the document actually covers.
What penalties apply if an ePHI pipeline falls short of the Security Rule?
For penalties assessed on or after January 28, 2026, 45 CFR 102.3 sets an inflation-adjusted civil money penalty for violations on or after February 18, 2009, ranging from $145 per violation where the entity did not know of the violation to $2,190,294 per violation for uncorrected willful neglect. The calendar-year cap is $2,190,294. The amount actually assessed depends on the level of culpability and how quickly the problem was corrected. Penalties are one reason to fund the risk analysis and evidence work early, but they are not the only cost of a breach, which can include notification, investigation and remediation expenses.
How DeployOpen scopes and delivers HIPAA-ready Kafka configuration
DeployOpen plans, builds and hands over self-hosted Apache Kafka in infrastructure you control, with the transmission, encryption, access and audit controls for ePHI streams written down as part of the scope. We do not issue HIPAA certifications or attestations; your compliance program, risk analysis and independent assessors remain yours.
How we scope
- Inventory which topics, Connect pipelines and consumers carry ePHI and where each one is stored.
- Agree the written scope in advance, covering listeners, identity sources, ACL design, encryption and logging.
- Document which implementation specifications are required and which addressable choices are being made, with reasons.
How we build
- Configure KRaft clusters with SSL or SASL listeners, disabling unsecured listeners for ePHI traffic.
- Set StandardAuthorizer policies so resources without ACLs stay closed, and test each role against the access model.
- Apply disk, volume or application encryption at rest, since KRaft metadata records are stored unencrypted, and verify it.
How we hand over
- Deliver configuration as code, runbooks, a control-to-evidence map and a test record for each access and encryption check.
- Document the audit log sources, retention and review procedure your team needs for activity examination.
- Train your platform and security staff on rotating credentials, ACL changes and the upgrade path for the cluster.
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after handover, scoped in writing.
- Review changes to topics, ACLs and listeners against the documented ePHI data flow before they reach production.
- Confirm with your counsel and business associate contracts which vendor terms and subcontractor obligations apply to the engagement.
