Why Apache Kafka fits public sector data exchange
Agencies that need departments to exchange events under sovereignty and procurement constraints need a messaging platform they can host, inspect and license without a per-seat or per-cluster contract tied to a single vendor. Apache Kafka is released under the Apache License 2.0 (SPDX: Apache-2.0), and the Apache Software Foundation project itself offers no paid edition. Running it in your own data centers, government cloud accounts or air-gapped network means the agency, not a managed provider, decides where brokers run, where data is stored and who may read each topic. That combination is what makes self-hosted Kafka a practical fit for benefits eligibility events, case updates, licensing notifications and other cross-department feeds where the data must stay under national or agency control.
The change for the agency is ownership. A managed service hides the broker configuration, the controller metadata and the access rules behind a provider console. With self-hosted Kafka in KRaft mode, your team owns the cluster metadata log, which is where the StandardAuthorizer stores ACLs, and it owns the authentication design: SSL or SASL with mechanisms such as SCRAM-SHA-512 or GSSAPI (Kerberos), set per listener. Authorization is pluggable and can integrate with an external service, so the agency can decide whether access policy lives in Kafka ACLs, in a central identity system or both. Resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so confirm that setting explicitly during design. The evidence auditors ask for, such as who could read a topic and when access changed, becomes a set of records your team must export, retain and review.
Which public sector requirements shape the Kafka build
This table maps the main public sector requirements to concrete Kafka design choices, including cluster location, key lengths, ACL model, scanning cadence and evidence. Read each row as a question to answer before the build starts, not as a finished compliance position. Several rules apply only to specific agencies, data types or contracts, so confirm scope with your compliance lead and assessor first.
| Requirement | What it means for the Kafka build | Evidence to prepare |
|---|---|---|
| NIST SP 800-53 Rev. 5 baselines | Low, moderate and high baselines in SP 800-53B select different control counts (149, 287 and 370 controls and enhancements). Choose the baseline from the data classification first, then map Kafka brokers, Connect workers and Schema Registry to the selected controls. | Baseline selection record, control mapping per component, and the OSCAL profile version used |
| FISMA and OMB Circular A-130 | FISMA places system security assessment under agency Inspectors General and reporting to OMB. A Kafka platform carrying agency data becomes part of an authorized system boundary, so its owner must be named and its controls documented. | System boundary diagram, owner assignments, and the authorization package for the platform |
| FedRAMP scope under M-24-15 | FedRAMP covers cloud products that handle federal information on behalf of an agency, but systems used only for one agency's own operations on cloud infrastructure are generally out of scope. Self-hosting in agency-owned infrastructure avoids the cloud service question, while a managed Kafka service from a vendor may fall inside it. | Written scope determination and, for managed services, the vendor's FedRAMP status |
| CJIS Security Policy v6.1 encryption | For criminal justice information, the policy requires FIPS 140-3 validated modules or AES with 256-bit keys in transit and at rest outside physically secure locations. Set TLS for all listeners and encryption at rest for log segments and backups with validated cryptographic modules, and confirm the JVM and OS crypto provider meets this. | Cryptographic module certificates, cipher configuration, and key length settings per listener |
| CJIS Security Policy v6.1 vulnerability scanning | The policy calls for vulnerability scanning at least monthly. Include broker, Connect, Schema Registry and host images in the scan schedule, and track Kafka patch releases against supported versions 4.1.2, 4.2.2 and 4.3.1. | Scan schedule, scan reports, and a patch record showing remediation dates |
| 32 CFR 2002.14 for CUI | Federal agencies must apply NIST SP 800-171 when setting protections for controlled unclassified information on non-federal systems, unless the CUI category sets other safeguards. Topics carrying CUI need access control, auditing and encryption aligned to that standard, and the authority for each dataset should be identified. | CUI category inventory, topic-to-category map, and access review records |
| CMMC cloud rules (32 CFR 170.17) | A defense contractor seeking CMMC Level 2 may store or process CUI in a cloud service only if it is FedRAMP Authorized at Moderate or higher, or meets equivalent requirements. A self-hosted Kafka cluster in the contractor's own scope is assessed there, so the cluster location and ownership determine the assessment path. | Asset inventory showing where each cluster runs and who administers it |
| ADA Title II for web consoles | State and local entities must meet WCAG 2.1 Level A and AA for web content and mobile apps, including those provided through contracts. Kafka has no default web UI, but any console, portal or self-service topic tool you add falls under this rule, with compliance dates of 26 April 2027 for entities serving 50,000 or more people and 26 April 2028 for smaller ones. | Accessibility conformance report for each console, and vendor accessibility statements |
| NIS2 scope | NIS2 covers central government entities regardless of size and lists public administration as a high-criticality sector. Member States may extend it to local administration, and it excludes entities in national security, public security, defense and law enforcement. Confirm your national transposition before deciding how much incident reporting and supply chain evidence the Kafka platform must support. | Scope determination by national authority and an incident reporting procedure for the platform |
| GDPR Chapter V transfers | Any transfer of personal data to a third country, including remote access for support that only displays data, is a transfer under EDPB Guidelines 05/2021. Keep clusters, backups, support access and monitoring data in approved locations, and assess any processor subject to third-country law before engaging it. | Data flow map showing every location and remote access path, and transfer impact assessments |
Choosing sovereign, government cloud or air-gapped hosting
Hosting Kafka in an agency data center keeps the brokers, controllers, storage and operators inside the agency's own physical and administrative boundary. This is usually the simplest position to explain to an authorizing official because the agency owns the facility, the hardware and the access paths. Kafka's own security features still have to be configured: TLS for client, broker and tool traffic, SASL with a mechanism such as SCRAM-SHA-512 or Kerberos, and ACLs stored in the KRaft metadata log.
- Authorization scope: A system run only for one agency's own operations on agency infrastructure is generally outside FedRAMP scope, so the agency's own Authority to Operate process and control baseline apply instead.
- Patching: The agency team applies Kafka releases and operating system updates, so it must follow the supported release lines and the rolling upgrade path in the Kafka upgrade notes.
- Offline installation: Packages and container images still need to be staged internally through the agency's own repositories and change process.
- Vendor access: Vendor support is limited to what the contract allows, and remote sessions must be approved and logged under the agency's access rules.
- Staffing: The agency must hold skills for storage, networking, certificates and Kafka operations within its own team or a contracted partner.
Confirm with your authorizing official and your security office whether this environment and its control baseline apply to the Kafka data before you size the cluster.
A government or sovereign cloud places Kafka in a cloud environment operated for public bodies, often with data residency and in-country operations guarantees. The main question is whether the cloud provider is itself offering a service that holds federal information on behalf of an agency. FedRAMP's scope covers cloud products and services that create, collect, process, store or maintain federal information on behalf of an agency, while a system used only for one agency's operations on cloud infrastructure and not offered as a shared service is out of scope. A Kafka cluster that serves several agencies, or that is sold as a shared platform, moves closer to that scope.
- Shared service test: Ask whether the platform is offered to more than one agency or customer, because that changes the FedRAMP analysis under OMB M-24-15.
- Patching: Responsibility for the operating system, Kafka version and managed components is split between the agency and the provider, so the responsibility matrix must be written down.
- Offline installation: Usually not required, but image and package sources must still be verified and mirrored if the environment restricts outbound access.
- Vendor access: Provider personnel access, support tickets and data access paths should be reviewed against the agency's requirements for foreign or third-party access.
- Data flows: Replication to another region or to an external system must be checked against residency and transfer rules.
Confirm the provider's authorization status and the exact services in scope with the agency's authorizing official before selecting the environment.
An air-gapped network has no routine connection to external networks, so every Kafka binary, container image, certificate authority file, dependency and security update must be brought in through a controlled transfer process. This environment is commonly chosen for classified or highly sensitive workloads. The Apache Kafka project is licensed under Apache-2.0, which allows internal redistribution, but the agency still needs an internal artifact repository, signature verification and an approved change process for each release.
- Offline installation: Plan a transfer procedure, checksum and signature verification, and an internal mirror for the Kafka release, its JVM and operating system packages.
- Patching: Security fixes arrive only when the agency imports them, so the release cadence and bugfix policy for supported lines must be tracked closely.
- Vendor access: Vendor support must work through approved media or controlled sessions, and remote access paths need documented approval.
- Certificates and keys: Internal certificate authorities, key rotation and hardware security module integration must run entirely inside the enclave.
- Replication: Cross-site replication with MirrorMaker 2 requires network paths that the enclave design allows, so confirm them before design starts.
Confirm with the agency's security office which transfer, verification and approval procedures apply to the enclave before committing to a release plan.
Kafka settings that matter for government data
Government Kafka deployments are shaped by a small set of decisions made before the first topic is created: how the cluster spans sites, how every connection is encrypted and authenticated, who may read and write, how long records are kept, and how data moves between sites. Each card below names one of those decisions and what to verify in your own environment.
KRaft cluster layout across sites
Kafka 4.x runs in KRaft mode only, so there is no ZooKeeper ensemble to place or secure. The controller quorum, brokers and their failure domains must be planned across the sites your agency operates, including what happens when one site is isolated. Confirm the quorum size, the site that hosts each voter, and how a lost site is recovered before you go live. Clusters that still run ZooKeeper must migrate to KRaft before any move to 4.x.
TLS listeners with no mixed listeners
Kafka encryption is optional by default, so a cluster can expose plaintext and secured listeners side by side. For government data, define only SSL or SASL_SSL listeners for clients, brokers and tools, and confirm in configuration review that no plaintext listener remains on any network. Assess which certificate authority issues broker certificates and how rotation is handled, since expired certificates are a common outage cause.
SASL with agency identity
Kafka supports GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. Where the agency already runs Kerberos or an OIDC identity provider, prefer that path so that joiners, leavers and service accounts are managed in one place. SCRAM-SHA-512 is a practical fallback when no directory integration exists, but credentials then live in Kafka and need their own rotation process. Avoid PLAIN without TLS, and document which mechanism each client class uses.
StandardAuthorizer ACLs by default
In KRaft clusters, the StandardAuthorizer stores ACLs in the cluster metadata log. Resources without an ACL are restricted to super users unless allow.everyone.if.no.acl.found is set to true. Keep that setting false so that every topic, group and transactional ID is granted explicitly. Review super user membership as a separate control, and confirm who approves each ACL change and how those approvals are recorded for auditors.
Topic and retention planning for records
Retention and topic design decide how long citizen or case data sits in the cluster and who can reach it. Set retention per topic from the records schedule your agency follows, not from the broker default, and separate topics that carry personal data from operational telemetry. Confirm how deletion is applied to backups and replicas, and document which topics are compacted, since compaction does not remove a key's history on a fixed schedule for every record.
MirrorMaker 2 between sites
MirrorMaker 2 is built on Kafka Connect and replicates topics with their configuration, consumer groups with offsets, and ACLs between clusters while preserving partitioning. It is the documented path for a second site or a disaster recovery cluster. Plan which topics cross sites, whether the data may leave a jurisdiction, and how failover and failback are tested, since replicated ACLs and offsets must still be verified after a switch.
How an agency rollout moves from design to accreditation
This sequence takes an agency from a classified data scope to a Kafka platform with documented controls, tested cross-site replication and an evidence pack that the agency's authorizing official and assessors can review. Each step ends with an artifact the next step depends on.
Classify data and fix the scope
The agency's data owners, security office and DeployOpen agree which topics, producers and consumers carry sensitive or controlled information, and which exchanges stay inside the boundary. The team records the data categories, the departments on each side of every flow and the systems that will be wired to the cluster. The step is done when the agency signs a scope statement that names the in-scope topics, the connected systems and the out-of-scope flows. Authorization requirements, such as the control baseline the agency applies, should be confirmed with the agency's authorizing body before this step closes.
Set the hosting and authorization position
The team decides where the cluster runs (agency data center, sovereign cloud region or air-gapped network) and who operates it, then maps that choice to the authorization path the agency must follow. Where cloud hosting is considered, the agency checks whether the provider's offering falls within the scope of FedRAMP or an equivalent national program, and confirms with its own compliance office how that applies to a single-agency system. The output is a written hosting decision record covering location, operators, access paths for vendors and the authorization owner. Ask the agency's authorizing official to confirm this record before any infrastructure is built.
Design identity, ACLs and the version baseline
The team designs how people, services and administrators authenticate, choosing among the SASL mechanisms the Kafka security overview documents and whether identities come from a directory the agency already runs. It then writes the authorization model: principals per application, topic prefixes per department, and an explicit policy for resources with no ACL, since the StandardAuthorizer blocks such resources to super users unless the allow.everyone.if.no.acl.found setting is changed. The Kafka version is chosen from the supported releases on the Apache Kafka downloads page, with the upgrade path in mind. Migrations from ZooKeeper-era clusters or 3.x clusters need their own steps first, as the upgrade notes require. The step closes with an approved identity and ACL matrix and a pinned version.
Build the KRaft cluster with TLS and SASL
Engineers install the brokers and controllers in KRaft mode on the agreed hosts, enable TLS on every listener that carries agency data, and configure the chosen SASL mechanisms with certificates from the agency's certificate authority. Unsecured or mixed listeners are disabled for production topics, because Kafka security is optional and must be turned on deliberately. Java runtime, operating system hardening and file permissions follow the agency's baseline. The step is done when a test client can connect only through secured listeners, unauthorized clients are refused, and the configuration is stored under version control with the reviewer's sign-off.
Test cross-site replication
If the design calls for a second site or a disaster recovery cluster, the team configures MirrorMaker 2 to replicate the agreed topics, their configuration, consumer group offsets and ACLs. Testing covers a controlled failover, a replication lag measurement against the agency's recovery objectives, and a check that replicated data stays within the jurisdiction the hosting decision allows. Before go-live, the team confirms that any data path crossing a national boundary has been reviewed by the agency's legal and privacy staff. The output is a replication test report with observed lag, failover steps and the signed approval of the data owner.
Run security testing and the scanning cadence
DeployOpen runs security testing against the brokers, listeners, authorization model and the surrounding services, and the findings are triaged with the agency's security team. The agency agrees a vulnerability scanning cadence that meets the control requirements it must satisfy, and the team documents patch-level tracking for the Kafka release and its Java runtime. Retests confirm that critical and high findings are closed or formally accepted. The step closes with a test report, a remediation log and a written scanning schedule owned by a named agency role.
Assemble the evidence pack
The team compiles the artifacts an assessor or authorizing official will ask for: the scope statement, hosting decision record, identity and ACL matrix, configuration baselines, TLS and SASL test results, replication test report, scanning reports and the operations runbook. Each artifact is mapped to the control or requirement it supports, so reviewers can trace a claim to its evidence. DeployOpen prepares this pack, but the agency's assessor or authorizing body decides whether the controls meet the required standard. The step is done when the agency's compliance owner confirms the pack is complete and current.
Hand over operations and the upgrade plan
Operators from the agency receive the runbooks, monitoring setup, access procedures, backup and recovery steps, and a documented upgrade plan. The plan states which supported releases the cluster can move to, the order of broker and controller upgrades, and the test environment used before each change. Knowledge transfer sessions are recorded, and the agency's operators run a supervised change, such as a configuration update or a broker restart, before DeployOpen steps back. Handover is complete when the agency's operations lead accepts the runbooks and the first supervised change is recorded in the change log. Ongoing support, upgrades and testing can be contracted as a retainer if the agency wants them.
When X-Road belongs in a Kafka-based exchange design
Kafka and X-Road solve different problems, and the first design decision is which one you are building. A Kafka cluster inside one agency, or between closely coupled systems that one operator controls, is an event backbone: producers and consumers share topics, the operator sets retention and quotas, and every participant trusts the same administrators. A formal inter-organization exchange layer is different. When departments or agencies that do not share an operator must exchange records under agreed legal terms, with each party's identity verified and each message traceable, the exchange needs its own trust model. X-Road is open-source software for that purpose, published under the MIT license, and it is developed by the Nordic Institute for Interoperability Solutions. Kafka can still sit behind the exchange, carrying events inside each organization, but it should not be the mechanism by which organizations authenticate each other.
X-Road's main components are the Central Server, the Security Server and the Configuration Proxy. Each participating organization runs a Security Server, which needs PostgreSQL (local or remote, including managed services), a supported platform such as Ubuntu Server 22.04 or 24.04 LTS on x86-64, or Red Hat Enterprise Linux 8 or 9, and at least 3 GB of RAM and 3 GB of free disk space. Signing keys can be held in hardware security modules through PKCS#11 drivers using the xroad-addon-hwtokens package, which supports RSA and EC keys. Confirm the HSM model and driver with NIIS before procurement, because key custody is often the item that auditors and security teams ask about first. Release status matters too: 7.8.3 is the current X-Road 7 release, and 7.8.0 was the last new-feature release of that line. NIIS intends to keep X-Road 7 patched beyond 2026, but has not decided how long. Ask NIIS for the written end date before you commit a multi-year program to 7.x.
X-Road 8 ('Spaceship') is available as 8.0.0 Beta 2, with a production release scheduled for December 2026, and it is positioned as a data space solution with improved monitoring and cloud compatibility. A beta is not a basis for a production design, so plan the first deployment on 7.8.3 and treat the 8 migration as a separate, funded decision. Ask which upgrade path NIIS will support from 7.x to 8, and whether your Security Server hardware and PostgreSQL version meet its requirements. The X-Road releases page is the place to check the status of each version before the design review.
Compliance evidence a Kafka platform must produce
Auditors and assessors will ask for proof that Kafka controls work as configured, not just that they exist. The cards below list the evidence a self-hosted cluster serving public-sector data should be able to produce, and where each item sits between Kafka settings, the platform around it and the agency's own processes. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications; independent assessors and certification bodies make those determinations.
Access control evidence from ACL metadata
In KRaft clusters, the StandardAuthorizer stores ACLs in the cluster metadata log, so an exported ACL listing is a point-in-time record of who can read, write or administer each topic and group. Confirm that resources without ACLs are restricted to super users, and that allow.everyone.if.no.acl.found is not set to true. Keep the list of super users short and documented, and reconcile ACLs against the agency's approved role model at each review. Client identities should come from the agency's directory through SASL, not shared accounts.
Encryption evidence for CJI and CUI paths
Kafka can encrypt client, broker and tool traffic with SSL/TLS, but security is optional and clusters can mix secured and unsecured listeners. Evidence should show that every listener carrying criminal justice information or controlled unclassified information is TLS-only, and that the cipher and key sizes meet the applicable baseline. For CJI at rest and in transit outside physically secure locations, CJIS Security Policy version 6.1 requires FIPS 140-3 certified modules or FIPS 197 AES with at least 256-bit keys, up from 128-bit. Confirm the module validation status with your assessor before relying on it.
Monthly vulnerability scanning records
CJIS Security Policy version 6.1 calls for vulnerability scanning at least monthly, so the platform needs scan records for the Kafka brokers, Connect workers, schema and admin tools, and the host operating systems beneath them. Keep scan scope, dates, findings and remediation tickets together, and show how each finding was closed or accepted. Scanning alone is not enough: record which Kafka release is running and whether it is one of the supported releases on the downloads page, since earlier versions no longer receive bugfixes.
Data residency and third-country access analysis
GDPR Article 44 applies to every transfer of personal data to a third country, including onward transfers, and EDPB Guidelines 05/2021 treat remote access from a third country, even display-only support sessions, as a transfer. Document where brokers, backups, logs and support access sit, and who can reach them from outside the EEA. Article 48 limits recognition of third-country disclosure orders to those based on an international agreement. For any non-EU cloud or managed operator, record the assessment of extraterritorial exposure before engaging it, as the EDPB guidance requires.
Data Act switching and Article 32 measures
The Data Act gives customers the right to switch from a data processing service to another provider or to on-premises infrastructure, with a maximum two-month notice period and a 30-day transition. Article 32 requires providers to take technical, organisational and legal measures against third-country governmental access to non-personal data held in the Union. Where the platform is run by a provider, obtain written evidence of export formats, topic and offset migration paths, and those Article 32 measures. Confirm how these terms apply to your specific contract with counsel.
Accessibility of admin web interfaces
Kafka itself has no browser interface, but the admin consoles, Connect dashboards, schema tools and monitoring front ends an agency deploys around it may. Under the ADA Title II rule, state and local government web content and mobile apps, including those provided through contracts or licenses, must meet WCAG 2.1 Level A and AA, with compliance from 26 April 2027 for entities serving 50,000 or more people and from 26 April 2028 for smaller entities and special districts. Record the conformance testing for each interface and its owner, and ask suppliers for their accessibility conformance report.
Questions agencies ask before hiring a Kafka partner
What drives the cost of a public sector Kafka deployment?
Scope drives cost most: the number of clusters, sites, departments onboarded, topics and data classes, and whether the deployment must be air-gapped. Each added department usually adds topic design, access rules, onboarding and testing work. Authorization path also matters, because a system that must meet a federal baseline or a CJIS-style control set needs more documented evidence. Kafka itself is licensed under Apache-2.0 with no paid edition from the project, so the quote reflects labor and infrastructure, not license fees. Ask for a written scope that lists sites, departments, environments, data classes and the evidence the agency expects, then compare quotes against that same list. Infrastructure you pay for directly (servers, storage, network, hardware security modules) is separate from the implementation fee.
Can our own team run a self-hosted Kafka cluster without a partner?
Yes, many agencies run Kafka internally, and the decision depends on the skills and staffing you can commit. Operating Kafka in KRaft mode requires people who can manage broker configuration, TLS and SASL setup, ACLs, upgrades, monitoring and disaster recovery. Check whether your team already runs comparable platforms, whether on-call coverage exists for a messaging backbone, and whether you can keep a test environment that mirrors production. A partner can plan and build the platform, document it and hand it over, or support it under a retainer. Ask any partner to show what knowledge transfer and runbooks they deliver so your staff can operate the system after the engagement.
How long does a public sector Kafka rollout take?
Duration depends on how many departments connect, how many sites are involved, how far the authorization path is from existing controls, and whether the environment is air-gapped. Most of the schedule is spent on data classification, topic and access design, identity integration, network approvals and testing, not on installing brokers. Agencies with a formal accreditation or assessment step should add time for evidence collection and review by the assessor. Because the engagement is fixed-scope, the timeline is agreed in writing before work starts, and it should list the dependencies on your side. Ask the partner for a phased plan with milestones and the approvals each milestone needs.
What does our agency need to provide during the engagement?
The engagement needs named people and environments from your side before build work begins. The main inputs are:
- Data owners who classify each data flow and approve which departments may publish or consume topics.
- Identity contacts who connect Kafka clients to your directory and decide how SASL or TLS credentials are issued.
- Network staff who open the required ports, approve routes between sites and provide DNS and time sources.
- Security approvers who review access rules, encryption choices and the evidence pack.
Also confirm who signs off each phase and who can make decisions when departments disagree on retention or access. Missing approvers are one of the most common causes of delay, so name them in the statement of work.
Is a fixed-scope quote available for a Kafka implementation?
Yes, most DeployOpen engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. DeployOpen does not publish a price list, so the quote follows from the written scope. To make a quote comparable, ask every bidder to price the same items: sites, environments, departments, topics or data flows in scope, authentication method, encryption and key handling, backup and recovery testing, documentation and handover. Also ask which items are one-time and which recur, such as upgrades, monitoring or continuous testing, which DeployOpen can provide as retainers. Ask what is excluded, including infrastructure and third-party licenses, so the figures can be compared on equal terms.
How do we keep Kafka on supported releases after go-live?
Plan upgrades as recurring work and choose a partner or internal process that will run it. The Kafka project releases three times a year, and its time-based plan supports rolling upgrades from each of the last three releases to the latest version. The downloads page lists 4.3.1, 4.2.2 and 4.1.2 as the supported releases, and earlier versions are archived. Since Kafka 4.0, only KRaft mode is supported, so ZooKeeper-based clusters must migrate first, and the upgrade notes recommend moving KRaft clusters older than 3.3.x to 3.9.x before moving to 4.x. Brokers and Connect require Java 17. Ask the partner how they test each upgrade in a non-production environment and how they document the change for your accreditation records.
Does DeployOpen issue compliance certifications for our Kafka platform?
No. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors, assessors and certification bodies make those determinations. In practice, DeployOpen can help map the platform to the frameworks that apply to your data, configure the controls, document them and assemble the evidence an assessor will ask for. Before engaging, confirm with your assessor or accreditation authority which baseline and evidence format they require, because the assessment outcome is theirs to decide.
What to ask any partner before signing a Kafka scope
Start with the agency's compliance frame, because it decides who can touch the environment and what evidence exists afterward. Ask which federal or state frameworks the partner has implemented against before, and how they map a Kafka cluster to them: NIST SP 800-53 Rev. 5 control families for access, audit and configuration, and, where criminal justice data flows through topics, the FBI CJIS Security Policy requirements for encryption strength and vulnerability scanning. Ask for named examples of the control mapping they produced, without client names, and confirm that they prepare evidence for assessors rather than issuing any audit opinion themselves.
Next, pin down where support staff sit and who can reach the environment. Ask for the country and facility of every engineer who would log in to brokers, Kubernetes nodes, Security Server or the metadata quorum, whether remote sessions display data on a screen, and how privileged access is granted, logged and revoked. Ask how the partner delivers air-gapped installs: which artifacts are carried in, how the Kafka release is verified before it crosses the boundary, and how offline patches for Kafka 4.x, the JVM and the operating system arrive. Ask who signs off on each transfer. The answers should be written into the scope, not left to a kickoff call.
Finally, require the scope document to name every deliverable that touches access control and evidence. For ACLs, that means the export format and the method for producing it from the KRaft StandardAuthorizer metadata, plus a plan for allow.everyone.if.no.acl.found so that no resource is open by accident. For evidence, list the configuration baselines, TLS settings, audit log retention and test records the agency will receive. Also write in the acceptance tests, the handover of administrator credentials and the post-go-live support terms. Most engagements are fixed-scope, so anything not written into that document will be a change request with its own price.
How DeployOpen scopes and supports public sector Kafka work
DeployOpen plans, implements, hardens, documents and supports self-hosted Apache Kafka inside the agency's own infrastructure, including cloud accounts, data centers or air-gapped networks. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
How we scope
- Written scope covers the data exchanges, topics and departments the cluster will serve
- Hosting location, network zones and any air-gapped constraints are recorded before design begins
- Applicable control frameworks are identified so evidence needs are agreed up front
- Fixed-scope deliverables, timelines and responsibilities are confirmed in writing before work starts
How we build
- KRaft cluster deployed in your environment with planned partitions, retention and replication settings
- Authentication, TLS and ACLs configured for each producing and consuming system and team
- Cross-site replication with MirrorMaker 2 designed and tested where the agency requires it
- Security testing and verification of access controls performed against the agreed scope
How we hand over
- Architecture, configuration and access model documented so your staff can operate the cluster
- Runbooks cover upgrades, backups, topic onboarding and access requests for your teams
- Control evidence and configuration records are assembled for your assessors and auditors
- Knowledge transfer sessions are scheduled with your platform and security staff
How we support
- Ongoing support, upgrades and continuous testing are available as retainers after handover
- Upgrade planning follows the Kafka release plan and your own change approval process
- Retainer scope, response expectations and exclusions are confirmed in the written agreement
- Your team keeps control of the infrastructure, data and credentials throughout

