Event streaming for SaaS platforms

Event streaming for SaaS platforms: why the tenant model changes the build

For a SaaS engineering lead, the choice between Apache Kafka, Apache Pulsar and NATS comes down mainly to two things: the tenant isolation model the product must offer, and who will own operating the cluster once customers depend on it. All three are realistic options, but they isolate tenants differently. Pulsar is designed with multi-tenancy, authentication, authorization, quotas and optional hardware isolation as built-in concerns. NATS accounts isolate messaging domains within one server or cluster, which the NATS documentation describes as suitable for secure multi-tenant deployments and SaaS platforms. Kafka isolates tenants mainly through topic naming conventions, ACLs, quotas and, where needed, separate clusters, so the design work sits with your platform team.

A SaaS backbone differs from a single-enterprise streaming platform in three ways. First, noisy neighbor risk is per tenant: one customer's burst of events can consume broker, network or storage capacity that other tenants share, so quotas and per-tenant throughput limits must be tested under load, not assumed. Second, customer-facing SLAs turn internal latency and retention settings into contract terms, which means you need per-tenant monitoring that can show whose traffic was delayed. Third, contractual data residency can require that a given tenant's events stay in a named region, which may force separate clusters or accounts rather than a single shared pool. Confirm each of these with your sales and legal teams before you pick a tenant model, because changing it after customers are live is expensive.

Start by writing down, for each tenant tier, the isolation level, quota, retention and residency commitment you will sign. Then check which platform can enforce each one natively and which you would build around it, using the vendors' own documentation and a load test that mixes one large tenant with many small ones. The Apache Kafka security overview and the Kafka authorization documentation are the natural next reads for how ACL enforcement works in KRaft clusters. Treat these answers as the input to your build, partner or managed decision, not as a conclusion.

Which requirements a multi-tenant SaaS backbone must meet

This table maps each requirement a multi-tenant event backbone faces to its source obligation or control and to the design decision it forces on topics, accounts and clusters. Read each row as a question to answer before the cluster is sized, because most of these requirements are cheap to design in early and expensive to retrofit. The main takeaway is that tenant isolation is a legal and evidentiary question as much as a technical one.

Requirement	Source obligation or control	What it means for topic, account and cluster design
Tenant isolation of data and access	GDPR Article 28(3) requires processing only on documented instructions and Article 32 security measures	Decide whether tenants share a cluster with separate topics, or get separate clusters; document which tenant data sits in which topic and who can read it
Authentication per tenant	SOC 2 security criteria (CC6) on logical access and authentication	Kafka supports SSL and SASL mechanisms including SCRAM and OAUTHBEARER; issue distinct client identities per tenant rather than one shared service login
Authorization per tenant	Kafka KRaft StandardAuthorizer stores ACLs in the metadata log; resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is true	Keep that setting false, write prefixed ACLs per tenant, and test that a tenant identity cannot read another tenant's topics
Quotas and noisy-neighbor limits	SOC 2 availability criteria on capacity and performance	Set produce, consume and request quotas per tenant or client identity; Pulsar offers quotas natively, while Kafka quotas must be configured and tested per principal
Schema governance	SOC 2 processing integrity criteria on complete and accurate processing	Use a schema registry with compatibility rules per topic so one tenant's schema change cannot break consumers of other tenants
Partition and capacity scaling	NIS2 Annex I places cloud and managed service providers in high-criticality sectors	Plan partition counts and broker capacity against the largest tenants; confirm how partitions can be added without breaking per-key ordering for each tenant
Cluster placement and provider scope	NIS2 Annex I covers cloud, data center and managed service providers; Commission Implementing Regulation (EU) 2024/2690 sets technical requirements for them	Confirm whether your SaaS is in scope as a provider and whether the cluster operator, if a partner, falls under the same obligations
Switching and exit for customers	EU Data Act Articles 23, 25 and 50 require switching to another provider or on-premises infrastructure; Article 29 removes switching charges from 12 January 2027	Keep topic schemas, retention and tenant data exportable in open formats so a customer can move its streams without vendor-specific tooling
Audit evidence of controls	SOC 2 Type 2 reports cover operating effectiveness over a period; SOC 3 is general use	Retain ACL, quota and access logs per tenant over the audit period so an assessor can sample operating evidence

Choosing a tenant isolation model: shared cluster, accounts or dedicated clusters

A shared Kafka cluster serves every tenant from the same brokers, with isolation enforced by authentication and authorization on each topic. Kafka supports SSL and SASL for clients, brokers and tools, with the documented SASL mechanisms GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. In KRaft clusters, the StandardAuthorizer stores ACLs in the cluster metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. This default-deny behavior is useful, but the isolation is only as strong as the ACLs your provisioning code writes.

Authentication per tenant: Give each tenant its own SCRAM or OAUTHBEARER identity so that every produce and consume request maps to a named principal.
Topic-level ACLs: Scope read, write and group permissions to each tenant's topic prefix, and test that cross-tenant access is denied.
Quotas: Set client quotas per tenant so one noisy producer does not starve others. Confirm the quota options for your Kafka version.
Blast radius: A broker fault, a bad rebalance or a misconfigured cluster-wide setting affects every tenant at once.
Operating cost: One cluster is the cheapest to run, but ACL automation, credential rotation and per-tenant audit evidence become ongoing engineering work.

Ask your partner to show how ACLs are generated from tenant records and how a failed provisioning run is detected and rolled back.

Which open-source broker fits a SaaS event backbone

A SaaS event backbone has to isolate tenants, enforce quotas and keep schemas stable while the platform grows. The four brokers below make different trade-offs on those points, so the choice should follow from how your product isolates customers and what kind of messaging it needs.

Apache Kafka for ecosystem depth

Kafka is the default choice when the SaaS platform needs a large connector and client ecosystem and a broad base of operators who already know it. It is licensed under Apache-2.0 with no paid edition from the project itself. Tenant isolation is largely by convention: topic naming, ACLs enforced per principal, and quotas you configure yourself. Check that your team can run KRaft clusters, since Kafka 4.0 removed ZooKeeper mode, and confirm which of the supported releases (4.3.1, 4.2.2, 4.1.2) your upgrade plan targets.

Apache Pulsar for built-in multi-tenancy

Pulsar is designed around multi-tenancy, with authentication, authorization, quotas and optional hardware isolation as part of the platform rather than add-ons. Its brokers are stateless and storage runs on BookKeeper bookies, so capacity can grow by adding bookies independently of brokers. The trade-off is operational: Pulsar runs more components, including the metadata store. Pulsar 5.0 LTS was released on October 5, 2026, and 4.0 LTS reaches the end of active support on October 21, 2026, so plan around the version you adopt.

NATS for account-based isolation

NATS uses accounts to isolate messaging domains within one server or cluster, which suits teams building per-tenant or per-product messaging with a small footprint. JetStream adds persistence and replay on top of core messaging. NATS is a CNCF-related project, and the CNCF announced in 2025 an alignment with Synadia on the project's future, which is worth reviewing for governance. Confirm JetStream limits, storage replication settings and account export rules in the version you deploy, because multi-tenant guarantees depend on them.

RabbitMQ for task queues

RabbitMQ fits background job dispatch, request routing and work queues where each message should be handled once by a worker, with acknowledgments and dead-letter handling. It is a weaker fit for long-retention event logs that many independent consumers replay. For a SaaS backbone, it often complements a log-based broker rather than replacing it. Confirm vhost-based isolation, queue quotas and how your chosen version handles consumer acknowledgments and high-availability queues before you commit tenants to it.

How to manage schemas and scale partitions without breaking tenants

Schema governance and partition design decide whether one tenant's change can break another tenant's consumers, so set both before the first customer is onboarded. Start by choosing a compatibility rule per topic (for example, backward compatible only, so that consumers built against the previous schema keep working) and enforce it in a schema registry that rejects producers publishing incompatible changes. Confirm which registry tooling your chosen platform supports and how it handles multi-tenant subjects, since Kafka, Pulsar and NATS each expose schema handling differently. Write each event contract as a versioned document owned by the producing team, and name who approves a breaking change: usually a platform owner plus the consumer teams whose contracts would be affected. Tenant-specific fields should go in extensible, optional attributes rather than in separate per-tenant schema versions, which multiply the compatibility matrix you must test.

Partition count is the decision that is hardest to reverse for ordering. In Kafka, ordering is guaranteed only within a partition, so the partition key should be the tenant identifier, or tenant plus aggregate identifier, when per-tenant or per-entity ordering matters. Adding partitions later changes how keys map to partitions, which breaks ordering for keys already in flight, so size partition counts for peak tenant throughput and expected growth at the outset. Adding partitions or moving consumers also triggers rebalancing, which pauses consumption while group membership settles. Test rebalance time with your largest tenant's traffic before production. Pulsar separates these concerns: brokers route messages and BookKeeper bookies store them, and the architecture documentation states that storage capacity grows by adding bookies independently of brokers. That lets you add disk without changing topic partitioning, although you should still confirm how your Pulsar version handles key ordering and partition changes for each tenant.

Upgrades also change schema and partition tooling, so plan them with tenant impact in mind. The Kafka upgrade notes state that Apache Kafka 4.0 supports only KRaft mode, and that rolling upgrades are supported from the last three releases to the latest version, so check your current release before scheduling a move. For Pulsar, confirm which LTS line your platform sits on, since the release policy sets different active and security support end dates for each. Record every schema change, partition change and broker rebalance in an audit trail that names the tenant affected, the approver and the rollback path. That record is also the evidence a customer's assessor will ask to see when they review how your platform protects their data.

Deployment patterns for SaaS: cloud account, data center or dual region

Each pattern trades isolation, residency and recovery against the number of clusters you must upgrade, monitor and pay for. The four cards below cover the layouts SaaS teams usually choose between, and the integration points that every layout has to satisfy.

Single-region cluster per environment

One cluster per environment (development, staging, production), each in a single region and cloud account. This is the simplest layout to upgrade and monitor, and it keeps tenant traffic on one set of brokers. The trade-off is that a regional outage stops every tenant at once, and staging must match production closely enough to catch upgrade problems. Check the broker count, replication factor and whether environments share an account or identity boundary.

Per-customer-region clusters for residency

Clusters placed in the region where each customer group's data must stay, so events from that group never leave it by default. Routing is decided at onboarding, and the control plane must know which cluster holds which tenant. Confirm which event fields count as personal data, how cross-region analytics are handled, and whether support staff in other regions can read payloads. Each added region adds a cluster to patch and monitor.

Dedicated clusters for enterprise tenants

Enterprise customers that contract for isolation receive their own cluster or Pulsar cluster-level isolation, while smaller tenants share a pooled cluster. Shared clusters rely on ACLs, NATS accounts or Pulsar tenant policies and quotas for separation, so test those controls before promising isolation. Ask who pays for idle capacity on dedicated clusters, and how a tenant moves between pooled and dedicated placement without losing ordering or offsets.

Geo-replicated pairs with MirrorMaker 2

A second cluster in another region that receives copies of topics through MirrorMaker 2 for Kafka, or through Pulsar geo-replication for Pulsar. This supports recovery and read locality but is not a backup on its own, because deletes and bad writes can replicate too. Replication lag, consumer offset translation and failover steps need to be designed and tested. The sibling topic on multi-datacenter streaming and disaster recovery covers the recovery objectives and failover tests in detail.

Identity, observability and metering hooks

Every pattern needs the same integration points: an identity provider that issues client credentials per tenant, metrics and logs that carry a tenant label without exposing payloads, and a metering feed that counts bytes and partitions per tenant for billing. Kafka supports SASL mechanisms such as OAUTHBEARER and SCRAM, and NATS and Pulsar offer their own auth models. Confirm how credential rotation, per-tenant usage counts and billing exports behave when a tenant moves between clusters.

What self-hosting costs compared with managed Kafka

What are the main cost components of self-hosting Kafka for a SaaS platform?

Self-hosting moves costs from a vendor invoice into your own budget: compute for brokers and controllers, block storage for retained logs, network traffic, the people who run the cluster, and the tooling around it. The managed service bundles these into one meter, so the comparison only works when you price each component on both sides. Ask the managed provider to break its quote into throughput, storage, partitions, network and support, and build the same lines for your own environment. Costs are driven mainly by sustained throughput, how long you retain data, replication factor and the number of tenants whose topics you must isolate. Apache Kafka itself is licensed under Apache-2.0 with no paid edition from the project, so license fees are not the variable; infrastructure and labor are.

How do replication and cross-zone traffic change the bill?

Replication multiplies stored bytes and network transfer: each partition is written to several brokers, and producers and consumers spread across availability zones can incur inter-zone transfer charges in public clouds. Many SaaS teams underestimate this line because it does not appear as a broker cost. Ask your cloud provider how inter-zone and egress traffic are billed for your region, and model replication factor, consumer fan-out and any cross-region mirroring separately. Consumers that read from a follower in their own zone, where your Kafka version and configuration support it, can reduce this cost; confirm the exact settings for your release in the Kafka documentation before you rely on them. Managed services often include some of this traffic in their pricing, so compare what is included rather than the headline unit rate.

What does running Kafka on call cost in people and time?

The largest recurring cost is usually the people who keep the cluster healthy: platform engineers who handle broker failures, partition rebalancing, capacity planning, client onboarding, quota changes and incident response. A multi-tenant SaaS cluster needs an on-call rota that can respond to broker and controller alerts at any hour, so the cost depends on your staffing model and existing skills, not only on the cluster size. Estimate hours per month for routine operations and for on-call coverage, then compare that with the support tier in the managed quote. Partner retainers for ongoing support are another option to price against in-house staff. DeployOpen offers ongoing support and upgrades as retainers, and its scope and fees are set in writing for each engagement.

How much effort do Kafka upgrades take, and what recurs?

Upgrades are recurring work, and their effort depends on how far behind your cluster is. Kafka supports rolling upgrades from each release in the past year, which means the last three releases, to the latest version, so staying current keeps each upgrade small. The supported releases listed on the downloads page are 4.3.1, 4.2.2 and 4.1.2, with bugfix releases made only for supported lines. Kafka 4.0 supports only KRaft mode and removed ZooKeeper, so clusters still on ZooKeeper must migrate to KRaft before upgrading to 4.x, and the upgrade notes recommend moving KRaft clusters older than 3.3.x to 3.9.x first. Since Kafka 4.0, brokers and Kafka Connect require Java 17. Plan a recurring budget for testing client compatibility and for these steps, and confirm the current upgrade path with your engineers before committing to a schedule.

What does security hardening add to a self-hosted cluster?

Security is a set of one-time design decisions plus ongoing operating work. Kafka security is optional and must be enabled deliberately: you configure TLS between clients, brokers and tools, choose a SASL mechanism such as SCRAM-SHA-512 or OAUTHBEARER, and define authorization. In KRaft clusters, the StandardAuthorizer stores ACLs in the metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so ACL design per tenant is a real workload. Costs include certificate and secret management, key rotation, access reviews and penetration testing of the platform. These items are one-time when designed and recurring when operated. Ask each provider whether TLS and authentication are enabled by default, who maintains the ACL model, and how secrets are rotated.

What costs are one-time and what costs recur?

One-time costs include architecture and sizing, initial deployment, the migration of producers and consumers, security design, monitoring and backup setup, and the handover of runbooks. Recurring costs include compute and storage that grow with throughput and retention, network transfer, staff time for on-call and routine operations, upgrades, certificate rotation, testing and any support retainer. Retention is often the strongest driver of storage cost, and tenant count drives partition counts, quota management and ACL overhead. Ask a managed provider to quote each recurring line for your expected peak and for your retention period, then ask a partner to quote one-time and recurring work separately in a fixed-scope written proposal. Confirm which costs your own team already carries before comparing totals.

Rollout: from tenant model decision to supported production backbone

This sequence moves a SaaS team from a tenant model decision to a self-hosted event backbone that carries live tenant traffic under monitoring and a written handover. Each phase closes with a named output that the next phase depends on, so the platform is never onboarded to before its security, quotas and governance are in place.

Decide the tenant model and requirements
The product, platform and security leads agree which tenant model the backbone will use: one cluster with topic-level isolation, separate clusters per tier or region, or account-style isolation where the broker supports it. In Pulsar, tenants and namespaces map to this choice. In NATS, accounts isolate messaging domains. In Kafka, isolation depends on topic naming, ACLs and quotas. The team also records throughput, retention, ordering needs and which protocols clients use. The output is a signed requirements and tenant model decision record that names what each tenant may read and write and what a noisy neighbor is allowed to consume. Confirm with the chosen platform vendor documentation which isolation features are native and which you must build.
Size the cluster and design quotas
Engineers translate the requirements into a sizing model covering broker count, partition counts per tenant, replication factor, storage per tenant and network capacity. They define per-tenant produce and consume quotas, connection limits and maximum partitions so that one tenant cannot exhaust shared capacity. The output is a sizing workbook and a quota policy that the platform team can enforce in configuration, with the assumptions written down so they can be re-tested when traffic changes. Ask the implementation partner to show how the model will be re-run when a tenant grows.
Build the platform with TLS and SASL or OAuth
Engineers deploy the chosen broker in the organization's cloud accounts or data centers, with TLS on client, broker and inter-broker listeners and an authentication mechanism that matches the identity estate, such as SASL with SCRAM or OAUTHBEARER for Kafka. The metadata layer, storage nodes and monitoring stack are built in the same pass, and no listener is left unsecured by default. Authorization is configured so resources without an explicit grant are denied. The output is a hardened cluster in a non-production environment, with a build record that lists versions, certificates, listener configuration and the authorization model, ready for independent testing.
Govern schemas and topic naming
Data owners and platform engineers agree on topic naming conventions that embed the tenant identifier, a schema registry with compatibility rules, and a request workflow for creating topics and changing schemas. Producers register schemas before publishing, and breaking changes are blocked by the compatibility check. Retention and cleanup policies are set per topic class. The output is a governance standard with the registry configured, a documented change process and automated checks that run in the build pipeline. Confirm that the registry you choose supports the compatibility modes you need before you commit to it.
Migrate from managed Kafka with dual running
Where the SaaS product already runs on a managed streaming service, the team moves one producer or consumer group at a time. Producers write to both the managed service and the self-hosted cluster while consumers read from the source of truth, and offsets, ordering and duplicate handling are compared between the two sides. Replication tools move historical data where retention requires it. The output is a cutover runbook, a reconciliation report for each migrated topic, and a documented rollback path that has been exercised in a test window before any tenant traffic moves.
Onboard tenants through a repeatable process
Each tenant is provisioned from a template that creates its namespace or account, credentials, ACLs, quotas and topic set in one controlled run, with approvals recorded. The platform team onboards a pilot tenant first, then adds tenants in groups, checking quota usage and consumer lag after each group. The output is a tenant onboarding record per tenant and a provisioning procedure that can be run by the operations team without the original builders. Ask the partner how offboarding and credential revocation are handled, since these are common gaps.
Monitor and hand over to operations
Monitoring covers broker health, under-replicated partitions, consumer lag, quota breaches, certificate expiry and storage growth per tenant, with alert routing to the on-call team. Runbooks cover common failures, upgrades and scaling of partitions or bookies. Operations staff run a supervised period in which the partner resolves incidents while the internal team takes the lead. The output is a handover pack containing architecture, configuration baselines, runbooks, alert definitions, the upgrade path for the supported release line, and a signed acceptance by the operations owner. Support continuation beyond handover is a separate, written agreement.

Evidence a SaaS customer or auditor will ask for from the backbone

This table maps each piece of evidence a SaaS customer or auditor will request to the component that produces it. DeployOpen prepares the systems, controls and records; independent auditors issue any SOC 2 report, and certification bodies issue any certificate. Use the table to check whether your backbone can produce each item before a customer security questionnaire or audit window arrives.

Evidence	What it shows	Producing component
SOC 2 system description and control design (Type 1)	The streaming platform's boundaries, data flows and controls as designed on a specific date, for the auditor to assess	Architecture documentation, control matrix and runbooks maintained by the platform team
Operating effectiveness over a period (Type 2)	Evidence that controls such as ACL reviews, monitoring and change approval operated consistently across the audit period	Ticketing, change records, alert history and access review logs retained for the full period
ACL configuration exports	Which tenant principals can produce to or consume from each topic, stream or account	Kafka StandardAuthorizer metadata, Pulsar authorization policies or NATS account and user configuration
TLS configuration	Encryption in transit between clients, brokers and tools, and which listeners are secured	Broker listener settings, certificate inventory and client trust configuration
Access reviews	Periodic confirmation that operators, service accounts and tenant credentials still need their access	Identity provider exports, cluster admin role lists and quarterly review sign-offs
Change and upgrade records	Version history, patch timing and rolling upgrade sequence for brokers, clients and Java runtimes	Release tracking, change tickets, upgrade test results and the version inventory of each cluster
Quota and tenant isolation settings	How one customer's traffic is prevented from degrading another tenant's streams	Per-tenant quotas, namespace or account limits and capacity monitoring reports
Data processing instructions under GDPR Article 28(3)	Documented instructions the processor follows, including limits on transfers and security measures under Article 32	Data processing agreement, instruction register and data location records for topics and backups
CRA Article 14 reporting readiness	Process to report actively exploited vulnerabilities and severe incidents for software placed on the market	Vulnerability intake, triage runbook, notification contacts and an incident record template agreed with counsel

Questions to ask an implementation partner before signing

Start with tenant isolation, because it decides whether one customer can affect another. Ask the partner to show, for your chosen platform, how tenants are separated: on Kafka that usually means per-tenant topic naming, prefixed or resource-scoped ACLs, and separate principals for each service, while on NATS it means accounts with explicit imports and exports. Ask who designed the authorization model, how a new tenant is provisioned without a manual ACL edit by an operator, and what happens to a tenant's ACLs when it is deleted. For Kafka, ask how the partner handles the rule that KRaft StandardAuthorizer resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, and confirm that the setting is false in the proposed design. Ask for a written test plan that proves a tenant principal cannot read or produce to another tenant's topics.

Next, ask how quotas are enforced and tested. Request the quota types they will configure (produce and fetch byte rates, request rates or partition limits), where those limits are stored, and the test they will run under load to show a noisy tenant is throttled without starving others. Ask for the schema governance handover in concrete terms: which registry is used, who owns compatibility rules, how breaking changes are reviewed, and what documentation and runbooks your engineers receive at close. Ask about the upgrade path too. The Kafka release plan supports rolling upgrades from releases within the past year to the latest version, and Kafka 4.x requires KRaft, so ask the partner to state the current supported line, the Java version they will deploy (Java 17 for brokers and Kafka Connect since Kafka 4.0), and the test environment where each upgrade is rehearsed before production.

Finally, settle evidence, exit and scope before any work starts. For SOC 2 controls, ask which evidence the partner will produce for the CC common criteria and which ones remain your responsibility, such as access reviews, change tickets and log retention. Be clear that DeployOpen prepares controls and evidence but does not issue an examination report, so the independent auditor decides what is sufficient. On exit, ask how topic data, consumer offsets, schemas and configuration can be exported in usable form, and how this maps to the switching rights in the EU Data Act (Regulation (EU) 2023/2854), which requires providers to allow switching to another provider or to on-premises infrastructure. Last, insist on a written, fixed scope listing deliverables, acceptance tests, responsibilities after handover and what a retainer for upgrades and support would cover. Anything that is left as "to be determined" is a cost risk you will carry.

How much work is it to run a SaaS event backbone in-house?

What drives the total cost of running a SaaS event backbone yourself?

The total cost is mostly people, storage and replication, and it depends on throughput, retention and the number of tenants you must isolate. Self-hosting shifts cost from a managed service fee to your own cloud or data center spend, engineering time for installation and upgrades, on-call coverage, and monitoring tooling. Broker and bookie or storage capacity recurs every month and grows with retention. Cross-zone traffic and cross-region replication add network charges that often surprise teams. Ask any partner for a quote that separates one-time setup from recurring infrastructure, support and retainer fees, and that states which cloud or hardware assumptions it uses. Compare that quote with your current managed Kafka invoice using the same throughput and retention figures.

How long does it take to get a self-hosted streaming platform into production?

The timeline depends on written scope, so no fixed duration applies across SaaS platforms. The main variables are how many clusters you need, whether tenancy is handled by topics, accounts or separate clusters, how much identity and TLS integration exists already, and how many schemas and producers must be migrated. Ask for a phased plan with explicit gates: a reference cluster, security baseline, a load test at expected peak, a failover test, and a controlled migration of the first tenants. Treat any partner who offers a date without a scope document as unable to estimate your work. Agree the acceptance criteria before work starts so that the schedule reflects what you actually asked for.

What must our own team provide when we run the event platform?

Your team must own the platform outcome even when a partner builds it. Plan for platform engineers who understand broker configuration, storage, networking and upgrades, and for an on-call rota that can respond to broker failures at any hour. Product owners must govern schemas, topic naming, retention and consumer contracts, because those decisions affect every tenant. Security staff need to own authentication, ACLs and key management. A common failure is treating schema governance as an engineering task only, which leads to breaking changes reaching paying customers. Document these roles in a RACI before handover so responsibilities are clear on day one.

Can our own engineers run Kafka or Pulsar without outside help?

Yes, many teams do, but the answer depends on existing skills and tolerance for operational risk. Apache Kafka is licensed under Apache-2.0 and has no paid edition from the project itself, so the software cost is zero while labor is not. Kafka 4.x runs only in KRaft mode, and clusters on ZooKeeper must migrate before upgrading, which is a specialist task. Pulsar requires operating brokers, BookKeeper bookies and a metadata store, which adds components to monitor. Ask whether your team has run a production cluster through a rolling upgrade and a failover test. If not, a partner can handle the first build and hand over runbooks, with the scope defined in writing.

What does a partner cover, and what stays with our team?

A typical fixed-scope engagement with DeployOpen covers planning, implementation, customization, integration, hardening, backup, upgrades, documentation and support of a self-hosted open-source platform inside your own cloud accounts, data centers or air-gapped networks. Engagements are scoped and priced in writing before work starts. Schema governance decisions, business ownership of topics, and on-call accountability usually remain with your organization, and the written scope should say so. Retainers can cover ongoing support, upgrades and continuous testing after handover. Ask any partner to list each responsibility and name the party that holds it.

How do we exit or switch providers without losing data or customers?

Exit terms are set by your contracts and by regulation, so review them before you commit. The EU Data Act, Regulation (EU) 2023/2854, requires data processing service providers to let customers switch to another provider or to on-premises infrastructure, with a maximum two-month notice period and a 30-day transitional period. From 12 January 2027, providers may not impose switching charges, and until then reduced charges may not exceed costs directly linked to switching. Self-hosting on open-source software keeps the format of your streams and the client protocol under your control, which simplifies moving between clusters. Confirm in writing how topics, offsets, schemas and access policies are exported, and who pays for the migration.

When is managed Kafka still the better fit for a SaaS product?

Managed Kafka remains a sound choice when your team is small, your throughput is modest, and the event backbone is not a differentiator for your product. It also fits when you lack capacity for 24 hour on-call and do not need control over broker versions, network placement or data residency. Self-hosting becomes more attractive when tenant isolation must be enforced at the cluster level, when data residency or regulation requires control of the infrastructure, or when volumes make managed pricing a large recurring cost. Compare both options using the same throughput, retention, replication and staffing assumptions, and include on-call and upgrade effort in the self-hosted side.

What are the first operational tasks a team underestimates after launch?

Teams most often underestimate upgrades, capacity growth and schema drift. Kafka supports rolling upgrades from each release in the past year to the latest version, but each step still needs testing against your clients and connectors. Clusters older than 3.3.x are advised to move to 3.9.x before 4.x, and Java 17 is required for brokers and Kafka Connect since Kafka 4.0. Pulsar publishes LTS releases every 18 months with 24 months of active support, so plan an upgrade calendar. Partition scaling changes key-to-partition mapping, which affects ordering. Build these tasks into the operating model and budget before go-live.

How DeployOpen scopes and supports a SaaS event backbone

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source streaming inside the client's own cloud accounts or data centers. For a SaaS platform, the work starts with a written scope that names the broker platform, the tenancy model (topics, namespaces or NATS accounts), the regions and accounts in play, the connectors and client libraries that must work, and who holds the operating roles after handover. Most engagements are fixed-scope, so deliverables, acceptance tests and timelines are agreed in writing before any build starts. Ask the partner to list every component in that scope, including the metadata layer, monitoring, backup tooling and the schema registry, because items left out of the scope tend to become the client's unplanned work later. Hardening covers the controls a multi-tenant backbone depends on: TLS between clients and brokers, SASL or OAuth-based client authentication, ACLs or account-level isolation per tenant, quotas, and secrets handling for each tenant's credentials. DeployOpen also runs security testing against the backbone itself, including the client-facing endpoints, the administrative APIs and the tenant boundary, so that isolation claims are tested rather than assumed. The written deliverables should include runbooks for upgrades, partition changes and tenant onboarding, along with the configuration baseline and the test results that show the controls were verified. DeployOpen prepares systems, controls and evidence for compliance assessments, such as configuration records, access reviews and log exports that an auditor can sample. It does not issue audit reports, attestations or certifications, and the independent auditor or certification body still decides what the evidence supports. Upgrades, continuous testing and ongoing operational support are available as retainers, which keeps the platform on supported releases after the initial project closes. Confirm with any partner which of these are fixed-scope deliverables and which are recurring retainer work, and ask for both in writing.

Event streaming for SaaS platforms