What event streaming really costs when you run it yourself
The software is free under its license, but the total cost depends on throughput, retention, replication, network placement and who operates the cluster. Apache Kafka and Apache Pulsar are both licensed under Apache-2.0, and NATS server is also Apache-2.0, so the core projects carry no license fee. The money goes elsewhere: compute and disks for brokers, storage that must survive a node loss, cross-zone or cross-site traffic, and the engineers who patch, upgrade and wake up for the cluster. Building in-house, engaging a partner to implement and hand over, and buying a managed streaming service each move these components to a different party, so compare them on the same line items rather than on the headline price. No price is given here because none has been supplied for this topic, and any figure you see from a vendor should be checked against the scope it assumes.
Start by writing down the inputs that drive every line: sustained and peak message rates, average and maximum message size, retention period per topic, replication factor and the number of availability zones or sites the cluster spans. Retention and replication multiply storage, and a replication factor above one means each message is written and copied across brokers, which adds network traffic that cloud providers may bill when it crosses zones. For Kafka, confirm the operating model before costing it: ZooKeeper-based clusters must migrate to KRaft before upgrading to 4.x, and Java 17 is required for brokers since Kafka 4.0, so the upgrade path is part of the labor estimate. For Pulsar, storage capacity grows by adding BookKeeper bookies independently of brokers, which changes how you budget compute and disk. For NATS, JetStream replication at R3 is the production floor, so plan for at least three servers per stream group.
Ask every bidder, whether an internal platform team, an implementation partner or a managed provider, to price the same workload and to list what is in and out of scope: monitoring, backup and restore tests, security hardening, TLS and ACL administration, 24x7 on-call, and upgrades. Separate one-time work (design, migration, initial hardening, handover) from recurring work (infrastructure, support, upgrades, testing). Request the assumptions for throughput and retention in writing, and check whether egress and cross-zone charges are included. A fixed-scope written agreement makes these comparisons fair, because the same deliverables and timelines can be priced for each option.
Which cost components sit with in-house, partner and SaaS
This table compares who carries each cost component when you run Kafka, Pulsar or NATS yourself, bring in a partner to build or run it, or buy a managed streaming service. Read each row for who pays and whether the cost is one-time or recurring, then confirm the split in writing with any partner or vendor before you compare totals.
| Cost component | In-house | Partner (fixed scope) | Managed SaaS |
|---|---|---|---|
| Brokers and compute | Your team pays cloud or hardware costs and sizes the node pools, recurring | Partner designs and sizes; you still pay the infrastructure, recurring | Vendor bills capacity, usually recurring and metered by the provider |
| Storage and retention | You pay disks or object storage for every retained copy, recurring and growing with retention | Same infrastructure bill as in-house; partner sets retention policy in writing | Vendor bills storage, often recurring and tied to retention settings |
| Cross-zone and cross-region traffic | Your cloud account pays network charges for replication and client traffic, recurring | You pay the same network charges; partner designs topology to limit them | Often included in the price or billed separately; confirm the traffic terms |
| Replication | You pay for extra brokers, copies and cross-site links, recurring | Partner sets replication factor and tests failover; you pay the extra capacity | Vendor sets replication; check whether multi-site replication is an add-on |
| Metadata and controller operations | Your engineers run KRaft, ZooKeeper or Oxia metadata stores and handle leader or quorum faults | Partner configures and monitors the metadata layer during the engagement, then hands over | Vendor operates the control plane; you have no direct cost for it |
| Upgrades and patching | Your team plans releases, rolling upgrades and Java updates, recurring effort per release | Partner performs upgrades under a retainer or a fixed-scope project, one-time or recurring | Vendor applies upgrades on its schedule; check maintenance windows and version control |
| Monitoring and observability | You pay for metrics, logs and alerting tools and the people who respond, recurring | Partner builds dashboards and alerts; tooling licenses or hosting stay with you | Basic metrics included; advanced tooling may be a separate recurring charge |
| Security tooling | You pay for certificates, secrets management, ACL administration and scanning, recurring | Partner hardens the cluster and tests it; ongoing tooling stays with you | Vendor covers platform security; you pay for your own identity and audit integrations |
| On-call and staffing | Your staff carry the largest recurring cost, including 24/7 rotation for production streams | Partner provides a handover and optional retainer; your staff still own incident response | Lower operating burden; you still need engineers for topics, schemas and consumers |
| Support and escalation | Community support or paid third-party support, recurring if you contract for it | Support is set by the written scope or retainer, one-time or recurring | Vendor support tiers, usually recurring; confirm response commitments in the contract |
Throughput, retention and replication: what moves the bill
Four cost drivers set the size of a self-hosted streaming platform: how much data moves, how long it stays, how many copies are kept and where those copies live. Each one also sets how much people time the platform consumes, so review them together before comparing a self-hosted build with a managed service.
Message throughput and partition count
Throughput sets how many brokers, how much network capacity and how much CPU the cluster needs at peak, not on average. Partition count sets how work spreads across brokers and consumers, and adding partitions later changes key-to-partition mapping for ordered data. Ask your partner to size against peak message rate and average message size, and to show the headroom kept for failover. Managed services usually bill throughput directly, so compare peak figures, not monthly averages.
Retention period and storage growth
Retention decides how much data sits on disk at any time, so storage cost grows with the product of ingest rate and retention window. Audit or replay requirements often push retention well beyond what consumers need for live processing. Pulsar adds storage by adding BookKeeper bookies independently of brokers, which lets storage grow without resizing compute. Confirm the retention period per topic, who owns that decision, and how much retained data must be restorable.
Replication factor and copies
Each extra replica multiplies storage and inter-broker traffic. NATS JetStream documentation calls R3 the production floor and limits a stream to at most five copies, so a three-node cluster with R3 keeps three copies of every stream. Your partner should show which streams need R3 and which can accept lower durability, since not every topic needs the same protection. Ask what replication costs in disk and network at your message rate, and which copies remain available during a node loss.
Number of sites and geo-replication
Every additional site adds a second or third cluster, its hardware, its monitoring and its replication link. Cross-site replication also adds sustained bandwidth charges in cloud accounts and ongoing operator effort to test failover. Kafka geo-replication and Pulsar geo-replication each carry their own configuration and offset handling. Ask whether the business needs active-passive or active-active operation, and price the replication link and failover test as separate line items.
Cross-zone traffic
In public clouds, traffic between availability zones is often billed separately from instance and storage charges, and it grows with replication and consumer fan-out. A three-zone cluster that replicates every message across zones and serves consumers in several zones can generate far more inter-zone traffic than the producers' own volume suggests. Ask your partner to model producer, replica and consumer traffic separately, and to confirm which zone consumers read from. Managed services may absorb some of this cost, so ask whether it appears in the quote.
Team skills and upgrade frequency
Self-hosting moves operating labor onto your team or your partner's retainer. Kafka releases three times a year on a four-month cadence, and rolling upgrades are supported from the last three releases, so upgrades are recurring work, not a one-time project. Kafka 4.0 removed ZooKeeper mode, and clusters older than 3.3.x are advised to move to 3.9.x before 4.x. Ask who performs upgrades, who is on call, and what evidence of a tested upgrade path the partner will provide.
Why the license line is the smallest line in self-hosted cost
The license line is zero for the core projects. Apache Kafka is released under the Apache License 2.0 and has no paid edition from the project itself, and Apache Pulsar and NATS are also Apache-2.0 licensed. That means the cost you will model is labor, infrastructure and vendor support contracts, not a software fee. Treat any vendor distribution or commercial support subscription as a separate line item to quote, and confirm its terms with the vendor rather than assuming them.
The recurring cost sits in upgrades. Kafka's project aims for three releases a year, one every four months, and rolling upgrades are supported from each release in the past year to the latest. Pulsar runs a different rhythm: its 5.0 LTS line has active support until October 5, 2028, while the 4.0 LTS line leaves active support on October 21, 2026, so a cluster still on 4.0 will need a planned move soon. Someone has to own these windows, test each release, and schedule rollouts before support lapses. Upgrade obligations also convert directly into engineering hours. Kafka 4.0 supports only KRaft mode, so any ZooKeeper-based cluster must migrate to KRaft before moving to 4.x, and the upgrade notes recommend moving KRaft clusters older than 3.3.x to 3.9.x first. Since Kafka 4.0, brokers and Kafka Connect require Java 17, which pulls in JVM patching and connector compatibility checks. Each of these steps needs a test cluster, a rollback plan and a change window, and the people doing that work are the largest recurring cost in self-hosting.
Build, partner or managed: which cost profile fits your team
Building in-house means your own engineers install, configure, upgrade and run the cluster in your data centers or cloud accounts, with no outside operator. The cost is mostly people and time, and it recurs every month because the cluster never stops needing attention. Your team owns every decision, from broker sizing to the authorization model, and every failure lands on your on-call rota.
- On-call: Your staff carry the pager for broker failures, disk pressure, under-replicated partitions and client lag, around the clock.
- Upgrades: Kafka 4.0 supports only KRaft mode and requires ZooKeeper-based clusters to migrate first, and rolling upgrades are supported from each release in the past year to the latest version, so your team must plan and test each step.
- Security configuration: Your team chooses among the documented SASL mechanisms (GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER), enables TLS, and writes ACLs in the KRaft authorizer, where resources without ACLs are restricted to super users unless you change that default.
- Capacity planning: Your team forecasts throughput, retention and partition counts, then re-plans as volumes grow.
Confirm internally that you can staff the on-call and upgrade cadence for the life of the platform, and that you have a documented runbook for restoring topics.
With an implementation partner, the cluster runs in your own accounts or data centers, and an outside team designs, builds, hardens, documents and hands over the platform. You keep data residency and control of the infrastructure, while the partner's engagement covers the work your team cannot staff at launch. The scope, deliverables and timelines are agreed in writing before work starts, and ongoing support is a separate, recurring arrangement.
- On-call: Ask whether the partner provides out-of-hours response during the build, during handover, or under a retainer, and who holds the pager after handover.
- Upgrades: Agree who plans, tests and executes version moves and who signs off on production changes, since you still own the environment.
- Security configuration: The partner can set TLS, SASL, ACLs and secrets handling to your policy, and should hand over the configuration as code with an explanation of each setting.
- Capacity planning: Ask for a sizing model based on your throughput, retention and replication targets, and for a review schedule as volumes change.
- Handover: Agree what documentation, training and test evidence you receive, and what remains your responsibility afterward.
Confirm the scope boundary in writing, including who owns the cluster after go-live, who can access the brokers, and what a support retainer covers and excludes.
A managed streaming service moves most operating work to the vendor, who runs the brokers, storage and upgrades on its infrastructure, and bills for usage or capacity. Your recurring cost becomes the service fee plus any cross-zone or egress traffic, and your team focuses on topics, schemas and consumers. The trade-off is less control over where the data sits, which versions run, and how the security settings are configured.
- On-call: The vendor typically carries platform incidents, but your team still owns client failures, lag and application-side errors.
- Upgrades: The vendor schedules upgrades, so confirm the notice period, whether you can defer, and how breaking changes are communicated.
- Security configuration: Confirm which authentication methods, encryption settings, and ACL or role controls you can set, and whether you can export audit logs for your own retention.
- Capacity planning: Confirm how throughput and retention are metered, what limits apply, and how scaling works during traffic spikes.
- Data location: Confirm the regions where data is stored and replicated, and whether the vendor's support staff can access your data.
Confirm these points in the vendor's contract and technical documentation, not only in a sales conversation, and ask for the terms that apply when you leave the service.
How to build your own event streaming cost estimate
This sequence turns throughput, retention and site requirements into a cost model that you can compare line by line against self-hosted, partner-run and managed quotes. Each step produces an input or check that the next step depends on.
Define throughput and retention per topic
Start by listing every topic or stream, with its expected message rate, average and peak message size, number of producers and consumers, and how long each must be kept. Separate operational traffic from audit or replay needs, because retention often drives storage cost more than peak throughput does. Owners are the application teams that produce the events, reviewed with platform engineering. The step is done when each topic has a written row with rate, size, retention period and replay requirement, and the totals are agreed by both groups.
Choose the replication and site model
Decide how many copies of each partition or stream must exist, across how many zones, and whether a second site or region must hold a copy. Replication multiplies storage and inter-zone network traffic, so this choice changes the bill more than most teams expect. Confirm the replication settings your chosen platform supports for production, such as the replication factor and minimum in-sync replicas for Kafka, or the replica count for JetStream streams, and record the site topology. The output is a diagram and a table of copies per site that the sizing step can use.
Size brokers and storage from those inputs
Convert the throughput, message size, retention and replication figures into raw storage, then add headroom for growth, compaction, index overhead and rebalancing. Estimate broker or node count from sustained throughput and partition or stream counts, not from average load alone, and keep separate sizing for storage-heavy nodes where the platform separates compute from persistence, as Pulsar does with BookKeeper bookies. Ask your platform owner or implementation partner to show the sizing formula and assumptions in writing. The output is a bill of materials with node counts, CPU, memory, disk type and capacity per tier.
Price cross-zone and cross-region traffic
Multiply your replicated write volume and consumer fan-out by the data transfer rates in your cloud provider's current price list for traffic between zones, between regions and to the internet. Consumers in another zone or region can add more transfer than the brokers themselves, so map every producer and consumer to its zone before calculating. Pull rates directly from the provider's pricing page on the day you build the model, because these rates change and vary by provider and region. The output is a monthly transfer line per traffic path with the rate and its source noted.
Add people, on-call and skills cost
Estimate the operating effort for installation, configuration, monitoring, certificate and credential rotation, incident response, capacity changes and version upgrades. Include the on-call rota and the people who cover it, and be explicit about whether this work is internal staff, an implementation partner under a retainer, or a managed provider's responsibility. For a self-hosted platform, the upgrade path matters: for Kafka, for example, the project's upgrade notes state that ZooKeeper-based clusters must move to KRaft before upgrading to 4.x. The output is an hours-per-month estimate by role, with the one-time migration and build effort listed separately.
Add upgrade and testing cycles
Schedule the recurring work of applying supported releases, rehearsing rolling upgrades in a non-production environment, running failover and restore tests, and updating runbooks and evidence for auditors. Check the vendor's published support windows so you know when a version leaves support and forces an upgrade, and budget for the test environment that mirrors production sizing. Confirm with the project or vendor which releases are supported and how rolling upgrades are supported between them. The output is a yearly calendar of upgrade and test events with the people and environment each one requires.
Compare against managed quotes on identical inputs
Give every managed provider, implementation partner and internal team the same throughput, retention, replication, site, traffic and support assumptions from the earlier steps, and ask for a written quote with scope and assumptions attached. Compare the totals by category: infrastructure, network transfer, licensing or support, people, and one-time work. Where a quote omits transfer, storage tiers, support levels or upgrade effort, ask for those lines before comparing. The output is a side-by-side table in which every number traces to a stated assumption.
Questions buyers ask before hiring outside help
What drives the cost of self-hosted Kafka?
Most of the cost comes from the infrastructure the cluster needs, the people who run it, and the traffic it generates. The main components are broker compute, disk for retention, replication factor, network traffic (especially cross-zone traffic in cloud accounts), monitoring and backup tooling, and the engineering time to upgrade, secure and recover the cluster. Throughput and retention multiply the first two: more data per second means more brokers and more network, and longer retention means more storage on every replica. Ask any partner to show each component as a separate line in the quote, with what is one-time (design, hardening, initial deployment) and what recurs (infrastructure, support, upgrades).
Is self-hosting Kafka cheaper than managed Kafka?
It depends on scale, skills and how much operational risk you are willing to carry, so there is no general answer. Apache Kafka is licensed under Apache-2.0 and has no paid edition from the project itself, so the software license is not the cost. Self-hosting shifts cost from a vendor's service fee to your cloud bills, your staff and your on-call rota. Managed services bundle those operations but charge for throughput, storage and networking in their own pricing model. The fair test is a side-by-side model using your expected throughput, retention, replication and staffing, including the cost of a failed upgrade or a lost site. Ask us to model both paths against your written requirements before deciding.
How much staff time does running Kafka take?
The time depends on how many clusters you run, how often you upgrade, how strict your security and recovery requirements are, and how much automation exists. Routine work includes monitoring, capacity reviews, topic and ACL management, certificate and credential rotation, and patching Java and brokers. Kafka's time-based release plan targets three releases a year, and rolling upgrades are supported from each release in the past year to the latest version, so upgrades are a recurring task rather than a one-off. Ask a partner to state the expected operating hours per month for your cluster, who covers on-call, and which tasks they would hand over to your team.
How long does a streaming rollout take, and what does it depend on?
The timeline depends on the number of clusters, the security model, integration count, the state of your infrastructure and how quickly decisions are made. Work typically covers requirements, sizing, deployment in your own cloud accounts or data centers, TLS and SASL setup, ACLs, monitoring, backup and restoration testing, and client onboarding. Migrating from ZooKeeper-based clusters adds a step: Kafka 4.0 supports only KRaft mode, so older clusters must move to KRaft before upgrading to 4.x. Most engagements are fixed-scope, so the duration appears in the written scope, which should list milestones, dependencies and acceptance criteria.
Can our own team run a self-hosted streaming platform?
A capable platform team can run it, provided it has Kafka-specific skills, documented runbooks and time reserved for operations. The hard parts are less the first deployment than the ongoing duties: upgrades across major versions, authorization in KRaft clusters (where resources without ACLs are restricted to super users unless configured otherwise), certificate rotation, capacity growth and recovery testing. Be honest about who is on call at night and who has done a broker recovery before. Many organizations split the work, with a partner handling design, hardening and handover and their team owning daily operations. Ask for a written handover plan with named skills.
What should we ask a partner to include in a streaming quote?
Ask for a written scope that names every deliverable, environment and exclusion, so you can compare proposals on the same basis. A useful quote separates one-time work (design, deployment, hardening, migration, testing) from recurring work (monitoring, upgrades, patching, on-call, backup verification). It should state the assumptions behind sizing, including throughput, retention, replication factor and number of clusters, and what changes the price. Ask who owns the infrastructure, how credentials and keys are handled, how upgrades and security patches are delivered, and what acceptance tests prove recovery. DeployOpen does not publish a price list, and a quote follows from the written scope.
How DeployOpen scopes and delivers a fixed-scope streaming implementation
DeployOpen plans, implements, hardens, documents and supports self-hosted streaming platforms inside your own cloud accounts or data centers. We start from a written scope that separates one-time build work from recurring operating costs, so the cost comparison between self-hosting, a partner-run cluster and a managed service rests on your numbers rather than ours.
How we scope
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
- The written scope lists the platform choice, expected throughput, retention periods, replication targets and the number of clusters to be run.
- We ask for your current managed streaming bills, cross-zone traffic and storage estimates so the cost model reflects your own environment.
- The scope names who operates the brokers, who holds on-call duty and what evidence the handover must include.
How we build
- We deploy Kafka, Pulsar or NATS into your accounts or data centers with brokers, storage and metadata components sized to the agreed throughput and retention.
- We configure TLS, SASL or other authentication and ACLs according to the scope, and document each setting that affects cost or security.
- We run load tests at the agreed message rates and record broker, storage and network measurements that feed the cost model.
- We set up monitoring and alerting for lag, disk growth and replication health so operating cost stays visible after go-live.
How we hand over
- Handover documents cover architecture, sizing assumptions, upgrade paths for the chosen release line and the recurring cost drivers.
- Your team receives runbooks for scaling brokers or storage, rotating credentials and restoring topics from backups.
- We walk your platform and on-call staff through the operating model so they can run the cluster without us.
- The handover includes a cost worksheet your finance and platform teams can update as throughput and retention change.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped separately from the build.
- Retainers can cover patching, version upgrades and recovery testing, with the frequency and response terms agreed in writing.
- We can re-run the cost comparison when throughput, retention or replication requirements change, as a scoped piece of work.
- Support questions are answered against your documented architecture, so changes are tracked in the same records as the original build.
