Event streaming RFP questions

A credible partner starts from a written message profile, not a product default: average and peak message rate, average and maximum message size, number of producers and consumer groups, and the retention period each topic needs for replay and audit. From those numbers they derive raw ingress, replication factor multiplied storage, and the headroom needed for failover and rebalancing. Ask for the calculation in a spreadsheet you can change. Red flags include sizing by copying a vendor sample configuration, quoting broker counts before seeing your message sizes, or giving no retention math at all. Illustrative: a topic at 10 MB per second with 7 days of retention and replication factor 3 stores roughly 6 TB before compression, which changes the hardware answer completely.

Good answers describe load tests run against your message profile: realistic payload sizes, your partition counts, consumer lag under sustained peak, and a broker failure during load. The partner should state pass criteria before testing and record results for handover. Ask whether tests include producer acknowledgement settings and consumer restarts, since those change behavior more than raw throughput. A red flag is a benchmark run on synthetic one-kilobyte messages with no consumer side measured, or a partner who will not share the test scripts. Ask for the test plan as a deliverable before any production sizing is signed off.

The partner should model storage growth per topic as ingress rate multiplied by retention, multiplied by replication factor, and show how partition count affects open file handles, memory and rebalance time. For Kafka, partition counts can be increased later but this changes key-to-partition mapping, so ask how they will handle ordering-sensitive topics. For Pulsar, storage grows by adding BookKeeper bookies independently of brokers, which the partner should explain if they propose it. Red flags: no alarm thresholds for disk, no tiered storage or archive plan for long retention, and partition counts chosen by habit. Ask how they will revisit the numbers after the first quarter of real data.

The answer should map your requirements to the platform: Kafka API ecosystem and connectors, Pulsar's multi-tenancy and topic and queue semantics, or NATS's small footprint, low latency and JetStream persistence with at-least-once delivery. Ask the partner to name which protocols you need (Kafka API, AMQP, MQTT), what ordering and delivery guarantees each consumer depends on, and whether exactly-once processing is required end to end. Red flags include choosing a platform before the requirements are reviewed, recommending one tool for every use case, or claiming guarantees without describing the configuration that provides them. Ask for a short decision record that explains what was rejected and why.

Start by checking the supported releases on the Apache Kafka downloads page and confirming which release your partner will deploy. The project lists 4.3.1, 4.2.2 and 4.1.2 as supported at the time of writing, and rolling upgrades are supported from each release in the past year to the latest version. Kafka 4.0 supports only KRaft mode, so ZooKeeper-based clusters must migrate first, and KRaft clusters older than 3.3.x should move to 3.9.x before 4.x. Since Kafka 4.0, brokers and Kafka Connect require Java 17. Red flags: a partner who proposes a legacy release with no upgrade plan, or who cannot name the Java version.

For Pulsar, ask which line is deployed and its support window: Pulsar 5.0 LTS was released on October 5, 2026 with active support until October 5, 2028, while Pulsar 4.0 LTS reaches the end of active support on October 21, 2026. Pulsar uses LTS releases every 18 months with 24 months of active and 36 months of security support. For NATS, ask which nats-server line the partner targets and how they track patches, since v2.15.1 and v2.14.8 were both published on October 9, 2026. Red flags: no written support end date, or a plan to run a release that is already out of support. Ask who pays for upgrades under the engagement and how they are scoped.

Ask for an honest skills map: broker operations, Java tuning for Kafka, BookKeeper and metadata store operations for Pulsar, or JetStream clustering and account configuration for NATS. A good partner lists which roles your staff need, which runbooks and dashboards they will receive, and what on-call coverage looks like after handover. Red flags include a handover that is a single training session with no runbook, or no plan for monitoring without the partner's tools. Ask whether the work is fixed scope or a retainer, and what support is included after go-live.

Ask the partner to state the exact version of every broker and client in your estate, then compare it against the supported releases on the Apache Kafka downloads page. As of the supplied release data, the supported lines are 4.3.1, 4.2.2 and 4.1.2, and all earlier versions are archived. A partner who cannot answer this from your inventory has not yet read your environment. Ask them to list each cluster, its version, whether it runs KRaft or ZooKeeper, and which clients and Kafka Connect workers depend on it. That list is the starting point for every later estimate of effort and risk.

If the cluster still uses ZooKeeper, it must move to KRaft before it can upgrade to any 4.x release, because Apache Kafka 4.0 supports only KRaft mode and removed ZooKeeper mode. Ask the partner for a written migration plan that covers the metadata migration, the rollback path if the migration stalls, and the validation steps for topics, ACLs, quotas and client connections afterward. The Apache Kafka upgrade notes are the primary reference for this step. A partner who proposes upgrading to 4.x first and migrating later has misread the requirement and should be questioned on the rest of the plan.

Yes. The Kafka upgrade notes recommend that KRaft clusters older than 3.3.x first upgrade to 3.9.x before moving to 4.x. A partner should show this as a named step in the plan, with a checkpoint after each hop, not as a single jump. Each hop needs its own compatibility review, client testing and a rollback point. Ask whether the partner has a written map of every hop from your current version to your target, and who signs off on each one. If the answer is to skip the intermediate versions because the upgrade seems simple, treat that as a serious red flag.

The Kafka time-based release plan says rolling upgrades are supported from each release in the past year, which covers the last three releases, to the latest version. Anything older falls outside that documented path and needs a plan of its own. Ask the partner to confirm which of your versions sit inside that window, and what they will do with the ones that do not. Also ask how the rolling upgrade will be checked, including producer and consumer lag, under-replicated partitions and client compatibility during the rolling window. Availability during the upgrade depends on your replication and client settings, so ask the partner to describe those settings for your clusters.

Since Kafka 4.0, brokers, Kafka Connect and tools require Java 17, while clients and Kafka Streams applications need at least Java 11. Many estates have older JVMs pinned on application servers, connector hosts or build pipelines, so this requirement often surfaces late. Ask the partner to inventory the JVM version on every host and application that touches Kafka, and to list the upgrades needed before the broker change. Ask who owns those application upgrades, since a broker upgrade plan that quietly assumes application teams will move their JVMs is incomplete. The DeployOpen engagement scope for each step should be written into the statement of work before work begins.

A credible plan has dates for each hop, a staging environment that mirrors production topology and data volumes closely enough to rehearse the change, and written go or no-go criteria for each stage. Expect a rollback plan with named triggers, such as sustained consumer lag beyond an agreed threshold, failed produce requests or client errors after a version change. Ask the partner to show a rehearsal log from a previous engagement, or an equivalent test record, rather than a slide. Ask who approves the rollback decision and how long a rollback takes for your data volumes. Timelines should come from the written scope, and any duration the partner quotes should explain what drives it.

Connectors, transforms, custom authorizers and client libraries each have their own version and compatibility matrix, and they often break before the broker does. Ask the partner to list every connector plugin and custom component with its exact version, its source, and the Kafka version it was tested against. Plugins that are unversioned, copied into a directory without a record or pinned to a snapshot build are a red flag. Ask how plugin artifacts are stored, how they are promoted from staging to production, and how an old artifact is restored if a new one fails. Also ask who owns connector compatibility after handover.

Be cautious when a partner proposes an in-place upgrade without first checking the supported upgrade paths, or when they cannot name the current version of your clusters. Also treat as a warning any plan that ignores the Java requirements, skips KRaft migration checks, lacks a staging rehearsal or has no rollback criteria. Vague answers such as ready to upgrade when you are, or no downtime expected, without a written basis, are signals to press harder. A good partner will document the plan in writing, state what is out of scope, and explain how upgrades will be tested and supported after go-live. Ask for those documents before comparing prices.

Critical topics should use a replication factor of at least three with a minimum in-sync replica count that lets producers keep writing through a single broker failure while still refusing writes that would not be durable. Ask the partner to show the actual values per topic, set through default.replication.factor and min.insync.replicas or per-topic overrides, and to explain what happens when a broker drops out. Replication factor on paper says little until someone has tested a broker loss under load and recorded whether producers stalled, lost acknowledgements or kept going. A partner who cannot name the values for your top ten topics has not designed the setting, only inherited the defaults.

Each platform has its own mechanism, and the partner should name it for each cluster in scope. Kafka cross-cluster replication is usually handled with MirrorMaker 2, documented in the Kafka geo-replication guide. Pulsar uses its built-in geo-replication between clusters. NATS JetStream replicates streams inside a cluster using Raft, with R3 as the production floor and at most five copies per stream, so cross-site design needs an explicit plan for the second site. For each one, ask how lag between sites is measured, how often it is alerted on, and who verified that data written at the primary arrives at the secondary without gaps or duplicates.

The business owner of each workload sets the recovery point objective (how much data loss is tolerable) and the recovery time objective (how long the stream can be down), and the platform team designs replication and failover to meet them. A partner should not pick these numbers for you. Ask for a register that lists each topic or stream, its owning function, the agreed objectives, and the design that backs them. Payment streams, clinical orders and telemetry rarely share the same targets, so a single cluster-wide figure is a warning sign. Objectives agreed in writing also become the pass criteria for the failover tests described below.

Failover drills should be scheduled, repeated and recorded, not run once at handover. Your organization should agree the frequency with the partner and the risk owner, and the test plan should name the scenarios: single broker loss, loss of a full site, and a controller or metadata failure. Each run should record the start and end time, the messages produced and consumed, any duplicates or gaps, the measured data loss against the recovery point objective, and the time to resume against the recovery time objective. Ask to see the last three drill reports and the open actions from them. Where a regulator or auditor will review resilience, the same records serve as evidence.

Consumer offsets are the most common cause of duplicated or skipped processing after a site switch, because the offset numbers on the primary cluster do not match the secondary. With Kafka and MirrorMaker 2, the offset translation for each consumer group has to be checked and applied during failover, and consumers must be ready to handle redelivery. Ask who owns this step, whether it is scripted and tested, and how the team confirms that each critical consumer group resumed at the right position. Consumers that are not idempotent need a documented plan for duplicates. If the partner cannot name an owner for offset translation, the failover plan is incomplete.

No. A backup that has never been restored is an unverified file, and a replicated cluster is not a backup, because replication copies mistakes and corruption along with good data. Ask the partner to describe how topic data, cluster configuration, ACLs and schemas are backed up separately, and how each piece is restored into an isolated environment on a schedule. The restore test should compare restored data against a known checkpoint and record the elapsed time. The DORA and PCI DSS sibling topics go further into evidence, but for this question the test is simple: a restore that was run, timed and written down.

Yes, if the handover covers runbooks, access, and a tested drill your staff can repeat without the partner in the room. Your team needs named operators, documented procedures for failover and offset translation, monitoring of replication lag, and a practiced restore path. The cost of this depends on how many people you will train, how often drills run, and whether a retainer covers standby support during drills. Ask the partner for the handover plan, the training scope in writing, and which drill steps they will still perform after handover. Keep in mind that the first few drills are where gaps surface, so budget for them to be observed rather than assumed to work.

Be cautious when a partner describes replication without any test history, names a replication factor but cannot say what min in-sync replicas is set to, or says backups are in place without a restore record. Also worry about recovery objectives that were never agreed with the business, a single failover runbook written for one scenario, and no named owner for consumer offset translation. Vague claims that a cluster is highly available or fault tolerant, without test data, are not evidence. Good answers are specific, show recent drill results, and identify who signs off on each objective.

Ask for a named set covering consumer lag, under-replicated partitions, broker disk usage and controller health on Kafka, backlog and BookKeeper bookie health on Pulsar, and stream replica health and Raft leader status on NATS JetStream. Each alert should state its threshold, its severity and the action expected. Ask the partner to show the alert list in your own monitoring tool, not only a dashboard screenshot. Confirm which metrics come from the broker itself and which need a separate exporter, because that changes what you must run and maintain.

The answer should name each exporter, its version and where it runs. For NATS, the NATS tooling includes a Prometheus exporter, so ask whether it is deployed and scraped in your environment. For Kafka and Pulsar, ask which exporter or agent exposes broker and client metrics to your monitoring stack, and who updates it when the broker is upgraded. An exporter nobody owns becomes a silent gap in alerting after the first upgrade. Ask for the scrape configuration and the dashboards as files your team keeps in version control.

Each runbook should cover one alert or failure mode, state the symptom, the first checks, the commands or console steps, the decision points, the escalation contact and the rollback. Typical topics include a broker that fails to rejoin, a consumer group that stops advancing, a full disk, a lost controller or quorum, and a failed certificate rotation. Ask the partner to show two or three runbooks in their current form. A good runbook names the exact configuration keys or commands for your version, such as the relevant Kafka or NATS settings, rather than generic advice.

The on-call model should name a primary and secondary team for each alert, the hours covered, and the boundary between your staff, the partner and the application teams that produce or consume events. Ask who responds when a consumer lag alert fires because a downstream application is slow, and who responds when a broker is unhealthy. Ownership written as a RACI table with named roles is more useful than a sentence saying the partner supports you. Support hours and response expectations belong in the written scope, and retainers are the usual vehicle for ongoing cover. Confirm the terms with the partner before signing.

A handover is accepted when your staff perform the tasks themselves and the results are checked against written criteria. Useful tests include restarting one broker or node while producers and consumers keep running, rotating a client certificate, adding a partition or stream replica, triggering a consumer lag alert and following its runbook, and restoring a topic or stream from backup into a test environment. Agree the pass criteria in writing before the test. A handover defined as a list of documents that nobody has executed is not an acceptance test.

Ask for a skills plan that names the people who will operate the platform, the sessions they attend, the labs they complete and the tasks they lead under observation before the partner steps back. Shadowing for one or two incident drills is more reliable than a classroom course alone. Ask how the partner records what each engineer can do, and how the plan covers staff turnover after handover. The effort depends on your team's existing Java, Linux, Kubernetes and networking experience, so confirm your starting point in the scoping discussion.

Be cautious if the partner delivers dashboards but no runbooks, if handover means a document dump with no tests, or if the partner has no plan for skills transfer. Also be wary of alerts without owners, of metrics that only the partner can read, and of any promise that the platform will not need on-call work. Ask whether monitoring data stays in your storage and whether you can export it if you change partners. An answer that depends on the partner staying indefinitely is a dependency you should price and document.

Monitoring and operations cost splits into one-time work (building dashboards, alerts, runbooks and handover tests) and recurring work (on-call cover, upgrades, certificate rotation, capacity reviews and runbook updates). The recurring share grows with the number of clusters, tenants, topics and integrations, and with how many environments you run. Ask the partner to separate these in the quote, to name who is paid for each line, and to state whether support is a retainer or a separate engagement. Your own staff cost for on-call should be included in your internal comparison, since it is the cost most often left out.