Real-time event streaming and messaging between systems on self-hosted Apache Kafka, Apache Pulsar, NATS or RabbitMQ, run in the organization's own data centers or cloud accounts instead of managed streaming services.
Streaming payments, trades and account events in banks and fintechs: ordering and exactly-once expectations, encryption and ACLs, retention for audit, cross-site replication for resilience, and DORA and PCI DSS evidence.
Moving admissions, orders, results and device events between clinical systems in real time: HL7 and FHIR integration patterns, PHI in topics, access control and retention under HIPAA.
Collecting machine and sensor data from plants: MQTT at the edge, brokers that bridge OT and IT networks, buffering when links drop, and feeding analytics and maintenance systems under ISA/IEC 62443 segmentation.
Streaming smart meter, grid sensor and SCADA-derived data: volume and retention planning, keeping streaming platforms outside or inside NERC CIP electronic security perimeters, and replication across control centers.
Running the event backbone of a SaaS product: multi-tenant topics or accounts, quotas, schema management, scaling partitions, and the operating cost of self-hosting compared with managed Kafka.
Running brokers on isolated networks: importing releases and connector plugins, one-way transfer between classification levels, monitoring without cloud tooling and keeping Java and broker versions patched.
Moving off US-hosted managed streaming to brokers under EU control: personal data in event payloads, self-hosting in EU facilities, replicating data across regions without unintended transfers.
Keeping streams available when a site fails: active-passive and active-active patterns, MirrorMaker 2 or Pulsar geo-replication, consumer offset translation, recovery objectives and failover tests.
Streaming row-level changes out of operational databases with Debezium on Kafka Connect: supported databases, impact on source systems, schema changes, snapshots and where CDC beats batch extraction.
When card data flows through topics: keeping the cluster in or out of the cardholder data environment, TLS between clients and brokers, ACLs, log retention and the evidence assessors sample.
For financial entities whose critical functions depend on streaming: backup and restoration of topics, redundant capacity, yearly continuity tests including switchover, and recovery objectives set per function.
Requirements to agree before choosing a streaming platform: throughput and retention, ordering and delivery guarantees, protocols such as Kafka API, AMQP or MQTT, multi-tenancy, security, geo-replication and operating skills.
Questions for partners proposing to build or run a streaming platform: sizing, upgrade path, security defaults, replication and recovery testing, monitoring and handover, with answers that should worry you.
Cost of running Kafka, Pulsar or NATS yourself against managed streaming: brokers and storage, cross-zone traffic, replication, people and on-call, and how throughput and retention drive each.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.