Apache Kafka vs Redis Streams: when a Redis stream is enough
A Redis stream is enough when you already run Redis, your event history is short, and a small number of services consume the same messages. Redis Streams behaves like an append-only log with consumer groups, which covers many internal fan-out cases. Apache Kafka fits when the event log itself is a shared platform: many teams and systems read the same topics, history must be kept for long periods, and replay is a routine operation. Redis Streams also gives you at-least-once delivery, while Redis Pub/Sub is at-most-once, so a plain Pub/Sub channel should not carry events you cannot afford to lose. Confirm the delivery and acknowledgement behavior you need with a test against your own consumers, not only the summary.
Four questions decide most of this. First, how long must messages stay available: hours for a work queue, or months for audit and reprocessing? Second, will you need to replay from an earlier point after a consumer bug is fixed? Third, how many independent systems will consume each stream, and will new consumers appear later? Fourth, what durability do you expect from replication and failover? For Redis, Redis Sentinel provides automatic failover when you are not using Redis Cluster, and its docs say a robust deployment needs at least three Sentinel instances. Map your durability requirement against that topology and against Kafka's replication settings before you choose.
Licensing is the last filter. Redis Open Source 8.0 and later is offered under a license of your choice among RSALv2, SSPLv1 and AGPLv3, and the Redis licenses page says Redis Ltd. plans to keep Redis Open Source under AGPLv3. Apache Kafka is Apache-2.0 with no paid edition from the project. Have legal review the Redis license terms against how you plan to offer the service, and confirm the exact Redis version and license option you will deploy. A useful test is a short proof of concept: run your heaviest consumer group, force a restart and a failover, and check what is redelivered and what is lost.
Which scenarios favor Redis Streams and which favor Kafka
This table maps common scenarios to the platform that fits them better and the condition that tips the choice. Read each row as a test: if the condition describes your team, the named option is usually the simpler and safer starting point.
| Scenario | Better fit | Condition that makes it so |
|---|---|---|
| Session or task queue inside an existing Redis deployment | Redis Streams | Redis already runs in your stack, workloads are modest, and losing a small amount of in-flight work during failover can be handled by the application. Adding a second platform would cost more than the queue is worth. |
| Service-to-service events with short retention | Redis Streams | Events only need to be consumed within a short window, consumer groups cover your delivery needs, and you can size memory for the stream. Confirm the failover behavior and acknowledgment handling your application needs before relying on it. |
| Multi-team event backbone with long retention and replay | Apache Kafka | Several teams must read the same events independently, at their own pace, and replay history weeks or months later. Kafka stores events on disk as a partitioned log designed for this use, so retention is a configuration decision rather than a memory budget. |
| Integration with many source and sink systems | Apache Kafka | You need to move data between databases, search indexes, data warehouses and other systems. Kafka Connect provides source and sink connectors managed through a REST API, while Redis Streams would need custom producers and consumers for each system. |
| Strict durability expectations across failures | Apache Kafka | Your business requires that acknowledged events survive broker loss and that you can explain replication settings to an auditor. Confirm the replication and acknowledgment configuration with your own architects, since Redis durability depends on how persistence and replicas are set up. |
| Small team that wants one operational skill set | Redis Streams | Your engineers already run Redis for caching and operate it with Sentinel or managed services. Kafka adds a separate cluster to operate, secure and upgrade, which is a real cost for a small platform team. |
| Security and access control at the event layer | Apache Kafka | You need per-topic ACLs, SASL authentication and TLS between clients and brokers managed as a documented part of the platform. Confirm which Redis deployment options and access controls match your own security requirements before choosing. |
How Kafka and Redis Streams are built and what that means for operations
Kafka and Redis Streams both give you an append-only log with consumer groups, but they get there differently. Kafka is a dedicated partitioned log cluster with its own metadata quorum, while a Redis stream is one data structure inside a Redis server that also holds your other data. Each card below ties one architectural choice to what your operations team must run, monitor and recover.
Partitioned log and consumer groups
Kafka splits a topic into partitions, and each partition is an ordered log that brokers store on disk. Consumer groups share partitions among instances, so the partition count caps how many consumers in one group can work in parallel. Your team must plan partition counts up front, watch consumer lag per group, and decide how replay will work after a consumer fails. A Redis stream also supports consumer groups with per-consumer pending lists, but it is a single key in one keyspace, so the work to scale and monitor it sits with the Redis instance that holds it.
KRaft-only metadata since Kafka 4.0
Apache Kafka 4.0 supports only KRaft mode, and ZooKeeper mode has been removed. Cluster metadata lives in a controller quorum inside Kafka itself, so there is no separate ZooKeeper ensemble to patch and back up. Clusters still on ZooKeeper must migrate to KRaft before upgrading to 4.x, and the upgrade notes recommend that KRaft clusters older than 3.3.x move to 3.9.x first. Confirm your current version and migration path before you plan any upgrade.
Replication and durability settings
Kafka replicates each partition across brokers and lets producers choose how many replicas must confirm a write before it counts as stored. These settings are part of the design, so an operator must set and review them per topic and per cluster. Redis Streams are replicated through Redis's own replication to replicas, and durability depends on how the server persists data to disk. Ask your team to document, for each stream or topic, what a lost write would cost and which settings protect against it.
Redis Sentinel for high availability
Redis Sentinel handles high availability for Redis when you are not using Redis Cluster. It watches the primary and promotes a replica automatically if the primary fails. The Redis Sentinel documentation says a robust deployment needs at least three Sentinel instances, so you must run and monitor those three processes separately from the data nodes. Failover can still lose writes that the old primary had not yet sent to its replicas, so test failover on your own environment before you rely on it.
Security and access controls
Kafka authenticates clients and brokers with SSL or SASL, using mechanisms such as SCRAM-SHA-512 or OAUTHBEARER, and in KRaft clusters the StandardAuthorizer stores ACLs in the cluster metadata log. Resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. Your team must define these ACLs as code, rotate credentials and audit who can produce or consume each topic. Confirm how your Redis deployment handles authentication, network exposure and access rules, since those controls are configured on the Redis server itself.
Upgrades and version support
Kafka publishes three releases a year and supports rolling upgrades from each release in the past year to the latest version. The currently supported lines are 4.3.1, 4.2.2 and 4.1.2, so your team must schedule upgrades before a line leaves support. Redis 8 is the current major line, with 8.10.0 released on July 29, 2026, and security releases arrive on their own schedule. Build a routine for tracking security releases on both platforms and test each upgrade in a staging cluster.
How durability and replication differ when a broker or node fails
The key question is where a message is acknowledged and how many copies exist when a node dies. In Kafka, durability comes from partition replication across brokers, governed by settings you choose per topic and per producer. Confirm the replication factor, the minimum in-sync replica setting and producer acknowledgement behavior in the Kafka 4.3 documentation before you assume a message survives a broker loss. In Redis Streams, entries are persisted and consumers receive at-least-once delivery, so a message can be redelivered after a crash. When you do not run Redis Cluster, Redis Sentinel handles failover by promoting a replica to master automatically. The Redis Sentinel docs say a robust deployment needs at least three Sentinel instances, so plan that quorum from the start.
Redis Pub/Sub is a different case. It offers at-most-once delivery, so a message is lost if the subscriber cannot handle it. Moving to Redis Streams improves the guarantee, but it does not remove the need for idempotent processing in your consumers. At-least-once delivery means a retry or a failover can deliver the same event twice. Illustrative example: an order service stores the event ID of each applied update and skips any ID it has already recorded. The same pattern applies to Kafka consumers, since neither platform removes duplicates for you. Check how each platform reclaims unacknowledged messages and how consumer groups track acknowledgements in your version, then test the behavior with a deliberate failover.
Access controls also shape what a failure exposes. Kafka clients, brokers and tools authenticate with SSL or SASL, and the documented SASL mechanisms include GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. Security is optional in Kafka, so confirm that every listener you rely on is secured. In KRaft clusters, the StandardAuthorizer keeps ACLs in the metadata log, and resources without ACLs are limited to super users unless allow.everyone.if.no.acl.found is set to true. For Redis, confirm the authentication, ACL and TLS settings for your version and whether Sentinel connections are authenticated. Then run a failure test in a non-production environment: stop a Kafka partition leader and a Redis master, and record any lost or duplicated events.
Capabilities side by side for event logs and consumer groups
Both systems give you an append-only log that several consumer groups can read independently, which covers many event-driven designs. Redis Streams offers consumer groups for distributing entries across workers, and Redis points to Streams as the stronger option over Pub/Sub because Streams persist messages and support at-least-once delivery. Kafka organizes parallelism around partitions, so ordering and throughput depend on how you partition topics.
- Delivery semantics: Redis Streams supports at-least-once delivery through consumer group acknowledgement, so your consumers must tolerate duplicates.
- Parallelism model: Kafka scales consumers by assigning partitions to group members, which makes partition count a capacity decision you make early.
- Ordering: Kafka guarantees order within a partition, so confirm how your key design affects event order across producers.
- Pub/Sub contrast: Redis Pub/Sub is at-most-once, so a subscriber that fails to handle a message loses it, which rules it out for durable event flows.
- Testing: Build a failure test that kills a consumer mid-batch and confirm redelivery and duplicate handling in both platforms.
Confirm exact redelivery and offset behavior against the Redis Streams documentation and the Kafka consumer documentation for your client library version.
Redis describes a stream as an append-only log with O(1) random access, which suits recent event history and real-time syndication. Kafka is designed as a distributed log where retention is configured per topic and consumers can rewind to earlier offsets. The question is how much history you must replay, for how long, and whether that history must survive a cluster failure without loss.
- Retention configuration: Kafka retention is set per topic by time or size, so you can keep different event classes for different periods.
- Replay: Kafka consumers can reset to an earlier offset, which supports reprocessing after a bug fix or a new downstream system.
- Memory pressure: Redis keeps stream data in memory, so retention length is bounded by the memory you provision and the trimming policy you apply.
- Event sourcing: Redis documents event sourcing as a use case, but a long replayable history needs a sizing plan for memory and persistence.
- Check: Measure your required replay window in days and bytes, then confirm the retention settings for each platform in your own environment.
Ask your Redis and Kafka teams to size retention for your largest topic before choosing a platform.
Kafka Connect is part of Apache Kafka and streams data between Kafka and other systems through source and sink connectors. It runs in standalone or distributed mode and is managed through a REST API. Redis has a broad module ecosystem, including JSON, time series, probabilistic structures and the Redis Query Engine, but it does not provide a connector framework comparable to Kafka Connect in the supplied facts. Your integration needs decide which ecosystem matters more.
- Connector framework: Kafka Connect provides source and sink connectors with a managed REST API for deployment and status.
- Stream processing: Confirm whether your team needs Kafka Streams, which requires at least Java 11 for applications, before committing to Kafka.
- Redis modules: Redis Open Source includes JSON, time series, probabilistic structures, vector sets in beta and the Redis Query Engine, so some data needs may be met inside Redis.
- Client libraries: Check that your language client supports the consumer group and acknowledgement features you rely on.
- Integration cost: Connector development and maintenance is a recurring effort, so list every source and sink before estimating it.
Confirm available connectors and their maintainers for each system with the vendor or community projects you plan to use.
Kafka authenticates clients, brokers and tools with SSL or SASL, and the documented SASL mechanisms include GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. Kafka encrypts traffic with SSL/TLS, but security is optional, so clusters can run unsecured or mix secured and unsecured listeners. In KRaft clusters, the StandardAuthorizer stores ACLs in the cluster metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. Redis deployment options differ, and you should verify each control against your own configuration.
- Kafka authentication: Choose SSL, or SASL with OAUTHBEARER if you need token-based identity from your identity provider.
- Kafka authorization: ACLs live in the KRaft metadata log, so review
allow.everyone.if.no.acl.foundand keep it at its restrictive default unless you have a reason to change it. - Kafka encryption: Enforce SSL/TLS on every listener, because the default allows unsecured clusters.
- Redis high availability: Redis Sentinel provides automatic failover when you are not using Redis Cluster, and its docs call for at least three Sentinel instances.
- Verification: Confirm Redis ACL, TLS and authentication settings for your version against the Redis documentation, and record the evidence your auditors will request.
Ask your security team to write the authentication, encryption and authorization baseline for each platform before a pilot begins.
Apache Kafka 4.x requires Java 17 for brokers, Kafka Connect and tools, and Kafka 4.0 supports only KRaft mode, so ZooKeeper-based clusters must migrate first. The supported releases on the Kafka downloads page are 4.3.1, 4.2.2 and 4.1.2. Redis Open Source runs on Linux, macOS, OpenBSD, NetBSD and FreeBSD, and on Windows only through Docker. Platform fit affects your base image, patching process and the team that will own the cluster.
- Kafka runtime: Plan Java 17 for brokers and Connect, and confirm the JVM version your build pipeline and security scanner support.
- Kafka upgrade path: Clusters older than 3.3.x should upgrade to 3.9.x before moving to 4.x, according to the upgrade notes.
- Redis operating systems: Linux is the common production target, while Windows requires Docker, so confirm that Docker is approved in your environment.
- Air-gapped delivery: Confirm how you will mirror binaries, container images and dependencies into an isolated network for each platform.
- Lifecycle: Track supported release windows for both projects and schedule upgrades before a line leaves support.
Confirm your operating system, container and JVM standards with your platform team before selecting a version.
What running Kafka 4.x takes compared with running Redis
Running Kafka 4.x starts with a version decision that Redis does not impose. Apache Kafka 4.0 supports only KRaft mode and removed ZooKeeper mode, so any cluster still on ZooKeeper must migrate to KRaft before it upgrades to 4.x. The Kafka upgrade notes also recommend that KRaft clusters older than 3.3.x first move to 3.9.x before going to 4.x. Confirm your current version and metadata mode before you plan a target release, and check the Apache Kafka 4.3 upgrade notes for the exact sequence. Brokers and Kafka Connect require Java 17 from 4.0 onward, so a JVM baseline and its patching belong in the operating plan from day one.
Release cadence is the recurring cost. The Kafka project aims for three releases a year, one every four months, and the supported lines at the time of writing are 4.3.1 (June 25, 2026), 4.2.2 (September 29, 2026) and 4.1.2 (March 17, 2026). Rolling upgrades are supported from each release in the past year to the latest version, so a team that falls behind the window loses the simple path and must plan a staged upgrade. Redis moves on a different rhythm: the current line reached 8.10.0 on July 29, 2026, and security releases 8.10.2, 8.8.3, 8.6.7, 8.4.7 and 8.2.10 shipped together on September 17, 2026. A Redis patch cycle is usually quicker to apply to a single-purpose instance, but each branch you run needs its own tracking. Ask whether your team will track both release lines in its change calendar.
The effort difference is mostly in the number of moving parts, not in any single upgrade. Kafka's KRaft controllers, broker configuration, authentication and authorization, and TLS listeners are settings your team owns and tests after every change, and in KRaft clusters ACLs live in the metadata log, with resources unprotected by default unless allow.everyone.if.no.acl.found is set to true. Redis Sentinel adds its own deployment decision, since a robust Sentinel setup needs at least three instances. Before committing, ask a vendor or your own platform team to walk through an upgrade on a test cluster and to document the rollback path, the time each step takes in your environment, and who is on call for it.
What drives the running cost of each platform
This table lists the cost components for running Redis Streams and Apache Kafka, who is paid for each one, and whether it is one-time or recurring. Use it to build a quote request, since most of the difference comes from memory sizing on the Redis side and broker, disk and skills on the Kafka side.
| Cost component | Redis Streams | Apache Kafka |
|---|---|---|
| Infrastructure | Servers sized mainly around RAM, since the dataset and retained stream entries live in memory. Recurring. | Brokers and controllers sized around disk throughput, retention period and replication factor. Recurring, with storage growing as retention grows. |
| Replication and failover nodes | Replicas and at least three Sentinel instances for a robust failover setup, per the Sentinel docs. Recurring. | Replicated partitions across brokers plus a KRaft controller quorum. Confirm node counts with your architect before quoting. |
| Licensing | Redis Open Source 8.0 and later is available under a license of your choice (RSALv2, SSPLv1 or AGPLv3). Confirm which license your legal team accepts for your deployment model. Redis 7.2 and earlier remain BSD-3-Clause. | Apache-2.0 with no paid edition from the project itself, so license cost is zero. Third-party distributions and managed services are separate costs to confirm. |
| Commercial options | Redis Software (self-managed, with added compliance, reliability and resiliency features) and Redis Cloud (managed on AWS, Google Cloud and Azure) are paid. Ask for the feature list and term length in writing. | No vendor-backed commercial edition from the Apache project. Paid support comes from third parties or a services partner, so ask each provider what the retainer covers. |
| Skills and operating effort | Lower operating surface for simple queues and caches, but memory tuning, persistence settings and failover testing still need an owner. Recurring. | Requires expertise in partitioning, consumer lag, KRaft, retention and security configuration. Recurring, and usually the largest share of the running cost. |
| Upgrade effort | Moving between Redis lines and reviewing license terms per release. Confirm the support window for your version before planning. | Kafka releases three times a year and supports rolling upgrades from the last three releases, so plan a recurring upgrade cycle. Prerequisite steps apply for older KRaft clusters. |
| One-time versus recurring | One-time: design, migration of consumers, failover testing and license review. Recurring: memory, hosting, support and upgrades. | One-time: cluster design, security setup, connector build and migration. Recurring: broker storage, staff time, upgrades and support. |
| What to ask for in a quote | Written scope covering memory sizing assumptions, Sentinel topology, license choice, support hours and the upgrade path for the Redis version you select. | Written scope covering broker and controller counts, retention targets, storage estimate, security configuration, upgrade plan and handover of runbooks. |
Moving from Redis Streams to Kafka, or back, in order
This sequence moves event flows between a Redis stream and a Kafka topic without losing messages or duplicating side effects, and each step ends with a check that the previous one held.
Inventory consumers and retention needs
List every producer, consumer group and downstream system that reads each stream key, along with how long messages must remain readable for replay or audit. Your platform engineering team and the owners of each consumer do this together, usually by reading stream metadata and consumer group state in Redis and the application code that calls XREADGROUP or XACK. The step is done when every consumer group has a named owner and a written retention requirement that you can compare against Kafka's retention settings. Confirm the retention options with the Kafka documentation for your target release before you set any value.
Map stream keys and groups to topics
Translate each stream key into a Kafka topic and decide how many partitions it needs, based on the ordering rules your consumers depend on. Redis streams are single keys with entry IDs, while Kafka orders messages only within a partition, so any consumer that relied on global order must be redesigned around a partition key. Consumer groups map to Kafka consumer groups, but offsets are not carried over automatically. The step is done when a mapping document lists each source key, target topic, partition key and ordering guarantee, and a platform engineer has reviewed it.
Set ACLs, schemas and connector plans
Define the access model for the new topics before any traffic flows. In KRaft clusters the StandardAuthorizer stores ACLs in the metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so every producer and consumer principal needs explicit grants. Decide which schemas will be carried over as they are and which will be formalized, and identify any integration that needs a Kafka Connect source or sink instead of custom code. The step is done when a security reviewer has signed off the principal list and each topic has an ACL entry.
Decide delivery and idempotency handling
Set the delivery expectations each consumer needs and make side effects safe under redelivery. Redis Pub/Sub is at-most-once, while Redis Streams with consumer groups give at-least-once delivery, and Kafka consumers also commit offsets, so a restart can replay messages. Record, for each consumer, the idempotency key, the deduplication store and the behavior on replay. Producers should be configured for the durability level your business requires, which you should confirm against the Kafka configuration reference for your release. The step is done when each consumer has a written replay rule and a test that sends the same message twice.
Run dual publishing or a bridge
Run both paths in parallel so the old and new systems receive the same events. Either the producer writes to the stream and the topic, or a small bridge process reads from the stream with consumer groups and republishes to Kafka. Compare counts, sampled payloads and consumer lag between the two paths over a period you agree with the business owners, since no duration is fixed here. The step is done when reconciliation shows no unexplained gaps and every discrepancy has a recorded cause.
Migrate consumers one group at a time
Move each consumer group to read from Kafka, starting with the least critical and keeping its Redis path available for rollback. Do not share one consumer group across both systems, because each group must track its own position. Confirm that the new consumer starts at the offset you chose, whether from the earliest retained message or from the cutover point, and that the idempotency controls from the earlier step are active. The step is done when each group has processed live traffic from Kafka, its error and lag metrics match the baseline, and its rollback path has been tested.
Retire the stream and keep the record
Stop writes to the Redis stream once every consumer group reads from Kafka and the rollback window you agreed has closed. Export the final stream entries and consumer group state to your archive, then remove the stream keys and related ACL or monitoring entries. Update the runbooks, dashboards and architecture records so that the Kafka topics, their ACLs and retention settings are documented as the system of record. The step is done when no active consumer reads the stream key, the archive has been verified, and the change record is complete.
Where Kafka and Redis Streams projects usually run long
The longest delays on a Kafka project are rarely broker installation. They come from security configuration and ACL design decided late. Kafka authenticates clients and brokers with SSL or SASL, and the documented SASL mechanisms include GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. In KRaft clusters, the StandardAuthorizer stores ACLs in the cluster metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true. Teams that leave topic and consumer group ACLs for after go-live often find that producers and consumers fail in ways that look like network faults. Before scoping, confirm which identity provider you will use, who owns topic naming, and how service accounts are provisioned and rotated. The Kafka authorization documentation sets out the ACL model you should map your roles to.
Schema and replay design is the second common source of rework. Kafka retains records for replay, so a consumer that is rebuilt later will read history that was written under older event shapes. Decide early how event schemas evolve, which consumer groups may reset offsets, and how replays are gated in production. On the connector side, Kafka Connect runs in standalone or distributed mode and is managed through a REST API. Each source and sink connector adds its own credentials, error handling and version pinning, so confirm which connectors you need, whether each is maintained by a party you trust, and who patches them. Effort here depends on the number of event types, the number of downstream consumers and your data classification, so ask for these counts in writing before a fixed scope is agreed.
Redis Streams projects run long for different reasons. Failover is the usual one. Redis Sentinel provides automatic failover that promotes a replica to master when Redis Cluster is not used, and the Sentinel documentation says a robust deployment needs at least three Sentinel instances. Plan a failover test that measures what consumers see during promotion, including whether unacknowledged entries in consumer group pending lists are redelivered and how your application handles duplicates, since Streams offer at-least-once delivery. Finally, review licensing before you deploy. Redis 8.0 and later can be used under RSALv2, SSPLv1 or AGPLv3 at your choice, and the Redis licenses page explains how those terms apply if you run modified Redis code as a network service. Your legal and procurement teams should confirm which license option fits your use, and the Redis licenses page is the primary reference for that review. DeployOpen plans and implements these pieces inside your own infrastructure under fixed scope, and the effort depends on the written scope and your environment.
Questions buyers ask before choosing Kafka or Redis Streams
Are Redis Streams enough for event sourcing?
Often yes, when the event volume is modest, the consumers live close to one Redis deployment, and your team already runs Redis well. Redis documents streams as an append-only log with consumer groups, used for event sourcing and real-time event syndication, so the data model fits. The questions that decide it are how long events must be kept, how many independent consumers replay them, and how much you can tolerate a failover losing recent writes. Redis Sentinel provides automatic failover for non-cluster deployments, and the Redis docs say a robust Sentinel deployment needs at least three instances. Confirm your replay and retention requirements against the Redis streams documentation before treating Redis as the system of record.
What does Kafka actually cost to run?
The Apache Kafka project is licensed under Apache-2.0 and has no paid edition from the project itself, so the software line item is zero. The recurring costs sit in infrastructure (brokers, storage, network, and controller nodes if you run them separately), in staff time to operate and upgrade the cluster, and in monitoring and security tooling you add around it. Costs rise with retention, replication factor, throughput and the number of topics and connectors. If you buy a commercial support contract or managed service from a third party, that fee is separate and recurring. Ask any vendor for a quote that itemizes hardware or cloud spend, support scope and upgrade labor.
How long does a migration from Redis Streams to Kafka take, and what drives it?
No supplied figure gives a migration duration, so treat any number you hear as an estimate to verify. The main drivers are the number of producers and consumers that write to or read from streams, whether consumers depend on Redis-specific commands, how much history must be replayed or backfilled, and how consumer offsets and delivery semantics are carried over. Teams usually run both systems in parallel, dual-write or mirror events, and cut consumers over one group at a time. Ask for a written inventory of producers, consumers and retention needs before you agree on a schedule.
Can our team run Kafka ourselves?
It can, if you have people comfortable with JVM services, Linux networking, TLS and SASL configuration, and rolling upgrades. Kafka 4.0 runs only in KRaft mode, and brokers, Kafka Connect and tools require Java 17, so the platform skills are different from a single Redis instance. Authorization in KRaft clusters uses ACLs stored in the metadata log, and resources without ACLs are restricted to super users unless allow.everyone.if.no.acl.found is set to true, so access control needs deliberate design. Assess your team against these tasks, and if the gap is real, DeployOpen can plan, implement, harden, document and support a self-hosted cluster inside your own infrastructure under a fixed scope.
Which Redis license applies to the version we run?
It depends on the version. Redis 7.2.x and earlier remain under BSD-3-Clause. Redis 7.4.x through 7.8.x, then called Redis Community Edition, were under the dual RSALv2 or SSPLv1 license, neither of which is OSI-approved. Redis 8.0 and later can be used under a license of your choice among RSALv2, SSPLv1 and GNU AGPLv3, effective from the Redis 8 general availability release. The Redis licenses FAQ says teams moving from 7.2 or earlier with unmodified artifacts see no impact, but teams offering modified Redis code as a network service must make that modified source available to the service's users. Have legal counsel confirm the terms against your exact version and usage.
Does Kafka's delivery model matter more than Redis Streams for regulated workloads?
It matters when the workload needs durable, replayable records with clear retention and access controls. Redis Pub/Sub is at-most-once, so a message is lost if a subscriber cannot handle it, and the Redis docs point to Redis Streams for at-least-once delivery. Kafka is built around a replicated, retained log with consumer offsets, and it authenticates clients with SSL or SASL and encrypts traffic with TLS when configured. Security is optional in Kafka, so you must enable and verify it. Compliance frameworks do not certify a platform. Your auditors will judge the controls you can evidence, so map those controls to your own design before choosing.
What does the work need from our side before an engagement starts?
Expect to supply the event inventory, current throughput and retention targets, identity and certificate sources, network zones, and the people who will own the platform after handover. You will also need to decide the deployment target, whether cloud accounts, data centers or air-gapped networks, and which Kafka release line you will run. DeployOpen engagements are typically fixed-scope, with scope, deliverables and timelines agreed in writing before work starts, and ongoing support, upgrades and testing can follow as a retainer. Ask any provider to confirm which of these responsibilities sit with your team.
How DeployOpen helps choose, implement or migrate
DeployOpen is a Chainscore Labs company that plans, implements, hardens, backs up, upgrades and documents self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For this decision, that can mean scoping whether a Redis stream is enough for your consumers or whether Kafka topics, partitions and consumer groups are needed, then building the chosen platform to your standards. DeployOpen does not favor either platform. Its job is to test the requirement against the facts: whether you need replayable retention beyond what a Redis deployment should hold in memory, how many replicas and Sentinel or broker nodes you will operate, and which Kafka security settings such as SASL mechanisms and StandardAuthorizer ACLs your environment requires.
Most engagements are fixed-scope, so the scope, deliverables and timelines are agreed in writing before any work starts. For a Kafka migration from Redis Streams, that written scope usually covers producer and consumer changes, delivery semantics for at-least-once processing, cluster layout, backup and restore procedures, and the upgrade path to supported releases. Ongoing support, upgrades and continuous testing are available as retainers. DeployOpen does not publish a price list. A quote follows from the written scope, so ask for the scope breakdown along with the quote, and confirm which items are one-time work and which recur under a retainer. Once the shortlist is settled, your own licensing review for Redis, including the choice among RSALv2, SSPLv1 and AGPLv3 for Redis 8 and later, should happen before the scope is signed.
