Resetting Kafka consumer group offsets safely

Resetting Kafka consumer group offsets safely: when a team needs it

A team usually reaches for an offset reset for one of two reasons. A bug in a consumer has been fixed and the group must reprocess records it handled incorrectly, or a poisoned batch is blocking progress and the group must skip past it. In both cases the change happens in Kafka, not in the application code. Each consumer group keeps a committed offset per partition, which is the position the group will resume from after a restart or rebalance. A reset rewrites those committed positions for the group. It does not delete messages, and it does not undo anything the consumer already wrote to downstream systems. That last point decides most of the risk, because a replay re-delivers records that may create duplicate side effects in databases, payment flows or external APIs.

The Apache Kafka project ships the tool that performs this change: kafka-consumer-groups, included in the Kafka distribution. It can show the current committed offsets for a group and change them to a target position. Before you run it, confirm the exact options for your release with the Kafka 4.3 documentation and the version you have deployed, since option names and behavior should be checked against the installed binary rather than remembered. Also confirm whether your retention settings still hold the offsets you intend to rewind to. Offsets that have aged out of retention cannot be replayed, and a reset to an earlier position will quietly start from the earliest record that still exists.

Decide the reason before you touch the group. Write down whether the goal is replay or skip, which topics and partitions are affected, what the consumers do with each record, and which downstream effects must be compensated or deduplicated. This note becomes the record of what changed. Confirm the idempotency of the consumer logic with the application owners first. A reset without that check is the most common way a short incident turns into a data correction project.

Which reset scenario fits the recovery goal

Each reset scenario answers a different recovery question: how much data to reprocess, how much to skip, and where to resume. The choice also determines whether downstream systems will see duplicates, gaps, or both, so pick the scenario before you touch any group.

Replay from earliest

Moves the group to the first offset still retained in each partition, so every retained record is consumed again. Use it when a consumer bug corrupted results and the source topic still holds the data. Check retention first: anything already deleted by time or size limits cannot be replayed, and the group will silently start at the oldest surviving offset. Downstream systems must be idempotent or cleared before the replay, or they will double count.

Jump to latest to skip backlog

Moves the group to the log end offset of each partition, so every record produced before the reset is skipped. This fits a consumer that fell hours behind and only needs current state, such as a cache or dashboard. The risk is a silent gap: records between the old committed offset and the reset point are never processed. Record the gap window and agree with the owning team whether it can be rebuilt from a snapshot.

Move to a timestamp

Resolves the earliest offset at or after a given time in each partition, then resets the group there. It suits incident recovery when you know when a bad deploy started, because you avoid counting offsets by hand. Partitions are independent, so the boundary may differ per partition. Confirm that the timestamp reflects the producer's clock or the broker's append time, since that choice changes which records fall inside the window.

Shift by a relative offset

Moves each partition forward or backward by a fixed number of offsets from its current committed position. It is useful for a small, known rewind, such as reprocessing the last few hundred records after a failed batch. Because partitions have different volumes, one shift value covers different time spans on each partition. Calculate the expected overlap per partition before running it, and confirm that the target is still within retention.

Reset to a specific offset

Sets one exact offset per partition, which gives the most control and the most room for error. Use it when you have already identified the first bad record and the last good one from logs or a dump of the topic. Build the offset list from the data itself, not from memory, and verify that every partition in the plan is covered. A single missing partition produces a gap that is hard to notice later.

Why a dry run comes before --execute

The dry run is the step that shows what a reset would do before anything changes. The kafka-consumer-groups tool prints the planned new offsets for each partition in the scope you select, and it changes nothing unless --execute is added to the same command. A reviewer should treat that output as the change request. For each topic and partition, compare the current committed offset with the proposed offset, confirm the direction is what the incident or backfill calls for (earlier to replay, later to skip), and check that the partition list matches the topics you meant to touch and no others. Record the output with the ticket so the approval refers to exact numbers rather than a verbal plan.

Be clear about what the dry run cannot tell you. It reads broker-side offset state and confirms the arithmetic of the reset, but it says nothing about what the consumer will do once it starts from the new position. A replay can re-send emails, re-post payments to a downstream API or re-write rows in a database that is not idempotent, and a skip can silently drop records that a downstream report depends on. Those side effects live in systems the Kafka tool never sees. Before approving, confirm with the application owner how the consumer handles duplicates and gaps, and whether its sinks are idempotent. Check the Apache Kafka 4.3 operations documentation only for version-specific tool behavior you need to verify in your release, and confirm the flag set against the help output of the exact Kafka version you run.

How to run a reset without losing control of the group

This sequence turns an offset reset into a controlled, recorded change: you capture the group's state, stop its consumers, preview the exact effect, apply it and confirm the committed offsets match the plan.

Record the group state and lag before changing anything
Describe the consumer group with kafka-consumer-groups.sh --describe, and save the output with the timestamp, group ID, topics, partitions, current committed offsets, log end offsets and lag per partition. The person running the reset should keep this file with the change ticket. The step is done when the saved file shows every partition the group consumes and you can explain the lag shown for each one.
Confirm every consumer in the group is stopped
Shut down all consumer instances in the group, including any copies in other environments or batch jobs that share the group ID. Then run the describe command again and check that the group reports no active members. A reset is refused or misleading while members are active, so do not continue until the group is empty, and record who confirmed the shutdown.
Identify the target topics, partitions and offset position
Decide which topics and partitions the reset affects, and whether it moves to the earliest offset, the latest offset, a specific offset, a timestamp or a shift from the current position. Write the chosen scenario and its scope down, and check the partition list against the saved state from the first step. The step is done when the scope is written as an explicit list, not as the whole group by default, and the owner of the data agrees with it.
Run the reset as a dry run
Run kafka-consumer-groups.sh with the reset option and the chosen scope, without the execute flag, so the tool prints the offsets it would commit. Compare each printed new offset with the saved committed offset and the log end offset. The step is done when every partition in the preview matches the intended change, and any surprise, such as an unexpected partition or a target outside the retained range, has been resolved before the next step.
Execute the reset with the same scope
Repeat the same command with the execute flag, using exactly the scope and offset target from the dry run. Save the command, the output and the operator name to the change record. The step is done when the tool reports the new offsets it committed, and that output matches the dry run line for line.
Restart consumers and watch the first processing
Start the consumers and confirm from their logs that they joined the group and fetched from the expected offsets. Watch the first records processed to confirm they are the records you intended to replay or skip. The step is done when logs show the expected starting positions and no unexpected reprocessing or gaps appear in the first batch.
Verify committed offsets and lag against the plan
Run kafka-consumer-groups.sh --describe again and compare committed offsets and lag with the plan and the saved state. Confirm that partitions outside the scope did not move. The step is done when committed offsets match the intended targets, lag falls as expected for the scenario, and the final output, the before and after states and the dry run are filed together in the change record.

Choosing topic, partition and group scope

Resetting all partitions of a topic is the broadest scope within one group. With kafka-consumer-groups, you select it by naming the topic with --topic and leaving out any partition list. Every partition where the group has committed offsets is then moved to the target. Other groups reading the same topic keep their own offsets. This scope suits cases where the whole stream must be replayed or skipped, such as a consumer bug that affected every partition.

Partition count drives the size of the change: a topic with many partitions produces many offset moves in one command, so read the dry-run output line by line.
Uncommitted partitions: partitions where the group has never committed an offset may follow the default reset behavior, so confirm what your target means for them.
Topic-wide flags: check your version's help output for --all-topics and what it selects, because it widens scope beyond a single topic.
Retention limits: a reset to earliest only goes back as far as retained data, so check log start offsets before you choose it.
Group state: the group should be stopped before any reset, so no member commits over your change.

Before you execute, confirm that the dry-run output lists the expected topic, partition count and group ID, and compare it with the group's current committed offsets.

Selecting specific partitions limits the reset to the ones that need it. The tool accepts a topic followed by a colon and a comma-separated partition list, such as --topic orders:0,3. Partitions not in the list keep their committed offsets. This scope fits incidents traced to a subset of partitions, for example when one key range or one broker-side issue affected only some data.

Partition IDs are per topic: partition 3 of one topic has no relation to partition 3 of another, so copy the IDs from the describe output rather than from memory.
Key ordering: ordering is only guaranteed within a partition, so resetting some partitions can replay or skip events that belong to a key but sit in a different partition.
Partition count changes: if the topic has been expanded, confirm which partition IDs existed when the bad data was written.
Range syntax: confirm whether your version accepts ranges in the partition list, and test the syntax with a dry run.
Multiple runs: separate commands for separate partitions are easier to record and roll back than one long list.

Before you execute, check that the partitions in the dry-run output match the list you intended and that no other partition appears in the output.

Targeting a single group with --group isolates the change to the application that needs it. The reset writes new committed offsets only for that group ID. Offsets committed by any other group remain as they were, so a single-group reset is the most contained option when the affected application is clearly identified.

Exact group ID: the group name must match the consumer configuration exactly, so run --describe first to confirm the group exists and shows the topic you expect.
Group ownership: confirm which team owns the group and who must approve a reset, since the group ID alone does not show that.
Active members: a group with live members will not accept the reset cleanly, so stop the application before you run it.
Multiple applications on one group ID: some deployments reuse a group ID across services. A reset then affects all of them, so check the group's members and their client IDs.
Record the group: note the exact group ID in your change record so the scope can be checked later.

Before you execute, confirm that --describe shows the group, its topic and its committed offsets match what you expect to change.

Reset options and when each one fits

This table compares the main kafka-consumer-groups reset choices by what they change, where they point the group and when each one is the right call. Read each row as a decision: pick the target position first, then the scope, then confirm with a dry run before any change is executed. Flag names vary by release, so confirm them against your installed version before you run anything.

Option	What it changes	When it fits
Dry run (no execute flag)	Prints the offsets the reset would set and changes nothing	Every reset, as the first step, so the planned change can be reviewed by a second person
Execute	Commits the new offsets for the group	Only after a dry run matches the intended topics, partitions and target positions
To earliest	Moves committed offsets to the start of the retained log	Replaying a full retention window, such as rebuilding a downstream store from scratch
To latest	Moves committed offsets to the end of the log	Skipping a backlog of data that the group no longer needs, with the skipped range recorded
To a specific offset	Sets each selected partition to one named offset	Resuming from a known point after an incident, when the offset was recorded earlier
To a datetime	Sets offsets to the first record at or after a timestamp	Replaying from a business time, such as the start of an outage, when offsets are not known
Shift by N	Moves the committed offset forward or backward by a fixed count	Small, precise corrections; a wrong sign or count can skip or replay large ranges
Scope: one topic	Limits the change to the topic named in the command	Groups that consume several topics and only one of them needs a replay
Scope: specific partitions or all	Limits the change to chosen partitions or applies to every partition	Partition-level repairs; all-partition resets should be the exception and be justified in the change record
Group selection	Chooses the consumer group ID the reset applies to	Confirm the group is stopped or inactive first; an active group can overwrite the reset

Access control and operational risks to manage

A reset changes where a consumer group reads next, so it touches both access control and the state of the consumers themselves. The cards below cover who should be allowed to run a reset, what the authorizer does when no ACL exists, and the side effects that appear when a group is not quiet while its offsets move.

Admin rights on the group and topic

A reset needs permission to describe and alter the consumer group and to read the topic's offsets. Grant those rights to a named operator role instead of a shared admin account, so every reset traces back to a person. Check which principals hold the group-level permissions today, because a broad grant on a topic can let an application reset its own group without a review.

Default deny without ACLs

In KRaft clusters the StandardAuthorizer stores ACLs in the metadata log, and resources without ACLs are limited to super users unless allow.everyone.if.no.acl.found is set to true. Confirm that setting on every cluster before you rely on it. If it is true, a topic or group that nobody has protected is open to any authenticated client, including one that should never reset offsets.

Separation of duties in production

Ask who proposes a reset, who reviews the dry-run output, and who executes it. In many organizations the person who owns the consuming application should approve the offset target, while a platform operator runs the command with a production-scoped identity. Record that split in the change ticket. Keep the production operator role separate from developer access to staging clusters.

Duplicate side effects

Replaying a range re-runs every side effect the consumer produces, such as payments, emails, inventory updates or writes to an external database. Before you move offsets backward, confirm that the consumer is idempotent for that range or that downstream systems can deduplicate. Illustrative case: a group rewound by a day to fix one bad deploy could resend a full day of notifications unless the handler keys on a message ID.

Active members overwriting offsets

Offsets can only be changed when the group has no active members, so the tool refuses or the change is overwritten when a consumer commits its own position. A running instance that commits after your reset will silently put its old position back. Stop every instance first, confirm the group reports no members, and only then execute. Restart consumers after the reset is verified.

Reset during a rebalance

A group that is rebalancing has members moving partitions between themselves, so the offset you planned for a partition may belong to a different consumer by the time you run the change. Check group state and member assignments immediately before execution, not only during planning. If the group is unstable, wait for it to settle or stop it, then repeat the dry run so the output matches the current state.

Recording what was changed and why

A reset is only defensible if someone can later show what the group's offsets were before the change, what they became, who approved it and why. Each change record should name the consumer group, the topics and partitions in scope, the committed offsets before and after the reset, the timestamp of execution, the operator who ran it, the approver and the business reason. Capture the before state with kafka-consumer-groups.sh --describe immediately before the reset, and capture the after state once the group has resumed. Record the exact reset selector used (for example --topic with --partitions, or --all-topics), because the scope of the change is the scope of the selector, not the scope you intended. If the reset used --to-datetime, --by-duration or --to-offset, record the resolved offsets, since a time-based reset can land on different offsets than the wall-clock time suggests.

Keep the dry run output with the record. The output of the command without --execute is the only preview of what the tool would have committed, so store it verbatim alongside the lag snapshot taken just before the change. The lag snapshot shows how much data the group was behind, which is what a replay will reprocess or a skip will discard. Store the file names, checksums or ticket links that tie these artifacts together. Confirm with your own change process how long these records must be retained and where they are held, since that depends on your internal policy and any framework that covers the data your consumers process.

Review the record after the group catches up. Compare the lag snapshot with the lag at the point the group returned to steady state, confirm that downstream outputs match what the reset was meant to produce, and note any duplicate or skipped work the consumer or its sinks reported. Close the record only when the reviewer has signed off on those points. Reviews catch a common gap: a replay that succeeded technically but reprocessed records that were not idempotent at the sink. Add the lessons to the runbook so the next operator starts from the same checks.

Limits to confirm in your environment

A reset looks like one command, but its outcome depends on the Kafka release you run, what retention has already removed, how the topic is cleaned, and whether the group is still consuming. Confirm each of these limits in your own environment before you plan the change.

Release-specific behavior

Reset tooling and its output can change between Kafka releases, so confirm how kafka-consumer-groups.sh behaves on the exact version your brokers and admin hosts run. The supported releases and their upgrade paths are listed on the downloads page, and the release notes are the place to check for changes to reset options. Run the dry run on the same version you will use to execute the reset.

Retention may have removed target offsets

Replaying from an earlier offset only works if those records still exist. Retention by time or size deletes old segments, so a requested offset can fall before the earliest available record. Before you reset to a past timestamp or offset, check the log start offset of each partition and confirm how far back the data goes. Records already deleted cannot be recovered by the reset itself.

Compacted topics keep only latest values

On a compacted topic, the log keeps the latest value for each key rather than every event. Replaying from an early offset therefore delivers a reduced history, and some intermediate states will not come back. Confirm the cleanup policy of each topic in scope and decide whether the consumer logic can rebuild state from the retained records.

Consumer protocol differences

Groups on the classic protocol and groups on the newer consumer group protocol differ in how membership and assignment are managed, so the reset steps and the way members react can differ. Identify which protocol each group uses before you plan the stop and reset sequence. Test the procedure on a non-production group that uses the same protocol.

Whether the group is still active

Offsets can only be reset when the group has no active members, and a consumer that restarts or is still running can commit offsets over your reset. Describe the group and confirm it shows no active members before you execute. Stop every instance, including those in other deployments or batch jobs, and check again just before the change.

Scope and change record

Confirm which topics and partitions the reset will touch, and whether the scope matches what the consumer expects. Keep the dry-run output, the exact command, the group state before and after, and the approver. Without this record, a later investigation cannot tell a planned replay from an accidental skip.

Questions teams ask before resetting offsets

What drives the cost of a consumer group offset reset?

The cost is driven mostly by how many groups, topics and downstream systems the reset touches, not by the reset command itself. Apache Kafka is open source under the Apache License 2.0, so there is no license fee for the reset tooling. The spending falls on people and planning: an engineer who understands each consumer's processing logic, a window agreed with the application owners, the time to stop and restart consumers, and the work to check results. Costs rise when consumers write to external stores that cannot easily be corrected, when many groups must move together, or when no one has a record of the current offsets. Ask any partner for a quote built from a written scope that lists the groups, topics, environments and deliverables involved.

How long does a typical offset reset take?

The duration depends on the scope and on the downstream work, so no single figure applies. A single group on one topic with idempotent consumers can often be planned and executed quickly, while a reset that replays data into databases, warehouses or payment systems needs time for each downstream owner to confirm their side. Plan for four stages: recording current offsets, a dry run, the stopped-group reset, and verification after the consumers restart. Ask for the timeline your partner expects for each stage and which steps depend on other teams' availability.

What does the team need to provide for the work?

The team needs to supply the people and facts that only it can know. These usually include:

The consumer group IDs, topics and partitions in scope, plus the business reason for the reset
Owners of each consumer application who can confirm processing logic and idempotency
Access to a broker account with the permissions the reset needs, which should be set through Kafka ACLs
A change window and a rollback plan agreed with operations and the downstream owners

The team also needs a place to store the before and after offset records, such as a ticket or change log.

Can our own engineers reset offsets without outside help?

Yes, a team with Kafka operations experience can usually do this on its own, because the reset is a documented command-line operation with a dry run option. The risk lies in judgment rather than tooling: knowing whether a consumer is idempotent, whether a stopped group has truly stopped, and which partitions must not move. Teams new to Kafka, or resets that affect payments, regulated records or many groups, benefit from a second reviewer. Any reset should be recorded and reviewed by someone other than the operator.

How do we handle duplicate messages downstream after a replay?

Replaying from an earlier offset will redeliver records that consumers already processed, so downstream systems must be able to absorb them. Kafka does not undo side effects that a consumer has already made. The usual approaches are to make the consumer idempotent with a stable business key, to use upserts instead of inserts, or to deduplicate in the target store using the topic, partition and offset as a reference. For systems that cannot deduplicate, plan a compensating step or a clean-up query before the replay, and test it on a copy of the data. Confirm each consumer's behavior with its owner before the reset window opens.

When should we bring in outside help for a Kafka offset reset?

Bring in outside help when the reset affects regulated or financial data, spans many consumer groups, or touches systems where a duplicate or skipped record has real consequences and no one on the team can confirm the consumer logic. Outside help also suits teams that need the reset documented as evidence for an assessment. DeployOpen plans and executes self-hosted Kafka work inside the client's own infrastructure, and most engagements are fixed-scope with deliverables and timelines agreed in writing before work starts. DeployOpen does not issue audit reports or certifications, so independent auditors remain responsible for any assessment.

Is a dry run enough to prove the reset is safe?

A dry run shows the offsets a reset would set, but it does not prove that consumers will handle the replayed records correctly. Use the dry run to confirm the group, topic and partition scope and the target offsets. Then verify behavior with the consumer owners, check downstream state after a restart, and compare lag and output against what was expected. Keep the dry run output with the change record.

What should we keep as evidence of an offset change?

Keep a written record of the group ID, topic, partitions, the committed offsets before the change, the target offsets, the command used, the operator, the approver and the time window. Add the dry run output, the consumer owner's confirmation and the post-restart lag check. Store the record where it can be retrieved for audits and incident reviews. Your compliance team should confirm which evidence its framework expects, since requirements differ by regulation.

How DeployOpen scopes and delivers offset resets

DeployOpen treats an offset reset as a change to production data flow, not a single command. We agree the scope in writing, rehearse the reset against a non-production copy of the topology, and hand over a record of what changed. The work runs inside your own infrastructure, with your team holding the approvals.

How we scope

Written scope names the consumer groups, topics and partitions in the reset, and which groups stay untouched.
We confirm the replay or skip intent, the target offsets or timestamps, and the downstream effects on each consumer.
Engagements are fixed-scope, so scope, deliverables and timelines are agreed before any reset work begins.

How we build

We write the stop, dry run, execute and restart sequence for your group, with named owners at each step.
Each dry run output is reviewed with your team before anyone runs the command with --execute.
We test the procedure on a copy of your topology and record how the group behaves after it restarts.

How we hand over

You receive a runbook covering the reset scenario, the scope, the commands used and the rollback path.
A change record lists each group, topic and partition touched, with before and after offsets and the operator.
We walk your operators through the procedure so the next reset can be run and reviewed without us.

How we support

Ongoing support, upgrades and continuous testing are available as retainers for your Kafka platform.
Support can cover reviewing a planned reset, updating the runbook after Kafka upgrades, and checking consumer lag afterwards.
Retainer scope is agreed in writing, so response expectations and included work are clear before they are needed.