Why shared Kafka clusters need client quotas
Client quotas limit how much traffic each client can push onto a shared Kafka cluster, so one team's runaway producer or consumer cannot starve the others. When several teams share one KRaft cluster, every client competes for the same broker network, request handler threads and disk I/O. Quotas set per-client limits on produce and consume byte rates and on the share of broker request time a client may use, and they can be applied as cluster-wide defaults or as overrides for a specific user or client ID. Without them, the cluster has no built-in way to tell a well-behaved application from one that is flooding it.
You need quotas once more than one team, or more than one application with its own release cycle, shares a cluster and the clusters' headroom is not guaranteed to cover a sudden burst. Typical triggers are a batch backfill that reads an entire topic, a misconfigured producer retry loop, or a new consumer that starts from the earliest offset. Before rollout, confirm how your clients identify themselves, because quotas are keyed on the authenticated user principal, client ID, or both, and that choice determines who gets throttled. Review the Kafka security overview so authentication is in place first, since unauthenticated clients make per-tenant limits hard to enforce.
Quotas do not protect against everything a tenant can consume. They do not cap broker disk usage, the number of partitions a team creates, or the retention of a topic, so a team can still fill storage or overload controller metadata within its byte limits. Those risks need topic-level controls such as creation policies, retention limits and storage monitoring, which other sections of this guide cover. Quotas also do not replace authorization: the authorization and ACL documentation explains that resources without ACLs are restricted to super users by default, so access control and quotas must be designed together. Ask your team to confirm the quota behavior for your exact Kafka release before you depend on it.
Which quota types Kafka enforces
Kafka gives a multi-tenant operator four distinct ways to limit a client: byte rates in each direction, a share of broker request time and a connection limit. Each one protects a different shared resource, so the right choice depends on which resource a tenant could exhaust first.
Produce byte rate
This quota limits how many bytes per second a client can write to the brokers, measured per client ID, per user or both. It protects disk, replication traffic and the network bandwidth of every other tenant on the same brokers. Check whether your tenants produce in bursts, because an average rate can hide spikes that still saturate a broker. Confirm in the 4.3 documentation how the broker slows a producer that exceeds the rate, since the client may see added latency before any error.
Consume byte rate
This quota limits how many bytes per second a client can read from the brokers, measured on the same client and user dimensions as produce quotas. It protects broker network capacity and page cache from a consumer that replays a large backlog or runs with a high fetch size. Consumers that start from the earliest offset are the usual source of sudden read load. Confirm in the 4.3 documentation how throttled fetches affect consumer lag and whether any rate applies to replication fetches from other brokers.
Request percentage
This quota limits the share of broker request handling time a client can use, expressed as a percentage of the network and I/O thread capacity. It catches clients that send many small or expensive requests, such as frequent metadata calls or heavy admin operations, that stay under byte limits but still consume broker CPU. Set it with the byte quotas rather than alone, since a client can hit either limit first. Confirm in the 4.3 documentation which request types count toward this measurement.
Connection limits
Connection limits control how many connections a client or source address can open and how quickly new connections can be created. They protect brokers from connection storms caused by misconfigured clients, reconnect loops or a service that scales out without limit. In a shared environment, check whether limits apply per listener, per IP address or per user, because a gateway or NAT address can group many tenants behind one source. Confirm the exact setting names and default behavior in the 4.3 documentation before you rely on them.
Quota scopes and when each one fits
This table compares the quota scopes Kafka can apply, from broad defaults to combined user and client rules, and matches each one to a tenant model. Read each row as a trade-off between tighter control of individual workloads and the number of quota entries your team must create, review and explain. Confirm scope precedence in the official Kafka documentation before relying on overlapping rules.
| Quota scope | What it limits | Best fit for the tenant model | Trade-off to weigh |
|---|---|---|---|
| Default quotas | Every principal or client that has no more specific override | Shared baseline for many small teams where no tenant needs special treatment | Protects the cluster broadly but cannot separate a noisy team from a quiet one |
| Per-user quota | Traffic from one authenticated principal across all of its client connections | One principal per team, such as a service account owned by a single team | Simple to map to ownership, but all services sharing the principal share one budget |
| Per-client-ID quota | Traffic from connections that declare the same client.id string | Teams that run several named applications under one shared principal | client.id is set by the client, so it is a labeling aid rather than a strong security boundary |
| Combined user and client scope | Traffic for one principal and one client.id together | Many services per team that each need their own limit under a team identity | Most granular option, but every service pair needs an entry and more review effort |
| Connection creation rate and count limits | How quickly new connections are opened and how many a principal or IP can hold | Multi-tenant clusters where connection storms from one tenant affect others | Limits must be sized against real connection pooling behavior, which you should measure first |
| Request percentage quota | Share of broker request handler time a principal or client may use | Tenants whose cost is driven by many small requests rather than large payloads | Harder to predict from application throughput, so validate with load tests |
Setting defaults first and overrides for exceptions
Start with a conservative default quota that applies to every principal, then grant explicit overrides only to the high-volume tenants that have been reviewed. The default is the control that matters most on a shared cluster, because it covers the service accounts, scripts and forgotten applications that nobody reviewed when they were created. Without a default, a new principal starts with no byte-rate or request-percentage ceiling and can consume whatever the brokers have left. In practice, set a low default produce and consume byte rate and request percentage for all users, and treat each override as a documented exception with an owner, a business reason and a review date. Confirm the exact default and override keys for your Kafka version in the quota documentation before you script them, since the names and entity types differ between user, client-id and combined scopes.
Quotas and authorization fail in different ways, so design them together. Under the StandardAuthorizer in KRaft mode, ACLs live in the cluster metadata log, and any resource without an ACL is restricted to super users unless allow.everyone.if.no.acl.found is set to true. Leaving that setting at its default means a new topic or group that nobody has granted access to is unusable by ordinary tenants, which is the safer posture. Setting it to true opens every unlisted resource to every authenticated principal, and that would undercut both the ACL model and the quota plan. Pair each quota principal with a matching set of ACLs, so a tenant that is throttled and a tenant that is authorized are the same identity you can audit together. Review the authorization and ACL documentation with your security team before you decide how the setting should be configured.
How to roll out client quotas on a running cluster
This sequence moves a shared Kafka cluster from no client limits to enforced per-tenant quotas without cutting off producers or consumers that already depend on current throughput. Each step ends with a check so the team knows when it is safe to tighten the next control.
Inventory principals and client IDs
The platform team lists every authenticated principal and every client ID that connects to the cluster, and maps each one to the team that owns it. Start from broker authentication logs and listener configuration, then confirm the list with each team lead. Quotas are applied to the principal (the authenticated identity) and optionally to the client ID, so this inventory decides what can be limited separately. Done when every active principal has a named owner and any shared service accounts are flagged for a decision. Confirm the authentication setup in your own environment, because the mechanisms in use (SASL PLAIN, SCRAM-SHA-256 or SCRAM-SHA-512, GSSAPI, OAUTHBEARER, or mutual TLS) determine how principal names are formed.
Tie authentication identities to quota subjects
The team confirms that each tenant connects with its own identity, so that a quota keyed to a principal really limits one tenant. With SASL, the principal is the SCRAM user or Kerberos principal. With SSL, it is derived from the client certificate subject through the broker's principal mapping rules. Shared credentials mean that one noisy team can consume another team's allowance, so these must be split before any per-tenant override is meaningful. Done when a test client authenticated as each tenant appears under its own principal in broker logs or metrics. Verify the mapping rules and mechanism choice with your security team, since they are part of your access model.
Measure current produce and consume rates
The operations team records byte rates and request rates per principal over a representative business period, using the broker's client metrics and the existing monitoring stack. Capture peaks, not only averages, and include batch jobs, replays and consumer catch-up traffic that run on schedules. The output is a table of observed normal and peak produce and fetch rates per principal and client ID. This table is the only basis for default and override values, so it should be reviewed by each owning team. Done when every principal in the inventory has a measured baseline and the owning team has agreed that the baseline reflects expected load. Confirm the exact metric names and retention in your monitoring system, since they depend on your metrics pipeline.
Set defaults in an observed state
The platform team sets default quotas for produce and consume byte rates, request percentage and connection limits at a level above the measured peaks, so that no current tenant is throttled on the first day. In this phase the goal is visibility, not enforcement: the team watches throttle metrics per principal and records any client that would have been slowed. If your Kafka version and configuration allow an observation-only period for a given limit, use it and confirm that behavior in your environment. Done when the defaults are in place, the throttle metrics are visible per principal on the dashboard, and the team has a list of principals that would have been throttled, which should be empty or fully explained.
Apply per-tenant overrides
For tenants whose measured peaks exceed the default, or who have a documented need for more, the platform team sets per-user or per-client overrides that match the baseline plus agreed headroom. Overrides are written as explicit entries, so each one has an owner and a reason that can be audited later. Keep the list short: a long override list usually means the defaults are wrong. Done when every override in the configuration corresponds to an entry in the inventory with an owner, and a configuration review confirms no principal is left on a default that conflicts with its measured rate. Confirm the procedure for changing quotas in your environment, including who may approve a change and where the change is recorded.
Check throttle metrics before tightening
Before any limit is lowered, the operations team reviews throttle time and throttled request metrics for each principal over a full business cycle. A principal that is throttled during normal operation indicates either a wrong baseline or a client that needs investigation, and the team resolves that before continuing. The check also confirms that alerts fire for sustained throttling and that the owning teams receive them. Done when the throttled-client list is empty for normal traffic, alerts have been tested, and each owning team has acknowledged its own throttle signals. Do not skip this gate: tightening without it is the most common cause of unexpected producer latency after a quota change.
Enforce quotas and monitor throttled clients
The platform team moves the agreed limits from observed to enforced, one tenant group at a time where possible, and watches latency, error rates and throttle metrics for each change. Enforcement is reversible, so the rollback steps for each quota change are written down before the change is made. After enforcement, throttle monitoring becomes a standing control: the platform team reviews throttled principals on a regular schedule and feeds the results into capacity planning. Done when enforced quotas are in place for all principals, rollback steps are documented, and the monitoring review has a named owner. Verify the enforcement behavior and the rollback procedure in a non-production cluster that matches your production configuration before you change production.
Monitoring and troubleshooting throttled clients
How do I find which clients are being throttled on a Kafka cluster?
Start with the broker-side quota metrics, which record throttling per client ID, user or both, and then map the throttled principal back to an owning team. Kafka's quota documentation describes throttle time metrics for produce, fetch and request quotas, so the first step is to confirm the exact metric names and tags in the 4.3 metrics reference for your build. Then query for the principals with the highest sustained throttle time over a window. Keep the mapping from principal to team current, because a throttled user name means little to an on-call engineer without an owner. Use the Kafka documentation as the reference for metric names, and confirm them in your own monitoring stack before you write any dashboard or alert.
Does throttling make Kafka clients fail with errors?
By default, no. When a broker throttles a client, it delays its responses rather than returning an error, so the client experiences slower produce acknowledgements and slower fetches. Producers may see rising request latency and buffer growth, and consumers may see lag increase while the cluster looks healthy. This is why throttling often goes unnoticed until a team reports that a pipeline is behind. Test this behavior in a staging environment with your client library versions, since application timeouts and retry settings decide whether a delay turns into a failure. Confirm the exact client-side behavior for your versions in the Kafka documentation before you set timeouts.
How should I alert on sustained throttling without paging on short bursts?
Alert on the duration and size of throttling over a window, not on a single sample. A short spike during a batch job is usually expected, while throttling that persists for many minutes for the same principal suggests a quota that is too low or a client that has changed behavior. A practical design has three parts:
- A warning when a principal is throttled above a low threshold for a long window, routed to the owning team's channel.
- A page only when throttling coincides with user-facing lag or failed deliveries, which requires correlating with consumer lag metrics.
- A weekly report of principals that were throttled at all, so quota limits can be reviewed before they cause incidents.
Set the thresholds from your own baseline, since no universal value is safe to copy.
How do I tell a quota problem from broker saturation?
Compare the throttling signal with broker resource signals. A quota problem shows throttling concentrated on one or a few principals while other clients on the same brokers stay at normal latency. Broker saturation shows high request queue times, high network or disk utilization and latency growth across many principals, often with throttling present but not isolated. If you see both, fix the saturation first, because raising or lowering one tenant's quota will not change a cluster that is out of capacity. Check request handler idle ratio, network processor metrics and disk metrics against your own baselines, and confirm metric names in the 4.3 documentation.
How do I tell a quota problem from consumer lag?
Consumer lag is a symptom, and throttling is one possible cause, so look at the fetch side before changing quotas. If the consumer group's lag grows while fetch throttle time is high for that consumer's principal, the quota is a likely contributor. If lag grows with no throttling, check the consumer itself: processing time, rebalances, partition count and downstream systems. A consumer that is slow because its database is slow will not be fixed by a higher fetch quota. Record both signals on the same dashboard keyed by principal so the comparison takes minutes, not an investigation.
What should a throttling incident runbook include?
A useful runbook starts with identification: which principal, which quota type (produce, fetch or request), and since when. The next steps are to confirm the owning team, check whether the workload changed recently, and compare with broker saturation signals. The runbook should list the options in order of impact: ask the owner to reduce or spread the load, temporarily adjust that principal's quota through a documented change, or add capacity. Every quota change should be recorded with a reason and an expiry, and reviewed afterward. Your team should write the runbook against your own alert names and ownership model, and test it in a drill before an incident.
Who should own quota monitoring in a shared Kafka platform?
The platform team usually owns the metrics, dashboards, alert rules and quota changes, while each tenant team owns its clients and the response to throttling affecting its workloads. This split works only if the platform team can map every principal to an owner and both sides agree on response times. Decide who can approve a quota increase, who is notified on a sustained throttle, and how exceptions expire. For a self-run platform, the cost is mostly internal staff time for building and maintaining these processes. If you want an outside team to design the monitoring and runbooks, ask for the scope in writing, including which dashboards, alerts and drills are delivered and which remain your responsibility.
Can DeployOpen set up quota monitoring for our Kafka cluster?
DeployOpen plans, implements, documents and supports self-hosted open-source platforms inside your own infrastructure, so monitoring and runbooks for a Kafka cluster you run can be part of a fixed-scope engagement. A written scope would state the metrics collected, the dashboards and alert rules delivered, the runbook content and any drills, and what your team must provide, such as access to the metrics stack and an owner list. Ongoing tuning and review can be covered by a retainer. Ask for a quote based on that written scope, since the effort depends on how many tenants you run and how much monitoring you already have.
Security and operational risks of quota design
A quota is only as strong as the identity it is attached to and the change process that controls it. The cards below cover the identity, change control, lifecycle and resource gaps that most often let one tenant affect another despite a configured limit.
Quotas on shared identities
Kafka applies quotas to a client identity, which is the principal, or to a client ID when no principal is set. If several teams share one service account or connect without authentication, every quota lands on the same bucket and one team can exhaust another's allowance. Check that each tenant has its own principal and that unauthenticated listeners are either closed or excluded from tenant traffic. Confirm how your clients authenticate before writing any quota.
Choosing the authentication basis
SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER, GSSAPI (Kerberos) and mTLS certificate identities all produce principals that quotas can target. Pick the mechanism your identity provider can issue per tenant and rotate without outages, and map each tenant to one principal. Whatever you choose, document how a principal name maps to a team so quota reports stay meaningful.
Unrecorded quota changes
Quota changes are cluster configuration. Without a change record, a raised limit or a removed override can appear as an unexplained latency incident weeks later. Restrict which operators or automation identities can alter quota entities, using ACLs on the cluster and your change tooling, and require a ticket or pull request that names the tenant, the old value and the new value. Confirm who can currently run the admin commands.
Overrides that outlive tenants
Per-user overrides are easy to add during an incident and easy to forget. When a team is retired or its service account is deleted, the override may stay in place and continue to shape limits for a principal that no longer exists or, worse, for a name later reused by another team. Schedule a periodic review that compares quota entities with the active principal inventory and removes entries with no owner.
Quotas that ignore storage growth
Byte-rate and request-percentage quotas limit throughput and broker CPU time, not the disk a tenant consumes. A team can stay within its produce rate and still fill shared volumes through long retention, many partitions or large compacted topics. Pair quotas with per-prefix storage monitoring, partition limits and retention policies, and confirm that alerts fire before a broker volume reaches capacity.
Partition and connection exhaustion
Quotas do not bound the number of partitions a tenant creates or the metadata load those partitions add to the controller. Connection limits and request quotas help, but a tenant opening many connections or creating thousands of partitions can still degrade the cluster. Set topic creation policies and connection limits together, and test the effect of each limit under a realistic client count before enabling it in production.
Questions teams ask before setting Kafka client quotas
What does it cost in effort to set Kafka client quotas?
Most of the effort goes into discovery and agreement, not into the quota commands themselves. Setting a quota on a broker is a small configuration change, but deciding which principals get which limits takes inventory work across every team that produces or consumes. The main cost components are the platform engineer's time to measure traffic, the time owners spend agreeing limits, and the testing needed to confirm that throttling does not break legitimate batch jobs. Quotas are mostly one-time setup with recurring review as tenants join, leave or change workload. Ask a partner to price discovery, baseline measurement, rollout, and an ongoing review cadence as separate lines in a written scope.
How long does a client quota rollout take?
Duration depends mostly on how many tenants you have and how long you measure before setting limits. A small cluster with a few well-known applications can move quickly, while a shared platform with dozens of teams needs time to collect traffic data across normal and peak periods, such as month-end batch runs or a seasonal load. Each tenant also needs an owner who signs off on the proposed limit. Ask for the measurement window in the scope, the number of tenants included, and the rollout sequence, for example monitor-only first, then soft limits, then enforcement. Treat any duration in a proposal as dependent on those inputs, not as a fixed figure.
What does our platform team need to supply before quotas can be designed?
Three inputs drive the design: a principal inventory, named owners, and traffic baselines. The principal inventory lists every Kafka user, service account or client identity that connects, because quotas apply to those principals. Owners are the people who can say whether a limit will hurt a business process and who approves changes. Baselines are produced byte rates and request rates per principal, ideally over a representative period that includes peaks. Without these, limits are guesses, and they tend to be set too low for bursty workloads or too high to matter. Confirm early whether your identity model maps each application to a distinct principal, since shared credentials make per-tenant quotas hard to enforce.
Can our own team set up Kafka client quotas without a partner?
Yes, if your team already runs the cluster and has authentication and authorization in place. Quota configuration uses the same administrative tooling as other broker settings, and the Kafka documentation covers the mechanics. The harder parts are organizational: keeping an accurate principal inventory, getting owners to engage, and reviewing limits as workloads change. Teams that lack Kafka operations experience often find that the first rollout reveals hidden clients, such as connectors or monitoring agents, that were throttled unexpectedly. A partner can add value by running the measurement period, documenting the limits and building the review process, but the decision on acceptable limits stays with your owners.
Which Kafka versions and security settings affect quota design?
The current supported releases listed on the Kafka downloads page are 4.3.1, 4.2.2 and 4.1.2, and quota behavior should be confirmed against the documentation for the release you run. Quotas depend on authenticated principals, so the security model matters: Kafka supports SSL and several SASL mechanisms, including SCRAM and OAUTHBEARER, and those choices determine how client identities are named. In KRaft clusters, ACLs live in the metadata log, and resources without ACLs are restricted to super users unless the allow.everyone.if.no.acl.found setting is enabled. Confirm which mechanism you use and whether unauthenticated listeners exist, because unidentified clients can fall into default buckets that are hard to attribute.
What should a quote for Kafka client quotas include?
A useful quote lists scope and exclusions in writing before work starts. Ask for the tenants and principals in scope, the cluster and listeners covered, the measurement method and period, the proposed quota types (produce and consume byte rates, request percentage, and connection limits where supported), the approval process for each limit, and the monitoring and alerting for throttled clients. Also ask what is handed over: configuration as code, runbooks, a dashboard for throttling, and a documented review cadence. Clarify what is excluded, such as application changes to fix throttled clients, and whether ongoing support is a separate retainer. DeployOpen plans and implements in your environment and provides fixed-scope quotes based on written scope.
How do we know quotas are working and not hurting tenants?
Monitor throttling before and after enforcement, and treat throttled clients as a signal to investigate rather than a success metric. Track per-principal byte rates and request rates against their limits, watch for sustained throttling on consumers, and check producer latency on tenants that approach their caps. Verify that defaults and overrides resolve as your design intends, since an unexpected default can throttle a principal nobody remembered. Ask your partner or team to define alert thresholds, a process for requesting limit increases, and a test that confirms a sample tenant behaves as expected under the new settings.
How DeployOpen scopes and supports Kafka quota work
DeployOpen handles Kafka quota work as a fixed-scope engagement, with the scope, deliverables and timeline agreed in writing before any change is made to your cluster. For this topic, the written scope normally names the quota design (which principals or client IDs are covered, the produce and consume byte-rate limits, request percentage limits, and any connection limits), the configuration method, the environments involved, and the verification steps that show each tenant is throttled as intended. Quotas touch every tenant on the platform, so agreeing the tenant list and the default values in writing early is the step that prevents rework later.
Configuration happens in your own cluster, in your own infrastructure, and is applied through the tooling your team already uses or has chosen, such as the Kafka admin tooling, your infrastructure-as-code pipeline or an operator. Verification covers test runs against throttled and unthrottled principals, checks on the quota metrics exposed by the brokers, and a review of how throttled clients appear in monitoring. The handover package is written for the people who will own the platform afterward: the quota design record, the list of overrides with their owners, the change procedure for adding a tenant, and the runbook for investigating a throttled client. Ask for a sample of each document before signing so you can judge whether your operations team can use it.
DeployOpen does not publish prices. A quote follows from the written scope, so the inputs that move it are the number of tenants and override cases, whether your team wants configuration codified in a pipeline, how many environments must be verified, and what evidence your governance or audit function expects. Ongoing tuning as tenant workloads change is available as an optional retainer, which is priced separately from the initial engagement and is scoped in writing in the same way.
