Ceph RGW quotas for accounts and buckets

Why shared Ceph RGW clusters need quotas per account and bucket

When several teams or customers share one S3 endpoint and one raw capacity pool, a single tenant that writes faster than planned can push every other tenant toward a full cluster. Storage quotas are the control that caps how much each account, user or bucket may consume, so one workload cannot quietly absorb the headroom that others depend on. In the Ceph Object Gateway, quotas are set at more than one level, and the choice of level determines who gets blocked and who is still able to write when a limit is reached.

Teams usually need limits at the account level when a business unit or customer is billed or allocated a fixed share of capacity, at the user level when individual service identities must be contained, and at the bucket level when a single dataset, such as logs or backups, has its own growth budget. Account-wide limits fit multi-tenant IAM designs, while bucket limits are easier to reason about for one data owner. Before choosing, confirm with your own RGW version and account model which quota scopes your release exposes and how they interact, since the sibling page on IAM accounts covers the tenancy model these limits sit on.

This page helps you decide which scopes to enforce, how to read usage statistics across an account, and what your customers see when a limit is hit, including write failures and the sizing questions that come before any number is entered. Treat the limits as a planning artifact: record who owns each boundary, how usage is reported to them, and who approves a change. Quotas do not replace capacity planning at the cluster level, and a quota that exceeds the physical free space still leaves the cluster exposed.

Quota scopes and when each one fits

This table compares the three scopes at which RGW quotas can be applied, what each one caps and who usually owns it. Read each row as a trade-off between a shared pool that is simple to run and per-team control that costs more administration. Confirm the exact quota fields and their behavior for your Ceph release before you design around them.

Quota scope	What it caps	Typical owner	Main trade-off
Account	Total storage and object count across all buckets owned by the account	Platform or tenant administrator	Simple to set for a whole team, but one busy bucket can use up the shared allowance for everyone in the account
User	Usage attributed to a single RGW user within an account or tenant	Account administrator or team lead	Finer control over individual service accounts, but more quotas to track as users are added and removed
Bucket	Storage and object count for one bucket	Application or data owner	Strongest guardrail for a single dataset, but many buckets mean many limits to review and update
Default quota for new users or accounts	Baseline limits applied when an account or user is created, if your release supports defaults	Platform team	Keeps onboarding consistent, but defaults that are too tight generate support tickets and too loose defaults defeat the purpose
Account-wide usage statistics	Reporting on consumed capacity, not a hard cap by itself	Finance, FinOps or capacity planning	Good for chargeback and forecasting, but statistics can lag real usage, so do not treat them as enforcement
Enforcement point	Whether writes are refused at the limit or the limit only triggers alerts	Security and platform teams	Hard refusal protects the cluster, but customers see failed uploads that applications must handle
Shared pool with no per-team quota	Nothing beyond cluster-wide capacity	Cluster operator	Lowest overhead, but no isolation between tenants when one team grows rapidly

Where quotas are set depends on how tenants are modeled

In the account model, each team or customer gets its own IAM space with a root user, IAM users, groups, roles and policies scoped to that account. Quota ownership sits with the account, so administrators can cap total usage for a tenant while the tenant manages its own users and buckets inside that cap. Confirm which Ceph release you run before designing this, because account-level limits and usage statistics depend on the features your release provides.

The account is the quota boundary: set the overall limit on the account, then confirm whether bucket-level limits can still be applied beneath it.
Usage reporting follows the account: check that the statistics you need are reported per account, so you can compare tenants with their limits.
Any user can create buckets: IAM users and roles inside an account may create buckets, so the account cap must hold no matter which identity writes the data.
Existing quotas need an explicit mapping: plan how current limits move to accounts, because you should not assume that old settings carry over.

Before writing any limit into production, test the account structure in a non-production cluster and confirm the quota and statistics output for your release.

How to set and verify RGW quotas on a cluster

This sequence takes a cluster from confirming its Ceph release and tenancy model through setting quotas at the right scope, checking usage statistics and testing limits on a non-production bucket. Each step ends with a check you can record before quotas reach customer-facing accounts.

Confirm the Ceph release and support window
The cluster owner checks which Ceph release is running on the RGW daemons and compares it with the active releases listed on the Ceph releases page. Record the release, its estimated end-of-life date and whether it receives periodic backports and security fixes. Quota behavior and admin commands can differ between releases, so the check closes when the team has written down the exact version and confirmed it is still supported for the planned rollout.
Confirm the tenancy model
Before any limit is set, the platform team decides whether quotas apply per account, per user or per bucket, and whether accounts are created with IAM or with the older tenant-level model. The owner of each customer or team mapping is named in writing. The step is complete when every account or user that will receive a limit maps to exactly one owner, and the team can list those identifiers from the cluster. Ask your DeployOpen or internal contact to confirm the scope against the Object Gateway documentation for your release.
Set quotas at the chosen scope
An administrator applies the quota using the radosgw-admin or Ceph Dashboard path that the Object Gateway documentation lists for your release, selecting the scope confirmed in the previous step. Use a documented command reference rather than copying syntax from forum posts, and record the exact command and the change window. The step is done when the quota settings can be read back from the cluster and match the agreed limits for each account, user or bucket.
Refresh and check usage statistics
The operator triggers or waits for the usage statistics to update, then reads the account-level and bucket-level usage figures for the scoped entities. Compare the reported object count and size against an independent count from a known test dataset. The step closes when the reported usage matches the expected figures within the update behavior documented for your release, and any gap is written down with a plan to investigate it before enforcement is relied upon.
Create a non-production test bucket
The team creates a test account and a bucket in a non-production zone or cluster, using the same release and tenancy layout as production. Set a small quota on the test bucket so that it can be reached quickly with a known number of objects. The step is complete when the bucket, its owner and its quota are recorded in the change log and the test account cannot touch any production data.
Test quota enforcement and customer-facing errors
Upload objects with an S3 client until the limit is reached, then record exactly what the client receives, including the HTTP status and error code. Confirm that writes stop at the limit and that reads and deletes behave as your customer documentation expects. Have the support team review the captured error messages so that they can explain them to customers. The step closes when the observed behavior matches the agreed test plan and the messages are saved for the runbook.
Document the verified quota configuration
The team writes a short runbook covering the release, tenancy scope, command path, usage refresh behavior, test results and the error responses customers see. It also names the owner who changes limits and the process for requesting an increase. The step is complete when the runbook is reviewed by the platform and support leads and stored with the cluster configuration, so that future quota changes can be checked against the same verified steps.

What customers see when a quota is reached

When a write would push a bucket or account past its limit, the client sees a rejected request, not a silent truncation, and the exact status code, error code and message are what your customers' code has to handle. Confirm the precise S3 error response for your Ceph release, because the error name and HTTP status are the contract your customers build against, and the Ceph Object Gateway documentation is the first place to check it. Do not assume an error that a public cloud returns will be identical here.

S3 clients and applications react differently to the same rejection. An SDK may retry a write with backoff, a CLI tool may simply print the error, and a batch job or upload pipeline may keep going and record partial failures. Multipart uploads are a common weak point: a part can be rejected while earlier parts remain stored, so cleanup and retry logic on the customer side matters. Rehearse these cases with the exact SDK versions and tools your customers use, and include the path where a quota is raised mid-job and the upload is resumed.

Usage figures are not always current. Account and bucket statistics can lag behind recent writes, so a customer may be blocked while reported usage still looks under the limit, or the reverse. Before you promise customers a limit behavior, verify how quickly usage updates in your environment by writing known volumes and comparing reported totals, and agree with your operations team how the figures are refreshed. Treat this lag as a planning input for both the quota thresholds you set and the alerts you send customers before they hit a hard stop.

Security and operational risks of enforcing quotas

Quotas only protect a tenant or a budget if the people who set them are controlled, the usage numbers are current, and the limits fit how accounts, users and buckets overlap. The cards below cover the risks that appear once quotas are enforced, and what to confirm in your own Ceph RGW environment before relying on them.

Who can change quotas

Quota changes are made by users with administrative rights on the RGW side, so the question is whether an account owner, a tenant administrator or an automation account can raise its own limit. Confirm which identities can call the quota administration interface, whether account root users can modify quotas that were set by the platform team, and whether those permissions are granted through the same IAM model that governs data access. Test the change path with a non-administrator identity before go-live.

Auditing quota changes

Ceph does not present quota changes as a dedicated audit trail in the way many customers expect, so changes may need to be captured through your own administration workflow, configuration management or RGW logging. Confirm what your team can reconstruct after a quota is raised: who made the change, when, and from which host or automation job. Decide where that record is kept and how long it is retained, because an unexplained increase is hard to investigate after the fact.

Stale usage data

Account and bucket usage figures are produced by RGW statistics that may lag behind recent uploads and deletes, particularly on large buckets. A quota check based on stale numbers can allow writes past the limit or block writes that should be allowed. Confirm how often usage statistics are refreshed in your release, whether a manual resync is needed after bulk deletes or lifecycle expiration, and how the team will reconcile reported usage against the real object count before setting enforcement thresholds.

Overlapping quota scopes

A customer can be constrained at the user level, the account level and the bucket level at once, and the effective limit is the one that is reached first. A bucket quota that looks generous can still be blocked by a tighter account quota, and this surprises application owners during incidents. Map every quota scope that applies to each tenant, document which limit governs which workload, and test writes against each scope so the team can explain the result when a customer reports a rejected upload.

Alerting before limits are hit

A quota that only fails writes gives customers no warning, and the first sign of a limit is often an application error. Alerts should fire at thresholds below the hard limit, and they need a reliable usage signal to trigger on. Confirm which metrics your monitoring stack can scrape from RGW, who receives the alert, and what the on-call procedure is for a tenant that is approaching its limit. Verify the alert with a test tenant before relying on it for customer notices.

Interaction with IAM and multisite

Quota limits interact with IAM policies, so a permitted write can still fail on quota, and a quota can apply differently on each side of a multisite configuration. Confirm how quota settings and usage are propagated in your multisite design, whether a secondary zone enforces the same limits after a failover, and how IAM policy evaluation order affects the error a client receives. Document these behaviors in the same runbook as your failover and access procedures.

How account-wide usage statistics support quota decisions

Account-wide usage statistics answer three questions before you change any limit: how much capacity each tenant really consumes, whether that consumption is growing toward a quota, and which tenants are close enough to their limits that a customer conversation or capacity purchase is due. Use the figures for capacity planning first, because a quota only protects the cluster if the totals behind it are trustworthy. The same figures feed chargeback discussions, but only after you have agreed with finance which measure is billed (stored bytes, object count or a peak value) and how usage is sampled over the billing period. Ask your team to document that choice before the first invoice is produced.

Usage data from the Object Gateway tells you what each account stores; it does not tell you whether the cluster can absorb that growth. Pair it with cluster-level capacity monitoring in your Ceph dashboard or metrics stack, watching raw used space, the fullness ratios that trigger warnings, and recovery headroom after a drive failure. A reasonable check each week is to compare the sum of account usage against raw cluster consumption, since replication or erasure coding multiplies stored data, and to flag any tenant whose trend would reach its quota before the next planned hardware order. Record the date and value of each check so trend lines survive operator turnover.

Before you rely on these statistics for enforcement or billing, confirm three things with the vendor documentation and in a test account: how RGW aggregates usage across all buckets owned by an account, how often the reported figures refresh, and whether the totals include every bucket, including buckets created by a user with lower privileges and buckets in other zones of a multisite setup. Create a known set of test objects in two buckets, wait for a refresh, and compare the reported totals with your own count. If the numbers lag or miss a bucket, treat account-wide reports as an estimate and enforce limits at the bucket level where possible.

Limits to confirm for your Ceph release and environment

Quota behavior in Ceph RGW depends on the release you run, the scopes you need and how your sites and clients interact with it. The cards below list what to confirm, where to confirm it and how to test it in a lab before you commit a tenancy model to production.

Quota fields in your release

Confirm which quota fields your Ceph release exposes for users, accounts and buckets, including size and object count limits and whether each one is enforced at both the user and bucket level. Read the release notes for the version you plan to run, not only the latest documentation, because quota options have changed across releases. Record the exact field names and the commands that set them, so your runbook matches the cluster you deploy.

Scopes that quotas apply to

Check whether quotas apply to the scopes your tenancy model needs: a single user, a bucket, or an account that groups several users. A limit that works on a user may not cover every bucket that user owns, and an account-level limit may not behave the same way on older tenant-level identities. Write down each scope you require and test one example of each in a lab before you rely on it.

Behavior during multisite replication

Confirm how usage counters and quota checks behave when a zone group replicates data between sites. Replicated objects can reach a secondary zone with a delay, so usage reported in one zone may differ briefly from another. Ask how quota enforcement is handled on each zone, and test a write near the limit while replication is running to see which zone accepts it.

Multipart upload accounting

Verify how in-progress multipart uploads count toward a quota, and what happens when an upload crosses the limit partway through. Parts that are uploaded but never completed can consume space that users do not see in listings. Test an upload that exceeds the limit, then abort it, and check whether usage counters return to the expected value and whether leftover parts need cleanup.

Dashboard and command line parity

Check whether the Ceph Dashboard exposes the same quota controls as the radosgw-admin command line for the release you run. Dashboard features can lag behind the command line or cover only part of the options. If your operators depend on the dashboard, make it a documented part of your test plan and confirm which actions still require the command line.

Questions teams ask before setting RGW quotas

What drives the cost of implementing RGW quotas?

The biggest cost driver is how many accounts, users and buckets need limits, and how those limits map to your billing, chargeback or tenancy model. Quota design is mostly planning and testing, so effort rises with the number of tenants, the number of distinct limit tiers and the number of applications that must behave correctly when a limit is reached. Cost also rises if quotas must be enforced alongside IAM accounts, encryption or lifecycle rules, because each interaction needs its own test. Ask for the quote to list the tenant count, the limit model, the usage reporting you need, the test cases for rejected writes and the documentation handed over at the end. Hardware and licensing are not part of this work, because Ceph's source is licensed under LGPL and has no paid edition from the project itself.

How long does a typical RGW quota rollout take?

Duration depends on scope, not on a fixed template. A rollout covering a small number of accounts with one limit model and a simple reporting need is much shorter than one that adds per-bucket limits for many tenants, integrates usage reports into billing and tests client behavior at each limit. The main schedule risks are waiting for decisions about limit tiers, getting access to a non-production cluster that matches production, and agreeing how customers will be notified. Ask for a timeline with named milestones and a clear list of what the client must provide by each one. Any duration in a written scope should be tied to the agreed deliverables.

What does the work need from our team?

You will need people who own the business rules and people who can run the cluster. Business owners must decide limit values, which accounts get which tiers, and what happens at a limit, such as rejected writes and customer notices. Platform engineers must provide a non-production RGW environment, administrative credentials scoped to quota changes, and access to the monitoring data that shows usage. Application owners should identify which S3 clients need retry handling for write failures, because those clients will see errors when limits are reached. Plan for review time from security and compliance staff if quota data is used as evidence for cost controls or tenant isolation.

Can an internal team set up RGW quotas alone?

Yes, a skilled internal team can set RGW quotas if it has Ceph administration experience, access to a test cluster and time to validate behavior. Quota commands are not the hard part. The harder work is verifying that usage statistics are accurate for your workload, that customers receive clear errors, and that changes do not disrupt other tenants. Teams without recent RGW experience often underestimate testing with real S3 SDKs and the process for upgrades. A hybrid model, where DeployOpen plans and tests the design while your team runs the changes, is one option to discuss. Whatever the split, record who owns each limit change after handover.

How do RGW quotas differ from lifecycle rules?

Quotas limit how much storage an account, user or bucket may consume, while lifecycle rules decide which objects expire or move to another storage class based on age or other criteria. A quota does not delete data, and a lifecycle rule does not stop a tenant from filling a bucket. The two work well together: lifecycle rules reduce usage over time, and quotas keep usage within agreed limits between cleanup runs. Quota enforcement and usage counts can lag behind real writes, so do not treat a quota as a hard real-time ceiling without testing it. Confirm the exact enforcement behavior for your Ceph release in the Object Gateway documentation before you write customer-facing terms.

What should we request in a written quote for quota work?

Ask for a written scope that names every account, user and bucket class in scope, the limit model, and the Ceph release and cluster topology it covers. The quote should list deliverables such as the configuration baseline, the usage reporting method, the test plan with S3 client cases, the runbook for changing limits and the handover documentation. It should state assumptions about access, test environments and decisions the client must make, plus what happens if those assumptions change. Ask how the work treats upgrades and whether a support retainer is needed afterward. DeployOpen plans and implements in your environment under a fixed scope agreed in writing, and does not publish a price list, so the quote follows from that written scope.

How does a Ceph upgrade change quota work?

An upgrade can change how quotas are stored, reported or enforced, so quota behavior should be retested on every release you move to. Ceph publishes release notes and an active releases list, and the current maintained releases are Tentacle and Squid, so confirm which one you are on and whether it receives backports and security fixes. Before upgrading, capture current quota settings and usage numbers, run the same S3 client tests on a staging cluster and check the release notes for changes to RGW behavior. Plan a rollback path and schedule usage verification after the upgrade. Include quota regression tests in your upgrade runbook so they are not treated as an afterthought.

How DeployOpen scopes and delivers RGW quota work

DeployOpen plans, implements, documents and supports self-hosted open-source platforms inside your own infrastructure, so RGW quota work runs in your cloud accounts, data centers or air-gapped networks. Engagements are fixed-scope, with scope and deliverables agreed in writing before any change is made to your Ceph cluster.

How we scope

We confirm which accounts, users and buckets need limits, and which teams own each quota decision.
The written scope lists the quota types, default values, exceptions and usage reporting you asked for.
We record the Ceph release and RGW configuration in your environment before agreeing any deliverables.

How we build

We apply account and bucket quotas in a non-production RGW zone first, then verify enforcement against test clients.
We test quota behavior with S3 clients your applications use, including how writes fail once a limit is reached.
We script repeatable quota changes so limit adjustments can be reviewed, tracked and rolled back safely.

How we hand over

You receive documentation covering quota settings, the usage statistics commands and the expected client error responses.
We provide a runbook for raising limits, investigating unexpected usage and recalculating statistics after an incident.
Handover includes a review session with the administrators who will own quota changes after the engagement ends.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer, scoped separately from the build.
Retainer work can cover quota reviews after Ceph upgrades, since release changes can affect how limits and statistics behave.
Confirm with us and your Ceph release notes which quota behaviors change before scheduling any upgrade window.

Ceph RGW quotas for accounts, users and buckets