What S3 buckets for Kubernetes apps with Rook ObjectBucketClaims give your team
Application teams that need S3 buckets can request them from Kubernetes with an ObjectBucketClaim instead of filing a ticket with storage administrators. The Ceph Object Gateway (RGW) provides the S3-compatible API that the bucket is served through, and Rook runs that gateway and its bucket provisioning as Kubernetes resources. A claim fits best when many small, short-lived or per-application buckets are needed and the platform team wants one repeatable request path with a consistent naming and credential model. A hand-built bucket, created directly with Ceph tooling or an S3 client, still fits when the bucket is long-lived, shared across unrelated teams, or needs settings that the claim workflow does not expose in your environment.
Rook is a CNCF Graduated project, a status it has held since October 7, 2020, and this page assumes you already have a working Ceph cluster and a Rook deployment with an object store running. It does not cover building either of those. Before you offer claims to teams, confirm which Rook and Ceph releases your environment runs, because the bucket provisioner and its resource fields depend on the Rook version. Check the exact API versions and fields against the Rook documentation for that release rather than copying examples from older guides.
The decisions that matter most are who can create claims, which namespaces receive the generated Secret and ConfigMap, and who owns the bucket when the claim is deleted. Those choices determine your access model, your backup scope and your cleanup process, so settle them with the security and platform owners before the first team places a request. Plan for credential rotation as well, since the generated access keys are only as contained as the namespace that holds them.
Which claim and bucket options fit each team
This table compares the configuration choices a platform team sets for ObjectBucketClaims: the bucket StorageClass, new versus existing buckets, reclaim policy and object store users. Read each row to see when an option fits, what changes for the application team, and what to confirm in the Rook release you run, since exact parameter keys vary by version.
| Option | Fits when | What changes for the application team | Confirm in your Rook release |
|---|---|---|---|
| Bucket StorageClass per object store | One Ceph Object Gateway serves several teams or environments and each needs its own default settings | Developers pick a class by name in the claim, so the class name becomes the contract they rely on | Exact StorageClass parameter keys and the object store reference field in the Rook object bucket provisioner documentation |
| Single shared StorageClass | A small platform with one object store and one trust boundary | Claims are simpler to write, but every namespace inherits the same store and defaults | Whether the provisioner needs a separate class for each namespace or store endpoint |
| New bucket claim | The application owns its data and the bucket should be created with the claim | Application gets a bucket name generated from the claim and credentials in a Secret, with the ConfigMap holding the endpoint | How bucket names are generated and whether a custom name is accepted in your release |
| Claim pointing at an existing bucket | Data already exists, or a team takes over a bucket created outside Kubernetes | Application reads the same Secret and ConfigMap shape, but deleting the claim must not delete data the team still needs | How the provisioner handles existing buckets and whether it adopts or refuses them |
| Reclaim policy set to retain | Bucket contents are business records or the claim may be deleted by mistake | Deleting the claim leaves the bucket and objects in place for manual cleanup | The reclaim policy values supported by your provisioner version and who owns cleanup |
| Reclaim policy set to delete | Buckets are disposable, such as caches, build artifacts or test data | Deleting the claim removes the bucket, so backups and retention must be proven first | Whether deletion also removes objects and how it behaves on non-empty buckets |
| Object store user per application | Each application needs its own access keys and an auditable identity | Credentials are scoped to the application's bucket rather than shared across teams | How the user is created, its capability or policy settings, and the key rotation procedure |
| Shared object store user | Several workloads read one dataset and access is already governed at the bucket level | Fewer credentials to rotate, but a leaked key exposes every bucket the user can reach | Whether bucket policies can narrow access and how the user is referenced from claims |
New bucket or existing bucket: which path fits your application
When a claim asks for a bucket that does not exist yet, the Rook operator provisions the bucket and its access credentials through the object store and the bucket StorageClass. The platform team owns the StorageClass, the object store it points to and the reclaim behavior. The application team owns the contents of the bucket and the claim that requests it. Because the bucket is created for this application, its lifecycle can be tied to the claim, so deleting the claim can also remove the bucket, depending on the reclaim policy in use.
- What the application receives: a Secret with access keys and a ConfigMap with the bucket name, host and port, which the workload reads at startup.
- Who controls deletion: the reclaim policy on the StorageClass decides whether a deleted claim leaves the bucket in place or removes it.
- Migrations: there is nothing to migrate, but any data the application later needs to load should follow a documented import step, not manual copies into the bucket.
- Access scope: confirm whether each claim gets its own object store user or shares one, since that affects audit trails and credential rotation.
Illustrative example: a new analytics service requests a bucket for exported reports, and the platform team confirms the reclaim policy keeps the bucket when the claim is removed. Before you approve the design, confirm the reclaim policy, credential scope and who signs off on deletion of production data.
A claim can also bind to a bucket that already holds objects, such as a bucket that an earlier application wrote to or a bucket populated by a data pipeline. In this path, the bucket lifecycle usually belongs to whoever owns the data, not to the claim. The claim becomes a way to hand the application credentials and a stable name, so it must be scoped carefully to avoid granting access to more data than the application needs. Ownership of existing content, retention obligations and any access reviews stay with the data owner.
- Ownership of content: the data owner decides who may read, write and delete objects, and the claim should not change that decision silently.
- Credential scope: confirm whether the object store user can see other buckets or only the one being bound.
- Migrations: existing data may use different key naming, lifecycle rules or content types than the application expects, so test reads against a copy first.
- Reclaim risk: deleting a claim on a bound bucket should not remove live data, so confirm the reclaim behavior before creating the binding.
Questions to ask before binding: who owns the data, what retention or legal hold applies, which applications already read or write it, and whether a snapshot or copy exists for rollback. Confirm each answer with the data owner in writing before the claim is created.
Binding a claim to live data changes who can reach that data from Kubernetes, so the decision should be recorded before the claim is applied. The questions below help the platform, security and data teams agree on scope, rollback and evidence. They do not replace the object store documentation, which describes the exact user, policy and bucket settings your Rook and Ceph versions support.
- Ownership: name the team that approves access and the team that can change or delete objects.
- Access: list the users and policies that will touch the bucket after binding, and remove any that are not required.
- Rollback: confirm how the application is returned to its previous bucket or credentials if the binding fails.
- Retention: check whether retention rules, lifecycle expiration or legal holds apply to existing objects.
- Audit: decide where access logs for the bound bucket will be kept and who reviews them.
Ask your storage and security teams to confirm each point against your own environment and the version of Rook and Ceph you run before you approve a live binding.
How object store users and generated credentials reach the application
When a claim is bound, the Rook operator produces two objects in the claim's namespace: a Secret holding credentials and a ConfigMap holding connection details, and the application reads both. In general terms, the ConfigMap carries the endpoint and bucket name, while the Secret carries an access key pair for an object store user. Your team should confirm the exact keys in each object against the Rook documentation before writing any deployment manifest that references them, because the workload's environment variables or mounted files depend on those names. The practical decision is who the credentials belong to, and that choice shapes everything after it.
A shared object store user is one account that several claims or applications reuse. It is simpler to create and reason about, but every workload holding its keys can reach every bucket that user owns, and rotating the key means touching every consumer at once. Per-claim users give each bucket its own identity, so access is scoped to one bucket and a leaked key exposes one dataset. The cost is more users to inventory, more Secrets to track, and more rotation events to schedule. Confirm with Rook how a per-claim user is named, how its permissions are set, and whether the operator rotates keys or only creates them, since the answer determines your rotation runbook.
Generated Secrets are credentials, not configuration, so they need the same protection as any other. Kubernetes stores Secrets unencrypted in etcd by default, so confirm that encryption at rest is enabled for your cluster and restrict get, list and watch on Secrets through least-privilege RBAC, since anyone who can read the claim's namespace may be able to read the key pair. Also check that the application copies the Secret into its own runtime only where needed, and that logs and debug output never print it. Review the Kubernetes Secrets good practices page before deciding whether an external secret store is worth the added moving parts.
Setting up the bucket StorageClass and your first S3 bucket claim
This sequence takes a cluster from a running Ceph Object Gateway to a test ObjectBucketClaim that produces working S3 credentials, so application teams can request buckets without touching Ceph directly.
Confirm the object gateway is running and reachable
The platform team checks that the Ceph cluster is healthy and that the Ceph Object Gateway (RGW) service is running, then confirms that pods in the Kubernetes cluster can reach its endpoint on the network path you plan to use. Run a basic HTTP request from a test pod to the gateway endpoint and confirm it returns a response. Confirm the endpoint's TLS certificate is trusted by the cluster if you serve HTTPS. Record the endpoint URL, because the StorageClass depends on it. The step is done when the endpoint answers from inside the cluster, not only from an administrator workstation.
Create the object store as the platform team
A platform engineer creates a CephObjectStore resource in the Rook operator's namespace, defining the metadata and data pools, the gateway instances and the endpoint exposed to workloads. Pool replication and failure domains should follow the design decisions made for the cluster, not defaults chosen during this step. Check the resource status with kubectl until Rook reports the object store as ready, and confirm the gateway pods are running. This step is done when the object store reports a ready phase and its endpoint matches the one you tested in the previous step.
Create an object store user for provisioning
The platform team creates a CephObjectStoreUser that Rook will use to create buckets on behalf of claims. Keep this user within the platform namespace and restrict who can read its generated Secret, since its credentials can create buckets across the store. Confirm the user resource reports a ready state and that Rook has generated its access key Secret. This step is done when the user is ready and its credentials exist only where the platform team can manage them.
Define the bucket StorageClass
Create a StorageClass with the Rook bucket provisioner, setting its parameters to reference the object store and the namespace where it runs, and set the reclaim policy you have agreed for buckets. Use the reclaim policy decision recorded for this environment, and confirm with your team whether buckets should be retained or deleted when a claim is removed. Apply the StorageClass and confirm it appears in the list of storage classes with the expected provisioner. This step is done when the StorageClass is visible to the cluster and points at the correct object store.
Create a test claim in a non-production namespace
In a separate, non-production namespace, an application owner or the platform team creates an ObjectBucketClaim that references the StorageClass and requests a bucket name. Use an illustrative name such as a test bucket for a sample application. Watch the claim with kubectl until its phase reports bound, since a claim that stays pending usually points to the StorageClass, the user permissions or the gateway endpoint. This step is done when the claim reports a bound state and the bucket appears in the Ceph object store.
Verify the generated Secret and ConfigMap
Inspect the Secret and ConfigMap that Rook created in the claim's namespace, using the names the claim reports. The Secret should contain the access key and secret key, and the ConfigMap should contain the bucket host, port and name that applications need. Confirm the values match the object store endpoint and the bucket that was created. Avoid printing the secret values into shared logs or terminal histories. This step is done when both objects exist with the expected keys and the endpoint values are correct.
Test an S3 request against the new bucket
From a pod in the test namespace, mount or load the Secret and ConfigMap values and use an S3-compatible client to list the bucket and write and read a small test object. A successful listing and round trip shows that the credentials, endpoint and bucket policy all work together. Remove the test object and, if the test claim is no longer needed, delete it and observe what happens to the bucket under the reclaim policy you chose. This step is done when the S3 request succeeds from inside the cluster and the cleanup behaves as the reclaim policy predicts.
Securing S3 credentials and bucket access on Kubernetes
An ObjectBucketClaim hands an application a bucket, a Secret with access keys and a ConfigMap with the endpoint. Each of those outputs is a credential or an access path, so the controls below apply to the claim, the Secret it produces, the object store user behind it and the pods that mount it.
Encrypt Secrets at rest in etcd
The Secret that a claim generates holds the S3 access key and secret key. Kubernetes stores Secrets unencrypted in etcd by default, so anyone who can read an etcd snapshot or backup can read those keys. Confirm that encryption at rest is enabled for Secrets on your clusters, that etcd backups are protected as carefully as the live cluster, and that the same applies to every cluster that consumes the bucket.
Restrict list and watch on Secrets
Reading a Secret is not the only risk. Any principal with list or watch permission on Secrets in a namespace can retrieve every generated credential there in one request. Grant these verbs only to the controllers and operators that need them, keep human access to read-only where possible, and review RoleBindings and ClusterRoleBindings that touch the namespaces holding claims.
Control who creates claims
Creating an ObjectBucketClaim is a request for storage and, through the generated Secret, for credentials. Decide which teams may create claims, in which namespaces, and whether a platform team reviews the bucket name, quota and StorageClass before approval. Limit claim creation with RBAC on the objectbucketclaims resource so that self-service does not become unbounded bucket sprawl.
Scope object store users to minimum buckets
Object store users created for claims or applications should only reach the buckets they need. Review the bucket policy and capabilities granted to each user, avoid sharing one user across unrelated applications, and check that a user created for one claim cannot list or write to other buckets on the same Ceph Object Gateway. Confirm the exact capability model in the Rook and Ceph version you run.
Rotate and revoke keys deliberately
Credentials from a claim are long-lived until someone changes them. Plan how a key is rotated without breaking the application, usually by issuing a new key, updating the Secret, rolling the pods that mount it and then removing the old key. Agree in advance how a key is revoked after a suspected leak or when an application is decommissioned, and who is responsible for each step.
Apply Pod Security Standards to consumers
Pods that mount the generated Secret can expose it through logs, environment dumps or a compromised container. Apply the Pod Security Standards at the namespace level, at least the baseline profile and the restricted profile where the workload allows it. Prefer mounting Secrets as files with tight permissions, and confirm that the application does not print its configuration at startup.
What happens to bucket data when a claim is deleted
The reclaim policy decides whether deleting an ObjectBucketClaim also removes the bucket and the objects inside it, or whether the bucket survives for later use. Treat this as a data-loss decision, not a cleanup setting. With the policy set to delete, a removed claim can take an application's stored data with it, and an application that still reads or writes that bucket will fail without warning. With the policy set to retain, the bucket and its objects remain, but the generated Secret and ConfigMap are removed, so the workload loses its credentials and endpoint details. Confirm the exact behavior of the reclaim policy, the handling of the generated credentials, and what happens to a bucket that the claim adopted rather than created against the Rook release you run, because the controller logic has changed across releases.
Existing buckets need separate care. When a claim points at a bucket that already holds data, deleting the claim should never be the only thing standing between that data and removal, so review how the bucket was created before any policy is applied. Object store users have their own lifecycle: a user tied to a claim may be removed with it, and a user shared across several claims or applications can break unrelated consumers. Map every claim to its bucket, its owner and its users before you change the policy on a live cluster, and check that each consuming workload tolerates the credentials being rotated or revoked.
Before a reclaim policy goes to production, confirm three things in writing. First, that object versioning, lifecycle rules or a replication target protect the data you cannot regenerate, and that a restore has been tested against the bucket, not assumed from the backup tool's status. Second, that your GitOps or change process prevents accidental claim deletion, for example by protecting namespaces and reviewing pruning settings. Third, that a documented runbook exists for a deleted claim, including who can recreate the claim, which bucket name it must bind to, and how applications reload their Secrets. Ask your Rook support channel or the Rook documentation for the policy behavior in your version, and test each step in a non-production cluster first.
Limits to confirm before S3 buckets for Kubernetes go into production
Bucket claims look simple from the application side, but they depend on several moving parts: the Rook and Ceph versions you run, how the Object Gateway is reached from pods, how namespaces and quotas are governed, and how claims behave during upgrades. Confirm each item below with the Rook documentation and with tests on your own cluster before application teams depend on the bucket endpoint.
Supported Rook and Ceph combinations
Confirm which Rook operator releases support which Ceph releases in your target version pair before you pin either one. Ceph's active releases are Tentacle (20.2.x) and Squid (19.2.x), and each has its own end-of-life date, so a Rook release that works with one may not be the right match for the other. Record the chosen pair in your change records, and test bucket creation, credential rotation and deletion against that exact pair in a non-production cluster.
Object Gateway exposure to pods
Decide how pods reach the Ceph Object Gateway: through a cluster-internal service, an ingress or a dedicated load balancer, and whether the endpoint is reachable from every namespace that needs it. Confirm whether traffic is spread across multiple RGW instances and what happens when one gateway pod restarts. Check TLS termination, certificate trust inside application containers and the DNS name that the generated ConfigMap will carry.
Namespace and quota controls for claims
An ObjectBucketClaim is a namespaced object, but the bucket it creates lives in the shared object store, so a team with permission to create claims can consume shared capacity. Confirm how RBAC restricts who can create claims in each namespace, whether storage quotas exist for buckets or users, and who approves new StorageClasses. Test what a claim in one namespace can see of another team's bucket credentials, and confirm that deleting a claim does not remove data you still need.
Claim behavior during operator upgrades
Rook upgrades change the operator, CRDs and the Ceph daemons that serve buckets, so confirm how existing claims, generated Secrets and user credentials are handled during each step. Check that an upgrade does not regenerate or invalidate access keys that applications already hold, and that the bucket endpoint stays available while gateway pods roll. Rehearse the upgrade on a copy of your configuration and record which resources must be verified afterward.
Gaps against native S3 services
Ceph's Object Gateway offers S3-compatible and Swift-compatible APIs with features such as lifecycle, compression and multisite, but application teams may still expect behavior from a managed cloud service. Ask each team which S3 operations, event notifications, object lock or versioning settings, and IAM policy features their code uses, and test each one against your gateway version. Also confirm that the claim workflow does not expose settings that the application expects, such as per-bucket quotas or automatic region placement.
How much effort and time setting up S3 buckets for Kubernetes takes
What drives the effort of setting up S3 buckets for Kubernetes apps?
The effort depends on how many object stores and namespaces are involved, how much existing data must be moved or kept, how much security review the design needs, and how much upgrade planning sits around it. A single object store with one namespace and a few new buckets is a small, bounded job. Several object stores, multiple teams with separate claim rights, existing buckets that already hold data, and a requirement to review access policies and encryption each add design, testing and documentation work. Upgrade planning matters because the Rook operator and the Ceph release both change over time, and buckets created today must keep working through those steps. Ask any partner to show how each of these factors changes the work in the written scope.
What does the client team need to provide before setup starts?
The work needs four things from your side. First, access to the Kubernetes cluster and the namespaces where claims will run, with enough rights to create StorageClasses, claims and test workloads. Second, access to Ceph administrators who can confirm the object store, its gateway endpoint, the pools behind it and the user policies that the bucket provisioner will use. Third, application owners who can state which buckets they need, their naming, retention and access expectations, and which existing data must be preserved. Fourth, a test window in which claims, Secrets and ConfigMaps can be created and removed without affecting production traffic. Confirm the exact access model and the change approval steps with your own change board before the schedule is fixed.
Can our own platform team set up Rook ObjectBucketClaims without outside help?
Often yes, if the team already runs Kubernetes and has someone who administers the Ceph object store. The required skills are a working knowledge of Kubernetes StorageClasses, Secrets and ConfigMaps, plus enough Ceph knowledge to create the object store user and confirm the gateway endpoint. Teams with little Ceph or Rook experience usually need more time for those steps and benefit from a partner review of the design. A partner is most useful for multi-store designs, existing data migration, security review and writing runbooks that the next person on call can follow. Estimate your own team's capacity honestly, list the skills you lack, and compare that list with the scope you need.
How is a fixed-scope engagement agreed before work starts?
Most DeployOpen engagements are fixed-scope. Scope, deliverables and timelines are agreed in writing before any work starts, so the quote follows from a defined piece of work rather than an open-ended retainer. The written scope should name the object stores in scope, the namespaces and applications, which existing buckets are included, the security and access review, the test plan, the handover documents and the change steps that the client approves. Agree on what is out of scope, such as application code changes or work on other Ceph pools, so both sides know where the boundary sits. Ask for the written scope to be signed before the schedule is confirmed.
What should a written quote for this work contain?
A written quote should list each deliverable, the assumptions it depends on and the client inputs it needs. Ask for the object stores, namespaces and bucket classes covered, the number of applications and claims to be tested, and whether existing buckets will be adopted or left in place. The quote should show the phases, such as design, build, verification and handover, and say which items are one-time work and which are recurring. It should also name who performs each step, what the client must supply and by when, and what happens if the scope changes. DeployOpen does not publish a price list, so the quote follows from this written scope.
What recurring work remains after the buckets are handed over?
Once the setup is handed over, recurring work continues in three areas. Access and credentials need periodic review, including the object store users behind each claim, the Secrets that applications read and rotation of those credentials. The Rook operator and Ceph version need upgrades, and each upgrade should be followed by a check that existing claims still provision and that applications can still reach their buckets. Capacity and lifecycle rules also need watching, because bucket growth affects the underlying pools. Retainers are available for ongoing support, upgrades and continuous testing if your team prefers to hand those tasks to a partner. Ask any provider which of these tasks the handover covers and which move to a retainer.
Which parts of the effort are one-time and which recur?
The one-time effort covers the design of the bucket StorageClass, the first object store users, the claim templates, the verification of new and existing buckets and the handover documents. Recurring effort covers credential rotation, operator and Ceph upgrades, claim reviews when new teams onboard and checks after cluster changes. Upgrades can be a large recurring item, so ask how the design will be re-tested for each Rook and Ceph step. Ask the provider to separate these two groups in the quote so you can budget for the recurring part.
What should we confirm about the Ceph and Kubernetes versions before scoping?
Confirm the Ceph release in use and whether it is still actively maintained, because the Ceph releases page lists which releases receive backports and security fixes. Confirm the Kubernetes minor version in the cluster and whether it is still within the project's support window, which Kubernetes lists on its releases page. Confirm the Rook operator version and the Helm or manifest method used to deploy it. These versions determine which upgrade steps appear in scope and whether the design needs changes before work starts. Write the versions into the scope so that the quote and the test plan refer to the same environment.
How DeployOpen scopes, delivers and supports S3 buckets for Kubernetes
DeployOpen plans, builds and hands over self-service S3 bucket provisioning for Kubernetes applications on Rook-managed Ceph, inside infrastructure you control. Each engagement starts with a written scope that fixes which bucket classes, claim patterns, users and reclaim behaviors are in place before any manifest is applied. Engagements are fixed-scope, and ongoing support is available as a retainer.
How we scope
- Inventory which applications need buckets, their access patterns, and whether they expect new or existing buckets.
- Define the bucket StorageClass, its object store, and the reclaim policy for each class of data.
- Agree in writing who creates object store users and how credentials are rotated and revoked.
How we build
- Configure the ObjectBucketClaim StorageClass and test claims that produce the Secret and ConfigMap as specified.
- Model new-bucket and existing-bucket claims in a non-production cluster before any production use.
- Test that consuming pods read connection details from the generated Secret and ConfigMap, with least-privilege access.
How we hand over
- Provide manifests, claim templates and a runbook covering naming, reclaim behavior and deletion steps.
- Document the object store user lifecycle, including where credentials are stored and how they are rotated.
- Walk your platform and application teams through requesting a bucket and checking its health.
How we support
- Offer a retainer for upgrades of the Rook operator and Ceph object gateway against the agreed scope.
- Review bucket quotas, unused claims and reclaim settings on a schedule agreed in the retainer.
- Confirm with your team which support hours, response paths and escalation contacts apply before work begins.
