Rook Ceph RBD storage classes for Kubernetes: what teams need before offering block volumes
A platform team that wants stateful pods, such as databases, to receive block volumes from Ceph without someone provisioning them by hand needs an RBD StorageClass. In short, an RBD StorageClass maps Kubernetes claims to Ceph block pools managed by Rook, and each class carries its own pool, replication, reclaim and expansion behavior. Rook is a CNCF Graduated project, a status it has held since October 7, 2020. This topic covers the storage-side decisions you make before offering block volumes, not the cluster build itself, which belongs to the cluster design and deployment work.
Start by deciding how many classes you need and what each one promises. A separate pool per class is the main lever, because replication size, failure domain and device class are set on the pool, so a database class and a general-purpose class should not share one unless their durability and performance needs match. Write down the replication factor and the CRUSH failure domain each class relies on, and confirm them against the Ceph hardware and capacity guidance for your media before the first claim is made. Ask your storage team to confirm how many failure domains your hosts span, because a replicated pool cannot protect data better than the layout beneath it allows.
Next, set the reclaim policy and expansion behavior deliberately. A Delete reclaim policy removes the RBD image when a claim is deleted, while Retain keeps the data for manual recovery, and the choice matters most for databases where an accidental deletion is costly. Check that allowVolumeExpansion is set on any class you expect to grow, then confirm with your CSI driver version that online expansion works for the filesystem and workloads you run. Finally, decide whether RBD is the right tool at all: databases and workloads that need exclusive block access fit RBD, while many pods that must read and write the same files belong on CephFS, a choice covered on the shared ReadWriteMany page.
Which StorageClass settings change data protection, cost and behavior
This table compares the settings a platform team fixes in each RBD StorageClass, from pool and replication to reclaim and expansion behavior. Read each row as a decision: when the setting fits, what it costs in capacity or performance, and what to confirm against the Rook and Ceph versions you plan to deploy.
| Setting | When it fits | Cost in capacity or performance, and what to confirm |
|---|---|---|
| Target pool (CephBlockPool) | Separate pools for databases, general workloads or a different device class, so each StorageClass maps to hardware that matches its I/O needs. | Each pool carries its own placement groups and capacity planning. Confirm the pool and device class fields supported by your Rook version in the CephBlockPool reference. |
| Replication factor | Replicated pools suit latency-sensitive databases where fast recovery from a single device loss matters more than raw capacity. | Every replica consumes full raw capacity, so usable space is a fraction of raw. Confirm the size and min_size behavior your Ceph release applies during degraded states. |
| Erasure coding | Erasure-coded data pools can suit large, mostly cold volumes where capacity efficiency matters more than write latency. | Lower raw overhead than replication, but RBD on erasure-coded pools needs a replicated pool for metadata and has write and recovery cost trade-offs. Confirm RBD support and overwrite settings for your release. |
| Failure domain | Set the CRUSH failure domain (host, rack or zone) to match how your Kubernetes nodes are physically or logically grouped. | A wider domain protects against more failures but needs enough hosts or racks to place every replica. Too few domains can leave pools unable to reach full redundancy. Confirm the CRUSH rule your cluster uses. |
| Reclaim policy | Retain for volumes holding data you must recover after a PVC is deleted; Delete for disposable or ephemeral data. | Retained volumes keep consuming capacity until someone removes them, so they need an owner and a cleanup process. Confirm how your Rook CSI driver handles deletion. |
| Volume expansion | Enable when stateful applications will grow over time and the CSI driver supports online resize for RBD. | Expansion can only grow a volume, and shrinking requires a new volume and data copy. Confirm allowVolumeExpansion support and whether the filesystem resize happens online for your Kubernetes version. |
| Filesystem type (csi.storage.k8s.io/fstype) | ext4 or xfs for most workloads; choose the type your database vendor certifies or documents. | The choice affects tooling, resize behavior and recovery procedures, not the Ceph replication layer. Confirm the values accepted by your Rook chart and node OS. |
| Image features | Select features such as layering, exclusive-lock and object-map only when your workload or snapshot and clone plan needs them. | Some features add metadata work or limit compatibility with older kernels or clients. Confirm the feature list and kernel requirements for the RBD client on your worker nodes. |
Choosing RBD for databases or CephFS for shared data
Databases and stateful services such as PostgreSQL, message brokers and key-value stores usually have one process that owns the data files and writes to them. RBD presents a Ceph image to the node as a block device, and the file system on top of it is mounted by a single pod at a time. That matches the single-writer I/O these engines expect, and the database manages its own locking, journaling and crash recovery. For this group, RBD is normally the first option to evaluate, and ReadWriteOnce is the access mode to request.
- Write semantics: The engine assumes exclusive control of its data directory, so a block volume with one attached writer keeps the storage model simple.
- Latency sensitivity: Commit-heavy workloads depend on per-write latency, so confirm the pool's media and replication settings with the Ceph team before setting targets.
- Snapshot needs: Point-in-time recovery plans often rely on RBD snapshots and clones. Confirm that your backup tool can quiesce the database before the snapshot is taken.
- Failover behavior: Check how quickly a replacement pod can attach the volume after a node failure, since the old attachment must be released first.
- Expansion: Confirm that the database and file system can grow online, because volume expansion is a separate step from resizing the engine's storage.
Before choosing, confirm the number of writers, the write latency the application needs, and whether snapshots will serve as backups or only as short-term rollback points.
Many stateful workloads run several replicas, and each replica needs its own private storage, such as a StatefulSet with a volume claim template. Here each pod binds its own RBD image through a StorageClass, so volumes are created, attached and reclaimed per replica. The Ceph side isolates each image, which limits the blast radius of a corrupted volume and lets you size each replica independently.
- Per-replica identity: Volumes follow the pod's ordinal name, so plan how a replacement pod reattaches its original volume after a rescheduling event.
- Scaling behavior: Adding replicas creates new images, so check quota and pool capacity before scaling out.
- Reclaim decisions: Decide whether volumes should be deleted or retained when a StatefulSet is scaled down, because the reclaim policy and the workload's data rules determine that outcome.
- Snapshot scope: Snapshots are taken per volume, so a consistent backup across replicas needs an application-level procedure.
- Node placement: RBD volumes are not shared, so pod anti-affinity and zone placement must be planned around where each volume can attach.
Before choosing, confirm how many replicas will run, how each replica is recovered after failure, and whether the application can be backed up consistently across all of its volumes.
Some workloads need many pods to read and write the same files at the same time, such as content directories, shared upload folders, build caches or legacy applications that expect a common path. RBD cannot serve this pattern, because a block image is formatted for one host at a time and concurrent writers from different nodes would corrupt it. ReadWriteMany requires a shared file system, and CephFS is the Ceph option for that access mode through the Rook CSI driver.
- Concurrent writers: Confirm that the application tolerates file-level concurrency, since CephFS coordinates access but does not change how the application handles conflicting writes.
- Metadata load: Workloads with many small files stress the metadata server, so size it for the file count, not only the capacity.
- Quotas and isolation: Plan per-volume quotas and subvolume groups so one team's volume cannot consume shared capacity.
- Snapshot behavior: Confirm how CephFS snapshots and clones map to subvolumes, and whether they meet your backup window.
- Latency expectations: Shared file access adds metadata round trips, so test realistic file operations rather than raw throughput.
Before choosing, confirm the number of concurrent writers, the typical file count and size, and whether the application expects a POSIX path shared across pods.
Setting up RBD StorageClasses in a safe order
This sequence confirms version compatibility, isolates each storage class on its own pool, and proves reclaim and expansion behave as intended before any stateful workload depends on them.
Confirm Ceph and Rook versions are supported together
Before touching the cluster, check the Rook release notes and the Ceph release you intend to run against the supported-version matrix for that Rook release. Confirm the Ceph release is one of the actively maintained releases on the Ceph releases page, and note its end-of-life date so the storage plan survives the next upgrade window. Your engineer records the Ceph release, Rook version and Kubernetes version in the change ticket. Done means the three versions are written down and each pair is confirmed as supported by its project documentation, not assumed from a previous install.
Verify cluster health and capacity headroom
Run the Ceph status command through the Rook toolbox and confirm the overall health is HEALTH_OK, with all monitors in quorum and all OSDs up and in. Check raw capacity against the replication factor you plan to use, because a replicated pool consumes multiples of the data written. Record which OSDs sit in which failure domains, such as host or rack. Done means health is OK with no warnings and you have a capacity figure you can show the owner of the workload.
Create a dedicated RBD pool per storage class
Create one CephBlockPool custom resource for each storage class, so replication, failure domain and device class are set per tier and not shared. Set the failure domain (for example host) and replica count to match how many nodes or racks you can lose. Enable the RBD application on each pool. Done means each pool reports Ready in its custom resource status and appears in the Ceph pool list with the replica size you specified. Confirm placement groups have reached active and clean before continuing.
Deploy the StorageClasses with explicit reclaim and expansion settings
Create one StorageClass per pool that points to the RBD CSI provisioner, the matching pool name, and the CSI secret references. Set reclaimPolicy explicitly to Delete for disposable data or Retain where the volume must outlive its claim, rather than relying on the default. Set allowVolumeExpansion to true if you intend to grow volumes later. Done means kubectl lists each class with the reclaim policy and expansion flag you chose, and the class names are unambiguous to application teams.
Create a test PVC and confirm it binds
Apply a small PersistentVolumeClaim against one storage class and confirm it moves to Bound, with a matching PersistentVolume and an RBD image visible in the target pool. Repeat for each class if you have more than one. Done means the claim is Bound and the image name in Ceph matches the volume handle. If the claim stays Pending, check the CSI provisioner logs and the pool application tag before retrying.
Write, expand and verify data on the test volume
Mount the test volume in a pod, write a known data set and record its checksum. Then increase the claim size, and confirm the PVC status shows the new capacity and the filesystem inside the pod reflects it after the resize completes. Check Ceph health again during and after the resize. Done means the checksum still matches, the claim reports the larger size, and Ceph health remains OK.
Delete the claim and check reclaim behavior
Delete the test workload and then the PVC. For a Delete policy, confirm the PersistentVolume and the backing RBD image are removed from the pool. For a Retain policy, confirm the PersistentVolume moves to Released and the image still exists, then clean it up manually under your change process. Done means the reclaim outcome matches the policy you set in the StorageClass, and Ceph health is still OK with no orphaned images left in the pool.
What reclaim policy and volume expansion do to your data
The reclaim policy on a StorageClass decides what happens to the RBD image behind a PersistentVolume after its PersistentVolumeClaim is removed. With Delete, Kubernetes asks the CSI driver to remove the image, so the data goes with the claim. With Retain, the PersistentVolume is kept in a released state and the underlying image is left in the pool for an operator to reclaim, export or delete by hand. Choose the policy per StorageClass, not per workload: a StorageClass for production databases often warrants Retain, while a scratch class for batch jobs may not. Confirm with your Rook and Ceph CSI versions how each policy behaves in a test cluster, including what happens when the PersistentVolume object itself is deleted, before you rely on either setting for data you cannot regenerate.
A retained volume still consumes pool capacity. Ceph does not know that the claim is gone, so every byte in a released image counts against the replicated or erasure-coded pool that holds it, and the pool's usage rises until someone acts. Plan an owner and a review step for released volumes, and make capacity reporting in your monitoring cover them. Snapshots and clones of a retained image can keep blocks in use even after the active volume is cleaned up, so check those dependencies before deleting anything. Treat the removal of retained images as a change with a record, not as routine cleanup.
Volume expansion is a planned operation. The StorageClass must set allowVolumeExpansion: true, and the expansion is requested by editing the requested size on the PersistentVolumeClaim. The CSI driver grows the RBD image, and for filesystem volumes the filesystem on it is resized as well, which may need the pod to restart or the kubelet to finish the step depending on your versions. Expansion only grows volumes. Shrinking an RBD-backed volume is not a supported path in place, so the reduction work is a migration to a new, smaller volume with a copy of the data. Confirm the expansion and resize behavior for your Kubernetes and Ceph CSI versions in a test environment, and rehearse it on a database clone before you apply it to production.
Security and operational risks for RBD-backed workloads
Block volumes for stateful workloads touch credentials, network traffic, pod permissions and cluster capacity at the same time. The cards below cover the controls and checks that connect those layers, so a single StorageClass does not become the weakest point in the platform.
CephX keys scoped per StorageClass
Each StorageClass should authenticate with a CephX client that can reach only the RBD pools it needs, not an admin key shared across classes. Review the caps on every client when you add a pool, and confirm that a database class cannot read or write the pool used by a development class. Ask your Rook and Ceph contact which capabilities the CSI provisioner and node plugins require for your version, and record who approves changes to those caps.
Secrets holding Ceph credentials
The CSI credentials that Rook creates are stored as Kubernetes Secrets, and Secrets sit unencrypted in etcd by default. Configure encryption at rest for the API server, restrict list and watch on Secrets with least-privilege RBAC, and keep backups of etcd under the same protection. Check which service accounts can read the Secrets in the Rook namespace, because any reader of those Secrets can use the Ceph keys.
Msgr2 secure mode on port 3300
Ceph messenger v2 listens on port 3300, and its secure mode encrypts all traffic after authentication with AES-GCM and adds integrity checks. Decide whether secure mode is required for client and OSD traffic in your environment, and confirm that every Kubernetes node and CSI client can negotiate it. Traffic between nodes and the storage network should be verified with a test client before you enforce the setting across the cluster.
Pod Security Standards for block device access
RBD volumes are mapped as block devices on the node, and workloads that need privileged access for mapping or formatting expand the attack surface. Apply the Pod Security Standards at the namespace level, and limit privileged and host-level settings to the Rook components rather than tenant workloads. Review which application pods request elevated capabilities, and document the exceptions with an owner and a review date.
Pool capacity alerts before full OSDs
A full OSD stops writes for the pools that use it, and every RBD volume on those pools is affected. Set alerts on pool and OSD utilization well before the full thresholds, and assign an owner for each alert. Plan capacity against the failure of a host or a failure domain, because a cluster that is comfortable when healthy can have no room to rebalance after a node loss.
Squid end of life as an upgrade date
Squid (19.2.x) first released on September 26, 2024, and its estimated end of life is October 31, 2026. Put that date on the upgrade calendar for any cluster running Squid, and plan the move to a supported release, such as Tentacle (20.2.x), with a rehearsal in a non-production environment. Confirm the upgrade path with your Rook version before the window, since the operator and Ceph versions are changed in separate steps.
Questions teams ask before building RBD StorageClasses
What drives the cost and effort of an RBD StorageClass design?
The biggest drivers are how many StorageClasses you need, how each one maps to a pool, and how much hardware the replication settings require. Each class usually means a separate RBD pool with its own replication or erasure-coding profile, its own failure domain and its own reclaim and expansion behavior. More classes mean more pools to size, monitor and test. Hardware matters because the Ceph hardware recommendations advise a dedicated drive per OSD, shared SSDs for WAL and DB on HDD-based OSDs, enterprise-class SSDs for metadata and bonded network links across two switches. Memory should be sized for peak use, and the recommendations say to benchmark before buying. Ask any partner to price the design, the hardware review, the test plan and the handover separately, and to state which parts are one-time and which recur through support.
What does our team need to provide before the StorageClass work starts?
You need administrative access to the Kubernetes cluster and the Ceph cluster, or a written plan for how the partner will get the access it needs. Your storage and platform engineers should be available to review pool layouts, failure domains and network designs. Each workload that will consume a StorageClass needs an owner who can describe its I/O pattern, data size, growth and recovery expectations, and who can run or approve a test plan against a representative volume. If the work touches a live cluster, you also need a maintenance window for any Rook or Ceph upgrade, plus rollback steps agreed in advance. Confirm the exact access model, change approval process and window rules with your own change board and with the Rook documentation for the version you run.
How long does an RBD StorageClass engagement take?
Duration depends on scope: the number of StorageClasses, whether the Ceph cluster already exists, how many workloads need test plans, and whether hardware or network changes are part of the work. A single class on an existing, healthy cluster with a short test plan is the smallest case. Adding a new cluster, several classes with different replication profiles, expansion and reclaim testing across workloads, and a maintenance window for upgrades takes considerably longer. No duration is given here because none has been confirmed for this scope. Ask for a timeline in the written scope, with milestones for design sign-off, test completion and cutover, and confirm with your team which dependencies, such as hardware delivery or change approvals, sit on the critical path.
Can an internal team build RBD StorageClasses without a partner?
Yes, a team with Kubernetes and storage experience can build them, but it needs to learn how Rook maps StorageClasses to Ceph pools, CSI provisioning, replication and failure domains, and how volume expansion and reclaim policies behave on RBD. Engineers should also understand how the Ceph cluster is deployed and monitored, because a StorageClass is only as sound as the pool and hardware under it. The main risk is testing: a team that has not run failure, expansion and reclaim tests on realistic workloads may find problems only in production. Check the current Rook and Ceph documentation for your versions, plan a lab environment, and assign at least one owner for ongoing operations before you decide.
What should we ask for in a written quote for this work?
Ask for a written scope that lists each StorageClass, its pool, replication or erasure-coding profile, failure domain, reclaim policy and expansion setting, along with the Kubernetes and Rook versions in use. The quote should state which workloads are covered by test plans, what those tests include, and who runs them. Request the hardware and network review as a separate line, plus the list of client responsibilities such as access, workload owners and maintenance windows. Confirm what documentation and runbooks are delivered, whether knowledge transfer is included, what support or retainer options follow the handover, and how upgrades to later Rook and Ceph releases would be scoped. Any figure in the quote should trace to a line item you can review.
How do RBD StorageClass reclaim policies affect data we need to keep?
The reclaim policy decides what happens to the underlying volume when its PersistentVolumeClaim is deleted. A Delete policy removes the volume and its data, while a Retain policy keeps the volume for manual recovery or cleanup. Choose the policy per workload and document it with the data owner, because a Delete policy on a database volume can remove data earlier than expected if a claim is deleted by mistake. Test this behavior on your own Rook version in a non-production cluster, and confirm how your backup process interacts with it before the class goes into use.
Should databases use RBD or CephFS through StorageClasses?
RBD provides block volumes that a single pod mounts for exclusive read and write access, which suits databases that manage their own file layout and locking. CephFS provides a shared file system that many pods can mount at once, which suits shared data but adds metadata server load and different performance behavior. For a database, confirm the database vendor's supported storage model, test latency and recovery under your expected load, and verify that backups and snapshots meet your recovery needs. Your storage engineers should make the final call on the workload, and the decision should be recorded with the data owner.
Limits to confirm in your environment before you commit
Several choices in an RBD design depend on versions and hardware that only your environment can confirm. The cards below list the checks to run before you commit to a Rook and Ceph combination, a StorageClass layout and a sizing plan.
Kubernetes and Rook version pairing
Confirm which Kubernetes minor releases your chosen Rook and Ceph CSI versions support before you fix either version. As of September 2026, the maintained Kubernetes minor releases are 1.35, 1.36 and 1.37, and the version skew policy requires kube-apiserver to be upgraded one minor version at a time. Check the Rook compatibility matrix for your target release, and plan the Kubernetes and Rook upgrade order together so that CSI drivers never fall outside their supported range.
Need for ReadWriteMany access
Decide whether any target workload needs several pods to mount the same volume at once. RBD volumes are block devices and are normally mounted read-write by a single node, so workloads that share files across pods need a different design, such as CephFS. Inventory each workload's access mode before you create StorageClasses, because changing a shared workload onto block storage later usually means a data migration.
Dedicated drives and enterprise SSDs
Compare your planned hardware with the Ceph recommendations: one dedicated drive per OSD, WAL and DB on shared SSDs for HDD-based OSDs, and enterprise-class SSDs for metadata. Pools that back latency-sensitive databases depend on this layout. Ask your hardware vendor which drive models are enterprise class and whether they can sustain write workloads, and treat any shared or consumer-grade drive as a risk to document.
Memory sized for peak use
Size OSD and monitor memory for peak load, including recovery, rebalancing and scrubbing, rather than for quiet periods. Ceph's hardware guidance says the cluster should be benchmarked before purchase because no two clusters behave alike. Confirm with your hardware team that each node keeps headroom for recovery events, since a node that swaps during recovery can slow every RBD volume it serves.
Snapshot and backup tooling support
Confirm that the snapshot controller, the CSI snapshot classes and your backup product support RBD snapshots in the exact Rook and Ceph versions you plan to run. Check how your backup tool quiesces databases before taking a snapshot, and whether restores into a different namespace or cluster are tested. Record these results against the versions you deploy, because support often differs between releases.
Ceph release within its maintained window
Check that the Ceph release you plan to deploy is still inside its maintained window. Tentacle (20.2.x) is listed with an estimated end of life of June 1, 2027, and Squid (19.2.x) with an estimated end of life of October 31, 2026. A release close to its end of life needs an upgrade plan before go-live, and the upgrade path must also fit the Rook version you choose.
How DeployOpen scopes and delivers RBD storage classes
DeployOpen plans, builds, documents and supports Rook Ceph RBD storage classes inside your own clusters and infrastructure. Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts, and ongoing support is available as a retainer.
How we scope
- Inventory the stateful workloads, their I/O profiles and which need ReadWriteOnce block volumes versus shared file access.
- Define each StorageClass with its pool, replication setting, reclaim policy and expansion behavior in the written scope.
- Confirm the Kubernetes and Rook versions, failure domains and Ceph release your environment will run before build starts.
How we build
- Create the RBD pools and StorageClasses as declarative manifests kept in your version control system.
- Test volume provisioning, attachment, expansion and deletion behavior in a non-production cluster before promotion.
- Validate reclaim policy outcomes with a planned test that confirms data handling when a claim is deleted.
How we hand over
- Deliver a runbook covering StorageClass parameters, reclaim behavior, expansion steps and health checks to run before and after changes.
- Document the decision record for RBD versus CephFS per workload so application teams know which class to request.
- Walk your platform and storage engineers through the handover and confirm ownership of each manifest and pool.
How we support
- Offer ongoing support, Ceph and Rook upgrade planning and continuous testing as retainers, scoped separately from the build.
- Review new workloads against the StorageClass model when teams request storage that falls outside the agreed classes.
- Confirm with your Ceph and Kubernetes vendors or release notes which versions and upgrade paths apply to your environment before each change.
