How Rook Ceph CSI snapshots and clones fit your Kubernetes storage
A platform team needs point-in-time copies or fast copies of stateful volumes on Rook Ceph when it wants to capture a database before a schema change, give a test environment realistic data, or roll back a failed application update. In Kubernetes, the VolumeSnapshot object asks a CSI driver to capture a PersistentVolumeClaim at one moment. A VolumeSnapshotClass names the CSI driver and its parameters, much as a StorageClass does for provisioning, and sets the deletion policy for the resulting snapshot content. To create a new volume from existing data, you create a PersistentVolumeClaim whose dataSource or dataSourceRef points to either a VolumeSnapshot (clone-from-snapshot) or another PersistentVolumeClaim in the same namespace (clone-from-PVC). Both paths are handled by the Rook Ceph CSI drivers for RBD and CephFS, but the mechanics differ between the two, so confirm the snapshot and clone support for each StorageClass in your Rook release before you depend on it.
Snapshots are not a complete backup strategy. A VolumeSnapshot normally lives in the same Ceph cluster as the source volume, so it does not protect against loss of that cluster, accidental deletion of the snapshot itself, or corruption that the snapshot faithfully captured. Application consistency is also your responsibility: the CSI snapshot freezes the storage, not the application's in-memory state, so databases usually need a flush or quiesce step first. Decide early which copies count as recovery points, where an off-cluster copy will live, and who restores from it. Tools such as Velero can take CSI snapshots of persistent volumes and move the data to object storage, but that is a separate design decision covered elsewhere on this page. Before you commit to a schedule, confirm with your Rook and Ceph versions how snapshot retention, deletion and restore are tested in your own environment.
How RBD and CephFS snapshots and clones differ
RBD volumes back ReadWriteOnce block devices, and they are the usual choice for databases and virtual machine disks. A CSI snapshot of an RBD-backed PersistentVolumeClaim maps to an RBD snapshot of the underlying image, and a volume created from that snapshot is a copy-on-write clone. The clone shares unchanged blocks with its parent, so the new volume is available quickly, and writes only consume space for the blocks that change. The snapshot is point-in-time at the block layer, not at the application layer.
- Image-level capture: the snapshot records the block device state at one moment, including whatever a database has written to disk but not yet flushed.
- Copy-on-write clones: a clone depends on its parent snapshot, so the snapshot cannot be deleted while dependent clones exist.
- Single-writer use: a clone is normally attached to one consumer at a time, which suits restore testing and dev copies.
- Crash-consistent by default: without coordination, the restored data looks like a sudden power loss, which databases usually recover from but application-level backups should not rely on.
- Quiesce first: flush or freeze the database (for example, a backup mode or a write lock) before the snapshot, then release it immediately after.
Confirm with your storage team how your Rook release exposes snapshot and clone classes, and whether your application's own backup tooling is needed alongside them.
CephFS volumes back ReadWriteMany file systems that many pods mount at once. Here the CSI snapshot maps to a CephFS subvolume snapshot, and a clone maps to a subvolume clone created from that snapshot. The snapshot captures the directory tree of the subvolume at a point in time, but the metadata and data paths remain shared, so the consistency question is about every writer that touches the tree.
- Subvolume snapshots: the snapshot covers one subvolume, so structure your volumes so that data that must be captured together lives in the same subvolume.
- Subvolume clones: a clone is an independent copy created by the cluster, so it does not depend on the parent after the copy completes. Check the clone status before using it, because large trees take time to copy.
- Many writers: applications writing to the same files at the moment of capture can leave a snapshot with half-written records or files that disagree with each other.
- Quiesce by writer: pause or drain the writers that matter, or have each application flush its own state, before the snapshot is requested.
- Quotas and limits: snapshot and clone size counts toward the volume's quota planning, so confirm the quota model with your storage team before you schedule frequent snapshots.
Ask your storage team which CephFS subvolume group and snapshot settings your Rook deployment uses, and test a clone of a live shared volume before relying on it for recovery.
Both backends take snapshots without knowing what an application has in memory, so application consistency is a shared responsibility between the platform and the workload owner. Kubernetes CSI snapshots provide a point-in-time capture of the storage, and they do not coordinate transactions, caches or open files inside your software. The gap is the most common reason a restored volume looks intact on the surface but fails when the application reads it.
- Transactions in flight: a snapshot taken mid-transaction may hold partial writes, so databases should be checkpointed or placed in a backup mode first.
- Write caches: application-level caches and buffered writes live outside the snapshot unless they are flushed.
- Freeze and thaw hooks: a pre-snapshot step that quiesces the application and a post-snapshot step that resumes it keep the window short.
- Multi-volume applications: when data and logs sit on separate volumes, their snapshots are not atomic with each other unless the platform or application coordinates them.
- Restore testing: a snapshot only proves the storage captured something, so restore a clone and run the application's own integrity checks.
Ask the application owner which quiesce steps the software supports, and document them next to the snapshot schedule so the two stay in step.
The backend you choose sets how snapshots and clones behave over their lifecycle, including how they are deleted, how many you can keep and how they fit into backup retention. Snapshots are not backups on their own, because they usually live in the same Ceph cluster as the source data. The tabs above cover what a snapshot holds, and this view covers what it means for your recovery plan.
- Same-cluster risk: a snapshot inside the cluster does not protect against loss of the cluster, the pool or an operator error that removes both source and snapshot.
- Dependency chains: RBD clones depend on parent snapshots, so deletion order matters for restore chains.
- Snapshot counts and churn: frequent snapshots increase metadata load, so confirm practical limits with your storage team and your Rook version.
- Off-cluster copies: when you need copies outside the cluster, consider a backup tool that exports data, and confirm its limits separately.
- Restore objectives: write down how far back each workload must recover, then match the snapshot schedule to that target.
Confirm the limits, deletion order and off-cluster copy approach with your storage team before you write a retention policy that auditors will read.
Snapshot and clone configuration options and when each fits
This table compares the settings that control Rook Ceph CSI snapshots and clones, and separates fields set on the VolumeSnapshotClass from fields inherited from the StorageClass. Read each row as a decision: confirm the field names and defaults against the Rook and Ceph CSI versions you run before you rely on them. The main takeaway is that pool and filesystem choices live with the StorageClass, so a snapshot class change will not move data to a different pool.
| Option | What it controls | When it fits | What to confirm |
|---|---|---|---|
| Driver and class name | Which Rook Ceph CSI driver handles snapshot calls for RBD or CephFS | One VolumeSnapshotClass per block or file backend so each snapshot uses the matching driver | Driver name matches the installed Rook operator and CSI release |
| deletionPolicy Delete | Removes the backing RBD or CephFS snapshot when the VolumeSnapshotContent is deleted | Short-lived snapshots, CI copies and test data where cleanup should be automatic | Behavior when a snapshot is deleted by mistake in your version |
| deletionPolicy Retain | Keeps the backing Ceph snapshot after the Kubernetes object is removed | Recovery points that must outlive a namespace cleanup, paired with an owner who prunes them | Who runs the cleanup job and how orphaned snapshots are found |
| Pool or filesystem parameters | Target RBD pool or CephFS filesystem for the volume and its snapshots | Set once in the StorageClass; choose it when you create the volume, not when you snapshot it | Parameter keys in the CSI StorageClass for your Rook release |
| Retention period | How long snapshots are kept, enforced by a tool or policy outside the CSI snapshot object | Daily or weekly recovery points where a backup tool or scheduler owns expiry | Whether your backup tool or scheduler, not the snapshot class, removes old snapshots |
| Clone from live PVC | Creates a new volume directly from an existing PVC | Fast copies for test or analytics when the source is quiet or consistent at the application level | Whether cloning from an in-use PVC is supported for your volume mode and data |
| Clone from existing snapshot | Creates a new volume from a point-in-time VolumeSnapshot | Restores or forks from a known recovery point, especially after a bad change | Snapshot is ready to use and the clone storage class matches the snapshot's pool |
| Namespace placement | Snapshot and PVC must share a namespace; the VolumeSnapshotContent is cluster-scoped | Keep app snapshots with the app; use a separate restore namespace to limit access to clones | Access rules for who can create snapshots and clones in each namespace |
Setting up snapshots and clones with Rook Ceph CSI
This sequence takes a Rook-managed Ceph cluster from missing snapshot prerequisites to a verified RBD and CephFS snapshot, and a restored clone that a workload can mount and read. Each step ends with a check so you know the next step is safe to run.
Confirm snapshot CRDs and controller
The cluster administrator checks that the VolumeSnapshot, VolumeSnapshotContent and VolumeSnapshotClass CustomResourceDefinitions are installed, and that the snapshot controller runs in the cluster. Use kubectl get crd to list the snapshot CRDs and kubectl get pods across the kube-system namespace (or wherever your snapshot controller is deployed) to confirm the controller pod is Running. Confirm the snapshot CRD versions match what your Kubernetes release expects, since the Kubernetes release notes and your distribution documentation describe which API versions are served. The step is done when every snapshot CRD is listed and the controller pod reports Ready. If the CRDs are missing, install them before continuing, because the CSI driver cannot create snapshot objects without them.
Verify CSI driver snapshot support
Confirm that the Rook Ceph CSI driver deployed in your cluster includes the snapshotter sidecar for both the RBD and CephFS provisioners. Check the operator and CSI provisioner pods with kubectl get pods in the Rook namespace and inspect the provisioner deployment containers for a snapshotter container. Compare the installed Rook and Ceph versions against the Rook documentation for your release, since snapshot support and supported CSI features depend on those versions. Also confirm that the Ceph cluster itself is HEALTH_OK with ceph status run from the toolbox pod. The step is done when you have written down the Rook version, the Ceph version and the snapshotter container presence for both drivers. If either provisioner lacks the snapshotter sidecar, stop and resolve the Rook version before creating any classes.
Create RBD and CephFS snapshot classes
Create one VolumeSnapshotClass for each driver, pointing the RBD class at the RBD CSI driver and the CephFS class at the CephFS CSI driver, and set the secret references and cluster ID that match your Rook CephCluster. Apply the manifests with kubectl apply and set the deletionPolicy deliberately, since Delete removes the Ceph snapshot when the VolumeSnapshotContent is removed and Retain keeps it. Confirm the names and parameters against the Rook CSI snapshot examples for your release. Run kubectl get volumesnapshotclass to verify both classes exist and that the driver names match the installed provisioners. The step is done when both classes are listed with the correct driver and deletion policy, which you should record because it controls what happens to backups later.
Take a test snapshot of a source PVC
Create a small test PVC on each StorageClass, write a known file into it from a short-lived pod, and then create a VolumeSnapshot that references that PVC and the matching snapshot class. Use kubectl apply for the snapshot manifest and wait for the object to report readiness. Run kubectl describe volumesnapshot and look for readyToUse set to true and a boundVolumeSnapshotContentName. The Ceph-side check is that the snapshot appears where expected: for RBD, run rbd snap ls against the image in the block pool, and for CephFS, check the subvolume snapshot list with ceph fs subvolume snapshot ls for the subvolume group. The step is done when readyToUse is true and the Ceph CLI shows the snapshot on the correct pool or filesystem.
Restore a clone into a new PVC
Create a new PersistentVolumeClaim whose dataSource references the VolumeSnapshot, with the same StorageClass and a requested size at least as large as the snapshot's source. Apply it with kubectl and watch the claim with kubectl get pvc until it reports Bound. For RBD, the new image should appear in the pool as a clone of the snapshot, which you can confirm with rbd info on the new image and check its parent. For CephFS, confirm the new subvolume exists in the subvolume group with ceph fs subvolume info, and that it is not dependent on the snapshot in a way that blocks deletion unexpectedly. The step is done when the PVC is Bound and the Ceph side shows the new image or subvolume, which confirms the clone was created rather than a plain empty volume.
Mount the clone and verify data
Start a pod that mounts the restored PVC read-only at first, then check that the file written before the snapshot exists with the same checksum as the original. Compare the checksum of the source file, which you recorded in the test snapshot step, with the restored file using sha256sum inside the pod. Once the data matches, you can remount read-write if the workload needs it. Keep the test pod and PVC until verification is complete, then delete the test resources in a deliberate order: the clone pod, the clone PVC, the snapshot, and finally the test source. The step is done when the checksums match and the pod reaches Running with the expected file contents, and you have recorded the result as evidence for your runbook.
Record limits before using for backups
Before you rely on snapshots as backups, confirm with your storage and platform owners what snapshots do not cover: they are typically kept inside the same Ceph cluster, so they do not protect against loss of that cluster, and they capture volume state without cluster resource definitions. Velero can take snapshots of persistent volumes through CSI, or use File System Backup with Kopia, but the File System Backup path backs up a live file system and requires object storage as its target, and its documentation warns that it uses a static common encryption key. Decide with your backup owner where copies go, how long they are kept, and how restores are tested. The step is done when you have written down the deletion policy, the restore test owner and the off-cluster copy plan.
What clones cost in capacity and performance
RBD and CephFS clones are copy-on-write, so creating one does not copy the data up front. A clone made from a snapshot shares the parent's blocks and only writes new blocks when the clone or the parent changes. Capacity grows with divergence, not with the size of the volume. The same principle applies to CephFS subvolume clones, but the clone runs as a background operation in the Ceph Manager, so the first data movement and its cost appear after the VolumeSnapshot is accepted rather than during the API call. Ask your storage team to confirm how the Rook release you run exposes clone progress, and watch for clones still in progress before you schedule snapshot-heavy jobs.
Deleting a source is constrained while dependents exist. An RBD snapshot that backs a clone cannot be removed until the clone is flattened or deleted, and CephFS will not let you remove a subvolume that still has pending or active clones. On Kubernetes, this shows up as a VolumeSnapshot or PersistentVolumeClaim that stays in a terminating state, so check the dependent objects before you blame the CSI driver. Flattening copies the shared blocks into the clone and removes the dependency, which frees the parent but consumes the full clone capacity, so treat it as a capacity decision rather than a cleanup step. Confirm the exact deletion and flatten behavior in the Ceph documentation for your release.
Clone creation adds load to a busy cluster, especially when many clones start from one snapshot or when a CephFS clone copies a large tree. Test the workload in a non-production pool with the same replication or erasure coding, media and network as production, then measure client latency while clones run. Ask the Ceph documentation for your release which limits apply to concurrent clones and how recovery and client I/O are prioritized. Schedule bulk clone jobs outside peak hours, and set a maximum concurrency in your own automation, because the cluster does not know which clones your restore runbook needs first.
Securing snapshot access and the data they hold
A VolumeSnapshot or clone is a new path to the same bytes as the source volume, so access control has to cover the Kubernetes API, the Ceph credentials the CSI driver uses and the namespaces that receive the data. The cards below cover the controls to design and verify before snapshots become a backup or data-sharing mechanism.
RBAC over snapshot resources
Creating a VolumeSnapshot or a PersistentVolumeClaim with a dataSource pointing at a snapshot is a permission in its own right, separate from reading or writing the PVC. Grant create, get, list and delete on volumesnapshots only to the roles that run backups or restores, and check which service accounts hold these verbs in each namespace. Test the role with a dry run before go-live, because a broad ClusterRole often grants snapshot access that nobody intended.
CephX caps for the CSI provisioner
The Rook CSI driver authenticates to Ceph with CephX users that have their own capabilities for RBD and CephFS. Review the caps granted to the provisioner and plugin users so they can create and delete snapshots and clones in the pools and subvolume groups they manage, and nothing outside them. Ask your storage team to confirm which CephX user each StorageClass and SnapshotClass references, and record the caps as part of the design.
Snapshots inherit source encryption and access
A snapshot is stored in the same Ceph pool as its source volume, so it carries the same encryption setup, pool-level access and replication as the data it copies. Deleting the source PVC does not remove the snapshot data, and anyone who can reach the pool or the backup target can read it. Confirm the encryption method for the pools with the storage team and decide who may read backup copies outside the cluster.
Namespace isolation for cross-team clones
A clone is created in the namespace of the new PVC, but its data comes from a snapshot in another namespace only when the snapshot is shared or the Kubernetes objects allow it. Decide which teams may clone production data into their own namespaces, and make that a documented approval step rather than an accident of shared RBAC. Use separate namespaces, quotas and network policy for test and development clones so copied data stays where its owners expect it.
Snapshot content and deletion policy
The deletionPolicy on a VolumeSnapshotClass decides whether deleting a VolumeSnapshot also deletes the underlying snapshot content in Ceph. A Retain policy keeps storage that nobody is tracking, while Delete removes the copy when the object is removed. Set the policy per use case, and include a periodic check that lists VolumeSnapshotContent objects without a matching owner.
Audit trail for snapshot operations
Kubernetes audit logging can record who created, restored or deleted snapshots and clones, which matters when copies of sensitive data appear outside the source namespace. Confirm that your cluster audit policy captures these resources at a level that shows the requesting user, and forward the logs to a store that the storage team cannot silently modify. Verify the setting with a test snapshot before relying on it for an investigation.
Questions teams ask before taking snapshots on Rook Ceph
Do Rook Ceph volume snapshots replace a backup?
No. A CSI snapshot is stored in the same Ceph cluster as the source volume, so it helps with logical mistakes such as a deleted table, a bad release or an accidental overwrite. It does not protect against losing the cluster, a failed pool, or a site-level incident. A backup needs a separate target, ideally outside the failure domain of the cluster, with its own retention rules and access controls. Velero can use CSI snapshots for persistent volumes and can also copy volume data to object storage through its File System Backup documentation, so the two methods often complement each other. Define restore objectives for each application before choosing which one covers it.
How long does a Rook Ceph snapshot take to create?
It depends on the volume size, how much data has changed since the last snapshot, the current load on the cluster and the layout of the pool. No fixed duration applies across clusters, so measure it on your own hardware. Create snapshots of representative volumes at realistic sizes, record how long each one takes to become ready, and repeat the test during busy and quiet periods, because recovery work and client I/O compete for the same OSDs. Keep creation time separate from restore and clone time, since a clone can take longer to become usable than the snapshot took to create. Ask any partner to include this measurement in the written scope.
Do databases need quiescing before a snapshot?
Usually yes for databases and for applications that hold in-memory state or write several files as one transaction. A crash-consistent snapshot captures the disk exactly as it stands at that moment, and many databases can recover from that state, but the result may not be application-consistent. Common approaches include flushing data and pausing writes through the application's own hooks, using database-native backup features, or snapshotting a replica. Velero documents that File System Backup backs up a live file system, which is less consistent than a snapshot. Test a restore for each database before relying on any method.
What does a snapshot project need from our team?
The work needs decisions and access more than heavy effort from your side. Plan for the following:
- Named owners for the storage classes, Ceph pools and the snapshot class that will be used
- Access to the Kubernetes clusters and the Ceph cluster so that the team can create objects and check health
- A decision on snapshot schedules, retention periods and who is allowed to restore
- A non-production namespace for restore tests and sign-off from the application owners
- A backup target, its credentials and a plan for handling encryption keys
Can our own team set up snapshots and clones without a partner?
Yes, if the team already runs Kubernetes and a Rook-managed Ceph cluster and has someone comfortable with CSI storage objects. The configuration itself is standard, so the harder work is verification: proving that each application restores to a usable state, that schedules run, and that the cluster stays healthy during snapshot activity. Your team's decision depends on how many workloads need snapshots, whether any database needs quiescing, whether an off-cluster backup is required, and whether there are people available to run restore tests and handle upgrades. DeployOpen implements, documents and tests this work under a written scope, and offers retainers for ongoing support.
What drives the cost of an outside snapshot engagement?
The cost follows the written scope, so the main drivers are the amount of work and the number of decisions involved. Ask a provider to price these items separately: the number of storage classes and workloads in scope, how many applications need quiescing hooks, whether an off-cluster backup target is built and tested, the restore test plan and documentation, and whether the work covers one cluster or several, including any air-gapped environment. Snapshot setup is usually one-time work, while upgrades, restore tests and monitoring recur, so ask which items sit in a retainer. DeployOpen does not publish a price list. A quote follows from the written scope, which should list deliverables and timelines before work starts.
Do CSI snapshots and clones behave the same on RBD and CephFS?
No, they use different Ceph objects, so plan them separately. An RBD snapshot is a point-in-time, read-only capture of a block image, and an RBD clone creates a new writable image from a protected snapshot. CephFS snapshots and clones work through subvolumes, which have their own groups and limits. Before you commit to a pattern, confirm in the Rook and Ceph documentation how each driver handles deletion of a parent snapshot, how clones are placed in pools, and what happens to clones during cluster recovery. Test these behaviors on your Ceph release, because they can change between versions.
How DeployOpen scopes snapshot and clone work for Rook Ceph
DeployOpen scopes snapshot and clone work through a written, fixed-scope plan agreed before any build starts. For Rook Ceph CSI, that plan names the StorageClasses that will carry RBD and CephFS volumes, the VolumeSnapshotClasses that map to each driver, the pools and subvolume groups behind them, and the reclaim and retention behavior for each class. It also records which namespaces and workloads are in the backup boundary, which are excluded, and who owns the restore runbook after handover. Prices follow from that written scope, so the more precisely the boundary and the number of StorageClasses are described up front, the more directly the quote can be compared with your own team's effort.
A large part of the engagement is test restores, because a snapshot that has never been cloned into a working volume has not been proven as a recovery path. The plan typically schedules restores from snapshots and clones into a separate namespace, checks that the application starts against the restored data, and records the time and steps taken. Work that recurs, such as re-running restore tests after a Rook or Ceph upgrade or reviewing snapshot retention, can be placed under a retainer rather than priced as a one-off project. Ask any provider, DeployOpen included, to state which of these items are one-time and which recur, and who performs each.
DeployOpen prepares configuration, runbooks and evidence that your internal or external auditors can review, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make those determinations. Before you sign, confirm with your own assessor which snapshot retention, restore testing and access records they will expect to see, and make sure those items appear in the written scope. DeployOpen does not hold vendor partner status for Rook or Ceph, so ask any partner to show the scope and the people who will deliver it.
