Ceph on Kubernetes with Rook guide

Ceph on Kubernetes with Rook: when the operator model fits your platform

If your platform team needs persistent storage for Kubernetes workloads, the core decision is whether Ceph runs inside the cluster through Rook or outside it as a cephadm-managed cluster. Choose Rook when your team already runs Kubernetes operations well, wants storage lifecycle managed through Kubernetes objects, and accepts that the storage layer shares the same failure domains and maintenance windows as the workloads it serves. Choose cephadm when storage must stay independent of cluster churn, when several clusters or non-Kubernetes consumers depend on the same Ceph pool, or when the storage team wants its own upgrade cadence. The Ceph documentation for cephadm points to Rook as the route for Kubernetes, and the CNCF lists Rook as a Graduated project since October 7, 2020, so the operator model is a supported path rather than an experiment. In the Rook model, the Rook operator creates and manages the Ceph daemons (monitors, managers, OSDs, metadata servers and gateways) as Kubernetes workloads. Consumers reach storage through the Container Storage Interface (CSI) drivers, usually one StorageClass for RBD block volumes and another for CephFS shared file systems. Day-one design work includes which nodes host OSDs, which disks are dedicated to each OSD, where the write-ahead log and database devices sit on HDD-based OSDs, and how monitors are spread across failure domains. The Ceph hardware recommendations advise a dedicated drive per OSD, shared SSDs for WAL and DB on HDD-based OSDs, enterprise-class SSDs for metadata, and bonded network links across two switches. Confirm each of these against the Rook version you plan to run, because device selection and node labelling rules differ between releases. The harder part is coupling. Rook pins a set of supported Ceph versions, so an operator upgrade and a Ceph upgrade are linked decisions, and the order matters. Check the Rook compatibility matrix for the exact Ceph releases your operator version supports before you schedule any change, and plan for the Squid end-of-life date and the Tentacle release line described in the Ceph release documentation. Node drains, kernel patches and cluster autoscaling can also take OSDs offline, so your failure handling must cover slow recovery, full-node loss and the rebalancing traffic that follows. Ask the vendor or your own platform team to show a tested runbook for a monitor quorum loss and for an OSD host failure before you commit. Day-two ownership is where many Rook deployments drift. Someone must watch cluster health, apply operator and Ceph security fixes, rotate keys, size memory for peak recovery rather than calm periods, and benchmark disks and network before adding capacity. The Ceph hardware recommendations specifically advise sizing for peak use and benchmarking before buying, because no two clusters behave alike. If the same team owns application releases and storage upgrades, expect contention during change windows. Write down who owns Rook operator upgrades, Ceph version upgrades, CSI driver updates and capacity forecasting before the first production volume is created.

Use this as a checklist: confirm your node count and disk layout can hold the failure domains you need, confirm the Rook and Ceph version pairing for your Kubernetes version, and confirm who is on call for storage incidents that also look like application incidents. If those answers are unclear, the cephadm route may be the safer starting point.

Which Rook and Ceph components you will configure and operate

Ceph on Kubernetes with Rook splits responsibility between the Rook operator, which reconciles Kubernetes resources into Ceph daemons, and your team, which still decides hardware layout, pools, storage classes and upgrade order. The cards below cover each component you must understand before building, and what the operator owns versus what you configure directly.

Rook operator and its scope

The Rook operator runs in your Kubernetes cluster and watches custom resources that describe the Ceph cluster. It creates, updates and repairs the Ceph daemon deployments that those resources describe. Your team still owns the Kubernetes version it runs on, its RBAC, and the upgrade sequence for the operator itself. Check the Rook and Ceph version matrix in the Rook documentation before choosing either version.

CephCluster resource and node selection

The CephCluster resource defines the cluster's Ceph version, data and monitor placement, and which nodes and devices Ceph may use for OSDs. The operator turns these settings into daemon placement, but your team decides which nodes are dedicated to storage and which disks are eligible. Mistakes here are expensive to reverse, so review device filters and failure domain layout with the hardware plan before the first apply.

Block pool and filesystem resources

CephBlockPool and CephFilesystem resources define the replicated or erasure-coded pools behind RBD images and CephFS volumes. The operator creates the underlying Ceph pools, but your team sets replica counts, failure domains and device classes. Confirm how each pool maps to your data protection target before any workload writes to it.

CSI drivers for RBD and CephFS

The Ceph CSI drivers run on Kubernetes nodes and handle provisioning, attaching, mounting and expanding volumes for RBD block devices and CephFS shares. The operator deploys them, but node plugin behavior depends on kernel modules and host configuration that your team controls. Validate that each node image supports the kernel features your chosen volume types need.

Storage classes and access modes

StorageClass objects expose RBD and CephFS pools to applications and carry the secrets and parameters the CSI drivers need. Your team defines the classes, reclaim policies and access modes, and decides which workloads may use ReadWriteMany on CephFS versus ReadWriteOnce on RBD. Review class parameters against the pool design so that applications cannot reach pools outside their intended scope.

Monitors, managers and OSDs

Monitors maintain cluster maps and quorum, Managers run modules such as the dashboard and metrics, and OSDs store and replicate data on individual devices. Rook schedules these daemons, but your team must place monitors across failure domains, keep an odd number of them, and confirm that enough OSDs remain to satisfy pool replication. Verify quorum and daemon placement after every node change.

MDS daemons for CephFS

Metadata servers, or MDS daemons, hold the namespace for CephFS and must run for any CephFS filesystem to serve clients. Rook schedules them from the CephFilesystem resource, but your team sets their count, standby behavior and placement, and sizes their memory for the number of open files and clients. Check MDS failover behavior in a test environment before relying on shared filesystems in production.

Rook in-cluster or external cephadm cluster: the decisions to settle first

This table compares running Ceph inside Kubernetes with Rook against consuming an external cephadm-managed cluster from Kubernetes. Read each row as a decision to settle before design starts, with the fit and the ongoing operational cost of each option. The main takeaway is that the upgrade owner and the skills your team must hold usually decide the model more than hardware does.

Decision	Rook in-cluster	External cephadm cluster consumed by Kubernetes
Converged or dedicated storage nodes	Fits when storage and workloads share hosts and you accept storage pods competing with applications for CPU, memory and disks. Operational cost: node drains and scheduling changes affect OSDs directly.	Fits when storage runs on dedicated hosts outside the Kubernetes failure domain. Operational cost: a separate server estate to patch, monitor and power.
Who upgrades Ceph and the Rook operator	Your platform team upgrades the operator, CRDs and Ceph daemons through Kubernetes releases. Confirm the supported operator and Ceph version pairing for each release before scheduling.	Your storage team upgrades Ceph with cephadm on its own cadence. Kubernetes teams upgrade only the CSI driver and client packages. Confirm CSI compatibility with the Ceph release you run.
Where Monitors and Managers run	Monitors and Managers are pods scheduled onto nodes you label. Quorum depends on node placement and on keeping enough Monitors up during drains. Confirm placement rules with the Rook documentation for your version.	Monitors and Managers run as containers on cephadm-managed hosts, separate from Kubernetes control plane nodes. Quorum problems are diagnosed with Ceph tools, not Kubernetes events.
How disks are assigned to OSDs	Rook discovers or lists devices in the cluster custom resource, so disk selection is declared in Kubernetes manifests. Changes to device lists need review as code. Confirm how your version handles metadata devices.	Disks are added with cephadm service specifications or manual commands on the storage hosts. Kubernetes never sees the disk layout, so the storage team owns the mapping end to end.
Failure domains shared with workloads	A host reboot, kernel fault or node eviction can remove OSDs and Monitors at the same time as application pods. Plan CRUSH host and rack domains so that maintenance on one node does not break quorum.	Storage hosts fail independently of Kubernetes nodes, which keeps application outages and storage outages separate. The cost is that network paths between the two sides become a shared failure domain.
Team skills required	Your team must run Kubernetes operators, custom resources and storage troubleshooting together. Staff need both Kubernetes and Ceph depth on call.	Your storage team needs Ceph and Linux operations skills. Kubernetes staff need CSI, StorageClass and network troubleshooting skills, which is a narrower set.
Fit for multiple Kubernetes clusters	Each cluster normally runs its own Ceph cluster, so each one carries its own upgrade and monitoring load. Confirm whether a shared external cluster is an option for your topology.	One Ceph cluster can serve several Kubernetes clusters through separate CSI configurations and pools. Access separation and quota policy then need explicit design.
Recovery and day-two access	If the Kubernetes control plane is down, storage management is affected too. Keep a documented break-glass path for Ceph command-line access.	Ceph stays manageable through its own tools when Kubernetes is degraded. Confirm the CSI consumption path still needs Kubernetes to be healthy for new volume attachments.

How the Kubernetes deployment changes across single-site, dedicated and air-gapped environments

In a converged layout, Rook runs its Ceph daemons on the same worker nodes that run your application pods. The operator schedules monitors, managers, OSDs and metadata servers onto nodes that have the disks and labels you select in the cluster custom resource. This is the simplest model to start with because there is one control plane and one place to look for storage problems, but it means storage and workloads compete for CPU, memory and network on the same machines.

Node placement: Storage daemons share nodes with application pods, so you need resource requests and limits that reserve capacity for OSDs during peak load.
Image sourcing: Ceph and Rook images are pulled from the registries you configure, and each worker needs access to them during node replacement.
Network separation: Public and cluster networks usually run over the same pod network unless you configure host networking or multiple interfaces, which you must plan before deployment.
Upgrade sequencing: Rook upgrades the operator first, then the Ceph version, and drains nodes one at a time, so application disruption depends on how pods are spread.
Confirm: Ask the vendor which Rook and Ceph versions your chosen operator release supports, and confirm your node disk layout and CPU and memory headroom in your own environment.

Building Ceph on Kubernetes with Rook, step by step

This sequence takes a Kubernetes cluster from an unchecked baseline to a Rook-managed Ceph cluster that serves block and file volumes to workloads. Each step closes with a check you record before moving on. Hardware and network choices should follow Ceph's hardware recommendations and a benchmark on your own equipment.

Confirm Kubernetes version and node labels
The platform team checks that the Kubernetes version is on the Rook compatibility list for the Rook release you plan to deploy, and that the cluster has the node roles, storage hosts and failure-domain labels the Ceph design depends on, such as zone or rack. Storage nodes are labeled so Ceph daemons and OSDs land only where the hardware exists. Done means a written inventory of nodes, labels, kernel and disk layout that the reviewer has signed, and a test pod schedules onto each labeled storage node. Confirm the supported version range with the Rook documentation before you start, not after the first failure.
Deploy the Rook operator and CRDs
Install the Rook operator and its custom resource definitions into a dedicated namespace, using the manifests or Helm chart for the release you have approved. Apply role bindings and service accounts exactly as the release documents, and keep the operator's image source in your internal registry if your policy requires it. The operator does nothing until a CephCluster resource exists, so this step only confirms the control plane is ready. Done means the operator pod reports ready, the CRDs are registered, and the operator log shows no errors on startup.
Define the CephCluster with monitors and managers
Write the CephCluster resource with the Ceph release image, the data directory on each host, and monitor and manager placement rules that spread daemons across the failure domains you labeled earlier. Confirm the monitor count and quorum requirements in the Ceph documentation for your release before you commit to a number. Apply the resource and watch the operator bring up monitors first, then managers. Done means the monitors form quorum, at least one manager is active, and the cluster status in the toolbox pod shows HEALTH_OK or a warning you have reviewed and accepted in writing.
Assign dedicated drives and place WAL and DB
Declare one dedicated drive per OSD in the CephCluster storage section, so no drive is shared with the operating system or another OSD. For HDD-based OSDs, Ceph's hardware recommendations advise placing the WAL and DB on shared SSDs, so define those partitions or devices in the same storage section and keep the enterprise-class SSDs used for metadata separate from bulk data. Size the SSD tier using the hardware guidance and a benchmark on your own equipment. Done means every intended drive appears as an OSD in the OSD tree, each OSD shows its WAL and DB location, and no OSD is marked down.
Configure bonded network links across two switches
Configure the host network so the public and cluster traffic use bonded links, with each bond carrying one physical link to each of two switches. The bond mode, LACP settings and VLAN design must match what your network team has approved and what the switches support. Keep the Ceph public and cluster networks defined in the CephCluster resource to match the bonds. Done means each storage node survives the loss of one switch link in a controlled test, the bond reports both members active, and the Ceph daemons bind to the expected addresses.
Create RBD pool and CephFS with CSI classes
Create a replicated block pool for RBD, with its CSI storage class, and a CephFS filesystem with its own CSI storage class. Set replication and failure domain in the pool specification so copies land on different hosts or racks, as your design requires. Confirm the CSI driver pods are running on each node that needs to mount volumes. Done means both storage classes appear in the cluster, the pool and filesystem report active, and the Ceph status shows the data pools with the expected replication size.
Test PVC provisioning and mounting
Create a test PersistentVolumeClaim against each storage class, then mount each volume in a test pod on more than one node. For RBD, confirm read and write from the pod and that the volume re-attaches after the pod moves. For CephFS, confirm a shared mount works from two pods on different nodes. Delete the test workloads and confirm that the reclaim policy removes the backing images or subvolumes as your design intends. Done means every test claim binds, every mount succeeds, and the deletion results are recorded.
Verify cluster health before handing over
Before any production workload uses the cluster, run a final health review with the Ceph status command, the Rook cluster status, and the operator logs. Check monitor quorum, manager failover, OSD states, placement group states, and recovery activity. Simulate one controlled failure, such as draining a storage node, and confirm the cluster returns to a healthy state without manual intervention. Done means the signed-off runbook records the health baseline, the failure test result and the owners for each alert, and the handover is approved in writing.

What drives effort and cost in a Rook-based Ceph build

This table lists the main factors that change the effort and cost of a Rook-based Ceph build, how each one pushes them up, and how to keep it manageable. Read it as a checklist for scoping, since most drivers compound when several apply at once. Quote requests should state each driver's value so the written scope reflects them.

Driver	Why it changes effort or cost	How to keep it down
Kubernetes clusters and namespaces consuming storage	Each cluster needs its own Rook operator configuration, CSI driver setup, and RBAC scoping. Each namespace adds StorageClass and secret handling, plus access reviews.	Consolidate consumers onto fewer clusters where isolation allows. Standardize namespace templates and document which teams own each StorageClass.
Raw capacity and disk count	More OSDs mean more hardware validation, more failure scenarios to test, longer recovery windows and more disks to replace over time. Usable capacity is lower than raw capacity once replication or erasure coding is applied.	Size usable capacity from the workload forecast, not the raw drive total. Ask the vendor or integrator to show the replication or erasure-coding overhead in the quote.
Network design	Ceph separates client and replication traffic in most designs, and bonded links across two switches add cabling, switch configuration and testing. Kubernetes pod networking can add a further layer to validate.	Agree the network layout before hardware is ordered. Reuse a standard switch and bonding pattern across nodes so the test matrix stays small.
Number of storage classes and access modes	Each RBD, CephFS or RGW-backed class needs its own pool or filesystem, reclaim policy, access mode validation and failure testing. ReadWriteMany needs CephFS, which has different sizing and recovery behavior from block.	Limit classes to the workload types you actually run. Map each application requirement to one class with a documented access mode before design starts.
Workload mix (databases, shared file, bulk data)	Databases need low latency and tight recovery targets, which push toward SSD-backed pools and benchmarking. Shared file and bulk data tolerate more latency but add metadata and capacity planning.	Separate latency-sensitive databases from bulk data into different pools or device classes. Benchmark representative workloads before purchase, as the Ceph hardware guidance advises.
Existing Kubernetes platform maturity	A cluster without GitOps, node pool standards, backup tooling or monitoring must have these built during the Rook work. Upgrades of Kubernetes and Rook must be sequenced together, which adds coordination.	Confirm which platform prerequisites already exist. Ask for the Rook and Kubernetes version compatibility check to be part of the design phase.
Compliance evidence requirements	Assessors expect documented configuration, access control records, encryption settings, backup and restore test results and change history. Producing this evidence adds documentation and test time that is separate from the build.	Decide early which controls need evidence and how it will be collected, such as exported configuration and recorded restore tests. DeployOpen prepares evidence for assessments but does not issue audit reports or certifications.
Number of environments (dev, test, production, disaster recovery)	Every environment repeats installation, configuration and testing. A disaster recovery site adds replication design, failover runbooks and periodic recovery drills.	Keep dev and test on the same Rook manifests as production, with only sizing and secrets differing. Scope disaster recovery as a separate phase so its cost is visible.
One-time versus recurring costs	One-time costs cover architecture and design, hardware validation, Rook and Ceph build, integration with Kubernetes and monitoring, performance and failure testing, and handover documentation.	Recurring costs cover Ceph and Rook upgrades, operations and on-call staff time, hardware replacement, monitoring and support agreements. Ask for both one-time and annual recurring estimates in any quote, and confirm which party owns each.

How to verify Rook-managed Ceph works and fails safely

Verification of a Rook-managed Ceph cluster should answer one question before go-live: when a node, disk or Monitor fails, does the cluster keep serving data, recover on its own, and return clients to a healthy state without manual data repair. Start with a baseline. Confirm that the CephCluster custom resource reports a healthy state, that the Ceph status shows all Monitors in quorum, that Managers are active, and that every OSD is up and in. Record the output of these checks as the reference point for every later test. Rook surfaces most of this through the CephCluster status and the rook-ceph toolbox, so confirm which of those your team will use as the source of truth.

Next, confirm provisioning for both storage classes across nodes. Create test PersistentVolumeClaims against the RBD and CephFS StorageClasses, mount them in pods scheduled onto different worker nodes, and write data from each pod. Check that the CSI provisioner and attacher complete without errors, that volumes bind in the expected access mode, and that a ReadWriteMany CephFS claim is readable and writable from more than one node at the same time. Repeat a subset of the checks after any change to node labels, topology or CSI driver settings, because those changes are a common cause of silent placement problems.

Then test the failure modes one at a time, and write down the expected result before each test. For a node loss, cordon and drain or power off one worker that hosts OSDs and a Monitor, and confirm that pods with attached volumes reschedule, that the cluster marks the affected OSDs down and begins recovery, and that the pods reattach their volumes. For a disk loss, remove or fail one OSD device and confirm that the OSD is replaced through your documented procedure and that data returns to full redundancy. For a Monitor loss, stop one Monitor and confirm that quorum holds, then confirm that Rook restores the Monitor count. Each expectation should name the observable signal, such as a health warning, a degraded placement group count or a pod event, so that a test can fail clearly.

Verify that recovery completes, not just that the cluster stops complaining. Track the time to return to HEALTH_OK, the number of degraded or misplaced objects, and whether client I/O from a running test workload resumed without remounts. Confirm the outcome against your own service levels, including how long an application may pause and how much recovery traffic your network can absorb. Note that recovery speed depends on the hardware, the replication or erasure-coding layout and the backfill settings in your environment, so set your targets from your own baseline rather than from any published figure.

Finally, rehearse an operator or Ceph upgrade in a non-production cluster that matches production as closely as practical, including the same Kubernetes version, storage devices and network design. Run the same health and client checks before, during and after the upgrade, and record every deviation between expected and observed results in a single log. Use that log to decide the production change window, the rollback criteria and the list of confirmations you need from your Rook and Ceph release notes before you approve the production change.

Securing the Rook and Ceph control plane in Kubernetes

The six controls below cover the Kubernetes layer that drives a Rook-managed Ceph cluster and the Ceph services it exposes. They connect because a weakness in any one of them, such as broad custom resource rights or an exposed Dashboard, can give access to the data plane. Each item is an input to your security team's review and should be confirmed against your Rook version and Kubernetes distribution.

Restrict who can change CephCluster and StorageClass

Anyone who can edit a CephCluster resource can change monitor placement, device selection and the storage classes that workloads consume, which affects every tenant on the cluster. Bind create, update and delete rights on these custom resources and on StorageClass objects to a small named group through Kubernetes RBAC, and keep read access in a separate role. Review which service accounts can write to the namespace where the Rook operator runs, because operator permissions reach the Ceph resources it manages. Confirm the role model with your security team before the first cluster build.

Keep Ceph keys out of plain manifests

Ceph authentication keys and Rook-generated credentials live in Kubernetes secrets, and manifests stored in Git or applied by CI pipelines can leak them if they carry values in plain text. Supply any credential you own, such as object gateway or Dashboard passwords, through an external secrets store or an encrypted form, and limit secret read access by namespace. Confirm with your security team whether secrets are encrypted at rest on your Kubernetes distribution, and list every identity that can read secrets in the Rook namespace.

Separate public and cluster networks

Ceph uses the public network for client and workload traffic and the cluster network for replication and recovery between OSDs. Place the cluster network on dedicated VLANs or physical links so rebuild traffic cannot be reached from application networks. In the CephCluster network settings, confirm which interfaces each network binds to on every node, and check that pod networking does not expose Ceph daemons to client subnets. Ask your network team to confirm bonding and switch redundancy for both networks.

Enable msgr2 secure mode and retest performance

The messenger v2 protocol listens on port 3300, and its secure mode encrypts all traffic after authentication with AES-GCM and adds integrity checks. Enable the mode and confirm in the cluster configuration and on running daemons that it applies to every peer, including clients outside the cluster. Encryption adds CPU work, so benchmark your own storage classes before and after the change and record the results in the acceptance report. Confirm the required mode and the minimum Rook version that exposes the setting with your security team.

Limit Dashboard exposure through ingress

The Ceph Dashboard is a management interface with cluster-wide reach, so it should not share a general ingress route with application traffic. Expose it through a dedicated ingress or internal service, restrict access with Kubernetes network policies to administrator subnets or a bastion, and require the single sign-on method your identity team selects, such as SAML 2.0 or OAuth2 where your cephadm or deployment model supports it. Check that the Dashboard service is not published as a NodePort or LoadBalancer by default. Confirm the exposure model with your security team.

Audit operator and administrator access

The Rook operator and Ceph admin tools hold broad rights over storage, so their use needs a record. Log Kubernetes API activity against Rook custom resources and the operator namespace, and keep a register of who can exec into the toolbox pod, read the Ceph admin keyring and hold cluster-admin bindings. Review the register on a fixed schedule and after staff changes, and move standing access to a break-glass process where possible. Ask your security team which audit log retention period and review cadence they require for storage administration.

Operating Rook-managed Ceph after handover

After go-live, the platform team owns a Rook-managed Ceph cluster as a set of Kubernetes objects: the Rook operator deployment, the CephCluster resource, CephBlockPool and CephFilesystem definitions, and the storage classes that workloads consume. Day-two operations start with monitoring Ceph health and OSD status. Decide early whether the team will use the Ceph Dashboard, the Prometheus and Grafana stack that Rook can integrate with, or existing monitoring tools already in your estate. Whichever you choose, alert on HEALTH_WARN and HEALTH_ERR states, on OSDs that are down or out, on monitor quorum changes, and on PGs that stay degraded or inactive. Confirm with your Rook version which metrics and dashboards it ships, rather than assuming the defaults match your alerting standards.

Capacity headroom needs its own routine. Track raw and per-pool usage, and plan against the failure domain you designed, because a cluster that is comfortably full in aggregate can still lose the ability to recover after a host failure. Recovery needs free space on surviving OSDs, so set a review threshold well below the nearfull and full ratios and treat each threshold crossing as a procurement or rebalancing trigger. Record how long a single OSD or host recovery took during testing, then compare that baseline against live growth. Your hardware vendor and the Ceph hardware recommendations both expect sizing to account for peak memory and recovery load, so revisit that sizing when the workload changes.

Back up the cluster configuration and the Kubernetes manifests as a single, versioned set. That includes the Rook operator and CephCluster manifests, pool and storage class definitions, Helm values if used, secrets and keys that Rook generates, and the namespaces and RBAC that surround them. Store these in a location separate from the cluster itself. Ask your team to confirm how cephx keys and monitor keyrings are captured and restored, and test a restore into a scratch namespace before you rely on it. Data backup is a separate question covered by the high availability and data protection topic, so do not treat a manifest backup as a data backup.

Plan upgrades in two tracks: the Rook operator and the Ceph release it runs. Follow the upgrade order documented in the Rook project for the operator version you deploy, and check the Ceph release notes for each step. Ceph's active releases are Tentacle (20.2.x, first released November 18, 2025, latest 20.2.4, estimated end of life June 1, 2027) and Squid (19.2.x, first released September 26, 2024, latest 19.2.7, estimated end of life October 31, 2026). Actively maintained releases receive periodic backports and security fixes, so schedule patch updates inside the minor line and plan a move to Tentacle before Squid reaches end of life. Confirm the current end-of-life dates on the Ceph releases page before you commit a change window, because estimates can move.

Keep four documents current. The build record captures what was deployed: hardware, network design, failure domains, pool layout, Rook and Ceph versions, and the configuration decisions behind them. The version matrix maps Kubernetes, Rook operator, Ceph release, and client components such as CSI drivers, so that upgrade planning starts from facts. Runbooks for node and disk replacement cover drain and cordon steps, OSD removal and re-creation, device identification, and the health checks to pass before closing the change. The verification results are the acceptance test outputs from handover, including failure injection and recovery timings, which give the team a baseline to compare against after future changes. Assign an owner for each document and review them at every upgrade.

Questions teams ask before running Ceph on Kubernetes

Is Rook the right choice compared with cephadm for our cluster?

Rook is usually the better fit when your workloads already run on Kubernetes and your platform team wants Ceph managed through Kubernetes custom resources, storage classes and the same GitOps and RBAC model as the rest of the cluster. cephadm is usually the better fit when you want Ceph to run as its own container-managed cluster on dedicated hosts, separate from your application platform. The Ceph documentation points to Rook as the Kubernetes route, so the choice is about where you want Ceph's lifecycle to live, not about feature parity. Ask both candidate paths to be tested against your failure domains, network design and upgrade process before you commit.

How much effort does a Ceph on Kubernetes build take, and what drives it?

Effort depends mostly on the number of nodes and disks, the network design, the storage classes you need and how many integrations you connect, such as identity, monitoring and backup. Hardware planning and benchmarking take time because Ceph's hardware guidance asks you to size memory for peak use and to benchmark before buying. Each additional pool type, device class or storage class adds design and test work. A fixed-scope quote should list these items as separate deliverables with agreed timelines, so you can see which part of the effort your environment drives.

What skills does our team need to run Ceph on Kubernetes?

Your team needs working knowledge of Kubernetes operations, including node pools, storage classes, persistent volume claims, RBAC and upgrades. It also needs enough storage knowledge to read Ceph health output, understand replication and placement groups, and recognize when a disk or OSD needs attention. Network skills matter because Ceph traffic is sensitive to bonding, MTU and switch redundancy. Plan for at least one person who can own day-two operations, and confirm the skill set against the Rook documentation and your own staffing before you assume a split of responsibilities.

Can our own team build Ceph on Kubernetes without a partner?

Yes, many teams build it themselves, because Ceph and Rook are open source and the documentation is public. The questions to answer honestly are whether you have the storage and Kubernetes skills in-house, whether you can test failure scenarios before production use, and whether someone can respond when a node or disk fails at an inconvenient time. A partner can take on design, build, testing and documentation under a written scope, which lets your team focus on operating the platform. Compare the internal time required with the scope of a fixed engagement rather than the build alone.

What does a fixed-scope Ceph on Kubernetes engagement cover?

A fixed-scope engagement covers the agreed deliverables written down before work starts, such as the architecture for hardware, network and failure domains, the Rook deployment, the storage classes for block and file workloads, testing of node and disk failure, and documentation for your operators. It does not cover work outside that written scope unless both sides agree a change. Ask for the list of deliverables, the acceptance tests, the handover items and whether ongoing support or upgrades are priced separately as a retainer. DeployOpen does not publish a price list, so the quote follows from the scope you agree.

How are Ceph and Rook upgrades handled on a running cluster?

Upgrades are planned as a controlled sequence, not a single event. Check which Ceph release your Rook version supports, confirm the target release is actively maintained, and follow the documented upgrade order for monitors, managers, OSDs and gateways. The Ceph releases page lists Squid with an estimated end of life of October 31, 2026, so plan a path to Tentacle well before that date. Before any upgrade, confirm cluster health is clean, take backups and test the sequence in a non-production environment. Ask whether the scope includes pre-upgrade checks, rollback steps and post-upgrade validation.

What happens to our data when a node or disk fails?

Ceph keeps data available by replicating or erasure-coding objects across failure domains that you define, such as hosts, racks or rooms. When an OSD or disk fails, Ceph marks it out and recovers the affected data onto surviving OSDs, and the time this takes depends on data volume, network speed, recovery settings and the amount of free capacity. Monitor quorum must stay intact for the cluster to keep serving I/O, so monitor placement across nodes matters. Confirm the exact behavior for your pool settings and failure domains in the Ceph documentation, and test a disk pull and a node loss in your own environment before go-live.

How DeployOpen scopes, builds and hands over Rook-managed Ceph

DeployOpen plans and builds Ceph on Kubernetes with Rook inside your own infrastructure, starting from a written scope that sets the Kubernetes platform, hardware, storage classes and test criteria. The work ends with documented storage classes, runbooks and a handover your platform team can operate without us.

How we scope

Engagements are fixed-scope, with the Rook version, Ceph release and deliverables agreed in writing before work starts.
We review your Kubernetes distribution, worker node layout, disk and network plan against Rook and Ceph hardware guidance.
The written scope names which RBD and CephFS storage classes and pools are in and which workloads are out of scope.

How we build

We deploy the Rook operator, CephCluster and OSD configuration in a staging cluster before anything reaches production.
We configure RBD and CephFS storage classes and test volume provisioning, attachment, expansion and reclaim behavior.
We run node drain, OSD failure and monitor failover tests and record results against the acceptance criteria in the scope.

How we hand over

You receive the Rook and Ceph configuration as code, with version pins and the agreed Ceph release recorded in the handover.
Runbooks cover storage class use, node maintenance order, OSD replacement and health checks your team will run.
Your platform and storage engineers walk through the failure test results and the upgrade path to the next Ceph release.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer, scoped separately from the build.
Upgrade planning covers the Rook operator, Kubernetes compatibility and the Ceph release lifecycle for your cluster.
Ask any provider to confirm which Ceph releases and Rook versions their support covers before you sign.

Ceph on Kubernetes with Rook