Kubernetes storage with Rook and Ceph: what the integration provides
A platform team that needs persistent block volumes for databases, shared file volumes for workloads that mount the same data from several pods, and S3-compatible object storage for backups and artifacts can serve all three from one storage system. Ceph provides block (RBD), file (CephFS) and object (RADOS Gateway) storage. Rook is the Kubernetes-side method for deploying and managing Ceph, running its daemons as Kubernetes workloads and handling their lifecycle through custom resources. The Cloud Native Computing Foundation lists Rook as a Graduated project, a status it received on October 7, 2020. The integration therefore gives the platform team a storage layer that is managed with the same tools as the rest of the cluster.
Ceph's source is licensed under LGPL-2.1 or LGPL-3, according to the COPYING file in its repository, and some bundled components carry other licenses. Your legal or open-source review should cover those components before the design is approved. The project itself offers no paid edition, so the software license carries no fee. Costs come from hardware, the staff who operate the cluster, any commercial support you contract from a separate provider, and implementation work such as DeployOpen's, which is quoted from a written scope. Ask each bidder to separate those cost lines so you can compare the options on equal terms.
The central decision is where Ceph runs. In the first option, Ceph runs inside the Kubernetes cluster and Rook deploys its monitors, managers, OSDs and gateways on the cluster's nodes. Storage and compute then share hardware, failure domains and upgrade windows, which suits a team that owns both layers. In the second option, the cluster connects to an existing external Ceph cluster that a storage team operates. Kubernetes upgrades and node drains stay separate from storage changes, but two teams must agree on access, capacity planning and incident response. Settle who performs Ceph upgrades, who is paged for OSD failures, and whether Kubernetes node maintenance can proceed without touching storage daemons before choosing.
Before you pick versions, confirm the Rook release and its Kubernetes compatibility in the Rook documentation, because the supported Ceph releases depend on the Rook version. The Ceph releases page lists Tentacle (20.2.x) and Squid (19.2.x) as the actively maintained lines, and it gives an estimated end of life of October 31, 2026 for Squid. Build the upgrade plan around those dates, and confirm each estimate against the current Ceph release page when you start. Record the chosen Kubernetes, Rook and Ceph versions in the design document so that each later upgrade starts from a verified compatibility matrix.
What the cluster, network and hardware need before Rook is installed
Rook can only be as stable as the platform under it. Settle the Kubernetes version, the Ceph release line, the drives, memory, network and air-gapped build inputs before anything is installed, because changing them later can mean rebuilding OSDs or redeploying the cluster. Each card below is a decision or check to close in writing with your team.
Supported Kubernetes version
Pick a minor version that the Kubernetes releases page lists as maintained. As of September 2026, the maintained minors were 1.37, 1.36 and 1.35, and 1.35 reaches end of life on February 28, 2027. Check the version skew policy as well: kubelets may run up to three minor versions behind kube-apiserver, and the control plane must be upgraded one minor version at a time. Plan the upgrade path now, because later storage upgrades must follow it.
Drives for OSDs and metadata
Give each OSD its own dedicated drive that Ceph uses exclusively. On HDD-based OSDs, Ceph's hardware guidance moves the write-ahead log and database (WAL and DB) to shared SSDs. Choose enterprise-class SSDs for that metadata role, and decide how many OSDs will share one SSD before procurement, because losing that SSD takes every OSD it hosts offline. Confirm the device path of each candidate drive in your node inventories.
Bonded links across two switches
Ceph's hardware guidance recommends bonding network links across two switches, so that a single switch failure does not isolate storage nodes. Decide whether public and cluster traffic will be separated, and agree the bond mode and MTU with your network team. Test failover in a staging environment by pulling one link and then one switch before the cluster stores any data.
Memory sized for peak use
Ceph's hardware recommendations say to size memory for peak load rather than calm periods, and to benchmark before buying because no two clusters behave alike. Recovery, backfill and rebalancing after a node or drive failure push memory use well above steady-state levels. Ask your hardware supplier for the per-OSD memory assumption behind the quote, and confirm it against benchmark results from hardware similar to yours.
Ceph release line and Rook matrix
Choose either Tentacle (20.2.x) or Squid (19.2.x). The Ceph releases page estimates end of life for Tentacle on June 1, 2027 and for Squid on October 31, 2026, so Squid has the shorter remaining support window. Confirm the pairing of Ceph, Rook and Kubernetes versions against the Rook compatibility matrix before you commit. Estimated dates can change, so reconfirm them on the day you build.
Certificates, DNS and private registry
Air-gapped builds need every image pulled from a private container registry that your nodes trust. kubeadm can list and pull control plane images and point kubeadm init at that registry with the image repository option, and the Rook and Ceph images need the same treatment. Issue certificates for the registry and cluster endpoints, and confirm that DNS names resolve inside the isolated network before the first install.
Running Ceph inside the cluster or connecting an external Ceph cluster
The first decision for Rook-managed storage is where the Ceph daemons run. Rook can run Ceph inside the Kubernetes cluster, with monitors, managers, OSDs and gateways scheduled as pods on storage nodes, or the cluster can consume an existing Ceph cluster that runs outside Kubernetes. Running Ceph inside the cluster ties storage lifecycle, placement and day-two changes to Kubernetes objects and to the people who operate the cluster. An external Ceph cluster separates storage failures, Ceph upgrades and capacity planning from the Kubernetes cluster, so a bad Kubernetes upgrade does not directly touch the OSDs that hold data. The external model also fits cases where the same Ceph cluster serves other platforms, such as OpenStack block storage, which is a common reason to keep Ceph outside Kubernetes.
Five drivers usually settle the choice. The first is who operates storage. If a platform team already runs Ceph for other workloads and has the skills to upgrade and recover it, an external cluster avoids two operating models for one technology. The second is whether other systems need the same Ceph. Shared pools, shared capacity and separate tenant or pool access are simpler to govern when Ceph is not owned by one Kubernetes cluster. The third is node pool design. In-cluster Ceph needs storage nodes with dedicated drives and enough memory and network capacity, which means those nodes cannot be treated as disposable compute. The fourth is upgrade cadence. Kubernetes minor releases arrive about three times per year, and Ceph has its own release train, so coupling both on one schedule increases the work and the risk for each change. The fifth is blast radius. A Kubernetes upgrade that drains storage nodes or breaks the Rook operator affects every workload that depends on those volumes.
For a new Rook deployment, the Ceph-native tool is cephadm, which manages Ceph as containers and bootstraps a cluster with one Monitor and one Manager before adding hosts and services. The cephadm documentation points to Rook as the alternative way to deploy Ceph when Kubernetes is the target platform, so the choice is between cephadm for a Ceph-first estate and Rook for a Kubernetes-first one. cephadm was introduced in v15.2.0 (Octopus) and does not support older releases, so any existing cluster running a release before v15.2.0 must be upgraded through a supported path before it can be managed with cephadm. Confirm the running release and the planned Ceph release with the Ceph release notes before fixing the design.
Kubernetes storage with Rook and Ceph: the build order and the check at each step
This sequence takes a team from a written storage design to a Ceph cluster that serves block, file and object storage to Kubernetes workloads. Each step closes with a check, so no later step starts on an unverified foundation. Confirm exact commands against the Rook and Ceph documentation for your chosen releases before you run them.
Confirm the storage design and failure domains
The platform team and the storage owner agree on in-cluster or external Ceph, the replication or erasure-coding layout, the CRUSH failure domains (host, rack or site) and which services are needed: block, file or object. The design is written down before any drive is touched. Check the failure domain map against the physical rack and site layout, and confirm that the planned pool layout survives loss of one domain the design claims to tolerate. Someone other than the author should sign off on it.
Prepare nodes, network and drives
Each storage node gets a dedicated drive per OSD, a bonded network link spread across two switches, and memory sized for peak load and recovery activity, as the Ceph hardware recommendations advise. Benchmark candidate hardware before purchase, because no two clusters behave alike. Wipe the drives and confirm the kernel sees them on every node, and label Kubernetes nodes so Rook placement rules select the right hosts. Done when each node lists the expected raw devices with lsblk and both bond members report as up.
Deploy the Rook operator
Install the Rook operator into its own namespace using the chart or manifests from the Rook release you have chosen. Confirm that the Ceph release that Rook targets is one that Ceph still maintains, using the Ceph active releases page. The operator is healthy when its pod reports Running and its logs show no CRD or RBAC errors. Check this with kubectl get pods in the operator namespace, and confirm the exact install steps in the current Rook documentation.
Deploy Ceph in-cluster or import external details
For an in-cluster design, apply the CephCluster resource with monitors spread across failure domains and OSDs bound to the declared devices. For an external Ceph cluster, run the import procedure from the Rook documentation to bring in monitor endpoints and scoped keys, and avoid administrator-level credentials. Check the CephCluster resource: in-cluster it should report a healthy state, and external it should show as connected. Then run ceph status through the Rook toolbox and confirm monitors are in quorum and every OSD is up and in.
Configure block, file and object services
Create a CephBlockPool and a StorageClass for RBD volumes, a CephFilesystem if shared file access is needed, and a CephObjectStore for the S3-compatible endpoint. Add Object Gateway authentication such as Keystone or LDAP only where your identity design requires it. Check block storage by creating a test PersistentVolumeClaim and confirming it binds. Check object storage by creating a bucket, writing an object and reading it back. Check file storage by mounting the filesystem in a test pod and confirming a successful write and read.
Enable messenger v2 secure mode where required
Where the design requires encryption of Ceph daemon traffic in transit, enable messenger v2 secure mode as described in the Ceph msgr2 documentation. Confirm first that every daemon and client in the cluster supports the setting. Secure mode adds CPU overhead, so run a throughput benchmark before and after the change. Check that monitors advertise v2 addresses on port 3300, that all daemons rejoin with no connection errors, and that the benchmark completes without retries.
Validate before workloads move
Run a bound test claim for each storage class, then write a known file and read it back with matching checksums. Restore a test object from a backup or snapshot to confirm the recovery path works. In a maintenance window, take one OSD or one host out of service and confirm that Ceph returns to a healthy state with data intact. Workloads move only after the restored test object matches its original checksum and ceph status reports healthy again.
Sizing Ceph and setting failure domains that match your node layout
Sizing Ceph is a measurement exercise. The Ceph hardware recommendations say to benchmark before buying because no two clusters behave alike, and to size memory for peak load. For a Kubernetes deployment, the sizing work should start from the workloads that will consume storage: their I/O pattern, throughput needs, object counts and expectations during a node loss, when recovery traffic competes with client traffic. Ask DeployOpen for a sizing workshop and a benchmark plan in the written scope, and confirm who supplies the test hardware, the test data and the acceptance criteria. A quote built only from a capacity spreadsheet should prompt questions about how the numbers were measured.
Drive layout is the first design decision that changes both cost and failure behavior. Ceph recommends a dedicated drive per OSD, so a failed device affects one OSD. For HDD-based OSDs, the write-ahead log and the RocksDB database (WAL and DB) can move to shared SSDs, which helps latency for metadata-heavy work but concentrates risk: losing a shared SSD takes down every OSD that depends on it. The hardware recommendations also advise enterprise-class SSDs for metadata. Decide the layout per node pool, and confirm how your Rook CephCluster device selection and device classes will place each pool on the intended media. Check that nodes in the same pool present matching devices, and that network links are bonded across two switches.
Failure domains tell CRUSH where copies or chunks may live. Map host, rack and zone to the topology labels your Kubernetes nodes carry, so the CRUSH hierarchy reflects the physical layout you operate. In cloud environments, zone labels are usually set by the provider. On bare metal or in your own data centers, rack labels typically come from your inventory process and must be applied when each node joins the cluster. Rook can read topology labels into the CRUSH map, but confirm the exact label keys and the Rook and Ceph versions in scope. Then test the result: drain one node and one rack in a non-production cluster and verify that placement and recovery behave as the map predicts.
The choice between replication and erasure coding depends on how many failure domains you have. A replicated pool needs enough distinct domains to hold every copy, and an erasure-coded pool needs enough distinct domains for every data and coding chunk. When the domain count is too low, placement rules either cannot be satisfied or place more than one copy in the same domain, which removes the protection the layout was meant to provide. Plan the domain count first, then choose the pool type for each workload. Ask for a written check of which pools keep data available after losing one host, one rack or one zone, and record that check as an acceptance test.
How Ceph services map to Kubernetes storage types
This table maps each storage need to the Kubernetes interface, the Ceph service behind it, the identity source and the data owner. Each row is marked to be confirmed in the Rook documentation for the release you choose, so treat the entries as a starting design rather than final configuration. Use it to decide which Ceph service each workload type should use before you write StorageClass or ObjectBucketClaim manifests.
| Storage need | Kubernetes interface and access mode | Ceph service and identity source | Data owner and confirmation |
|---|---|---|---|
| Block volumes for single-pod databases | Block StorageClass backed by RBD, ReadWriteOnce access | RADOS block device service; access controlled by Ceph cephx keys managed by Rook | Application team owns data; confirm the StorageClass parameters and CSI driver options in the Rook docs for your release |
| Shared file storage for multiple pods | CephFS-backed StorageClass, ReadWriteMany access | CephFS metadata and data pools; cephx client identities per filesystem; confirm mount and subvolume options in the Rook docs | Platform or application team owns shares; confirm ReadWriteMany support and quota settings for your release |
| S3-compatible buckets for applications | ObjectBucketClaim or external bucket access through the Ceph Object Gateway (RGW), S3 and Swift APIs | RGW users and access keys, with identity from Keystone, LDAP, or Ceph-native users; confirm the chosen source in the Rook docs | Data owner per bucket is set by the application team; confirm bucket provisioning and claim options in the Rook docs |
| Federated object access through OpenStack Keystone | RGW endpoint consumed by workloads using Keystone-issued tokens | Keystone as the identity source for RGW; confirm the Keystone integration settings and required roles in the Ceph RGW docs | Identity team owns the Keystone project and role mapping; confirm the RGW authentication options for your release |
| Directory-backed object access through LDAP | RGW endpoint used by applications with directory-based credentials | LDAP as the identity source for RGW; confirm the bind, search and token options in the Ceph RGW docs | Directory team owns user groups and bind accounts; confirm LDAP configuration against your directory schema |
| Multi-factor and policy-controlled object access | RGW endpoint with MFA and Open Policy Agent checks, plus AWS STS, IAM and SNS APIs where needed | RGW MFA, OPA integration and STS, IAM and SNS API support; confirm each option and its prerequisites in the Ceph RGW docs | Security team owns policy rules; application team owns bucket policies; confirm which APIs your release exposes |
Securing Kubernetes storage with Rook and Ceph
When Rook runs Ceph inside your cluster, storage traffic, Kubernetes Secrets and administrative access share one trust boundary. The cards below cover the controls to configure, the privileges Rook needs and the checks to run against the CIS Kubernetes Benchmark. You configure each control in your own environment and verify it against your own requirements.
Messenger v2 secure mode
Ceph's messenger v2 protocol listens on port 3300 and offers a secure mode that encrypts all post-authentication traffic with AES-GCM and adds integrity checks. Decide whether every daemon and client path in your deployment must use it, and confirm how the setting is applied in the Ceph version your Rook release deploys. Test that existing clients can still connect before you enforce it, and record which connections remain unencrypted during migration.
Public and cluster network isolation
Ceph separates client-facing traffic on the public network from replication and recovery traffic on the cluster network. Place these on separate VLANs or subnets, limit which hosts can reach the monitors on port 3300 and the OSD ports, and enforce Kubernetes NetworkPolicy where your CNI supports it. Confirm with your network team that the address ranges match what Rook is configured to use.
RBAC and the Rook namespace
Rook's operator, custom resources and generated Secrets live in the namespace you choose, commonly rook-ceph. Restrict who can read Secrets there, because Ceph keyrings and admin credentials can be exposed through them. Give storage administrators scoped roles for CephCluster and pool resources, keep cluster-admin for a small break-glass group, and review role bindings on a set schedule. Check which service accounts can list or watch Secrets in that namespace.
Pod Security Standards for Rook components
Pod Security Admission enforces the Privileged, Baseline or Restricted standard per namespace, and Rook's OSD and node-level components may need elevated access to disks and host paths. Confirm the exact privileges each Rook component requests in your Rook version, then document the namespace exception and the reason for each privilege. Keep application workloads outside the Rook namespace on a stricter level, and limit the exception to what your review shows is required.
Secrets encryption at rest in etcd
Kubernetes stores Secrets unencrypted in etcd by default, and Rook keeps Ceph keys and credentials in Secrets. Configure encryption at rest on the API server with a key management approach your security team approves, then confirm that existing Secrets are rewritten after the change. Apply the same protection to etcd snapshots, because a backup containing plaintext Secrets exposes the same data.
Dashboard SSO and RGW access
The Ceph Dashboard supports single sign-on through SAML 2.0, and through OAuth2 when cephadm runs the Management Gateway and OAuth2 Proxy services. Rook-managed clusters do not use cephadm by default, so confirm which option applies to your deployment before you design the login flow. For the RGW object gateway, confirm the authentication backend you need, such as Keystone, LDAP or multi-factor authentication, and document a process for rotating S3 access keys and removing those of departed users.
Common Kubernetes storage with Rook and Ceph failures and their fixes
This table lists failures that appear most often when Rook-managed Ceph provides storage to Kubernetes, with the likely cause, the fix and the status or log to check first. Read each row as a starting point for diagnosis, then confirm the cause against your own cluster state before changing anything. The main takeaway is that most storage incidents trace back to placement, device, network or version decisions made before go-live.
| Failure | Likely cause and fix | Check first |
|---|---|---|
| PVCs stuck in Pending | StorageClass name, pool or parameters do not match the CephCluster, or the CSI provisioner cannot reach the cluster. Correct the StorageClass to the right pool and clusterID, then recreate the claim. | kubectl describe pvc, events on the StorageClass and csi-rbdplugin-provisioner logs |
| PVCs stuck in Pending with a failure domain error | Too few OSDs or hosts exist in the failure domain for the pool's replica or erasure-code size. Add hosts or OSDs, or choose a failure domain that the topology can satisfy. | ceph osd tree, ceph status and the CRUSH rule used by the pool |
| PVCs stuck in Pending on topology-aware classes | Nodes lack the topology labels that the CRUSH map or StorageClass expects, so no OSD or host is eligible. Label nodes consistently and restart the operator reconciliation. | Node labels compared with the CephCluster failure domain setting |
| OSD down on a drive | The device has the wrong device class (for example HDD marked as SSD), or the OSD was built on a device shared with another DB or WAL in a way the layout does not allow. Correct the device class or redeploy the OSD with a documented DB device layout. | ceph osd tree for class, ceph health detail and the OSD pod logs |
| Slow writes and high latency | Undersized network links or a single switch path between OSD hosts. Bond links across two switches, check MTU consistency and measure throughput between nodes before tuning Ceph. | Ceph OSD latency counters, host network error counters and iperf-style tests between OSD nodes |
| Operator fails after a Kubernetes upgrade | The Rook operator release does not support the Kubernetes minor version now running, or CRDs and RBAC were not updated. Check the Rook support matrix and upgrade the operator and CRDs before the cluster. | Operator pod logs, CRD versions and the Rook compatibility documentation for your release |
| Ceph health warnings after a node drain | Drains removed OSDs or monitors faster than recovery could finish, or PodDisruptionBudgets did not hold. Drain one node at a time, wait for PGs to return to active and clean, and confirm monitor quorum before the next drain. | ceph status, ceph health detail and the number of degraded or backfilling PGs |
| External cluster connection failure | Wrong client key or user capabilities, or monitor ports unreachable from Kubernetes nodes. Regenerate the keys with the minimum capabilities required, and confirm that TCP 3300 and 6789 are open from every worker node. | Operator and CSI logs for authentication errors, and connectivity tests from a worker node to each monitor address |
Testing failure domains and upgrades before production traffic
Run the verification plan before any application data lands on the cluster. It has three parts: failure tests that match the failure domains you designed, backup and restore tests for block and object data, and an upgrade rehearsal across Rook, Ceph and Kubernetes. Each test should have a written expected result and a recorded actual result, so the outcome can be reviewed by your platform, security and application owners. Start with the failure domains from your sizing work. Drain a Kubernetes node that hosts OSD pods, take an entire rack or zone offline, fail a single OSD, and stop a single Ceph Monitor. For each test, confirm that the CRUSH rules place replicas or erasure-coded chunks across the domain you planned, then capture placement group states, degraded and undersized counts, client I/O latency and whether the cluster returns to HEALTH_OK.
Recovery timing is the measurement most often skipped. Record how long the cluster takes to re-establish full redundancy after each failure, and how long rebalancing takes when a drained node or OSD rejoins. Compare those timings against the recovery objectives your application owners have documented, and check that recovery traffic does not starve client I/O during business hours. Set the acceptance thresholds in the written scope rather than taking them from this page, because they depend on your data volume, network and workload. Release timing is a planning input for the same plan. The Ceph releases page lists Squid with an estimated end of life of October 31, 2026, and Tentacle with an estimated end of life of June 1, 2027. Choose the release you deploy with its support window in mind, and confirm the current dates with the Ceph project before you commit, since these are estimates.
Test backup and restore for both data types. For block data, test restoring a PersistentVolumeClaim from a CSI volume snapshot and, separately, from an exported RBD image, then confirm the restored volume mounts and the application reads expected records. For object data, test restoring a bucket through the Ceph Object Gateway, including object versions and any lifecycle or access policy you rely on. Restore into a separate namespace or cluster so a failed test does not damage production data. Confirm which backup mechanisms your chosen Rook and Ceph versions support for your topology, and where backup copies are stored. Document who can start a restore, how long the restore is expected to take, and how you verify integrity after it completes.
Rehearse the upgrade on a staging cluster built from the same manifests and Helm values as production. Follow the Kubernetes version skew policy: upgrade kube-apiserver one minor version at a time first, then kube-controller-manager and kube-scheduler, then kubelets. The policy allows kubelets to be up to three minor versions older than kube-apiserver, so plan the node upgrade sequence within that window. Upgrade the Rook operator and Ceph daemons in the order the Rook release notes specify, and confirm that the Ceph version is supported by that Rook release before you start. After each step, rerun the failure and restore tests, and record the stop points and rollback steps in the runbook. Ask your implementation partner to show these results in the written scope before you approve the production change window.
Questions teams ask before they run Rook and Ceph
How much does running Rook and Ceph cost?
Ceph and Rook carry no project license fee, so the cost of running them sits in other places. Expect spending on hardware (drives, memory, CPUs and nodes sized for your workloads), network (the Ceph hardware guidance recommends bonding links across two switches), staff time for design, build and operations, testing and benchmarking before purchase, and ongoing operations such as monitoring, capacity growth, upgrades and incident response. Hardware and network are mostly one-time purchases, with more bought at each capacity expansion. Staff time and operations recur. Costs rise with capacity targets, the number of storage services and the number of failure domains you need. Ask any quote to break out hardware, network, implementation labor, testing and recurring support separately, and to state which items are one-time and which recur.
How long does a Rook and Ceph implementation take?
Duration depends on the scope, hardware lead times, the number of storage services you need and the topology you choose, such as Ceph running inside the Kubernetes cluster or connecting to an external Ceph cluster. Each choice changes the node count, the network design and the testing effort. Hardware lead times are often the longest single item, so the date you order equipment can set the start date. Two inputs from your side set the pace: access to clusters, networks and accounts, and a change window for cutover and maintenance. You also make the hardware decisions and provide workload profiles that describe block, file and object demand. Fixed-scope engagements record the timeline in writing before work starts. Ask for a timeline tied to these inputs, along with the assumptions behind it.
What does the client need to provide for a Rook and Ceph project?
Plan to supply five things. First, access: accounts, Kubernetes clusters, network paths and administrative credentials for every environment in scope. Second, hardware decisions: node counts, drive types, memory and network equipment, or approval of a design proposed during scoping. Third, workload profiles: which applications need block, file or S3-compatible object storage, their expected capacity growth, read and write patterns, and how sensitive they are to latency. Fourth, a named decision owner who can sign off design and scope changes. Fifth, a change window for cutover, maintenance and any planned node restarts. Fixed-scope work agrees these inputs in writing before the build begins, so gaps show up while the scope is still open and can be priced before work starts.
Can an internal team run Rook and Ceph without outside help?
Yes, an internal team can run Rook and Ceph alone if it already has strong Kubernetes operations skills and at least one person who understands Ceph placement, recovery and upgrade behavior. Because the software carries no license fee, the decision is mostly about staff capacity and skills. The team has to handle drive and node failures, capacity planning, benchmarking before purchase, upgrades across Ceph releases and monitoring. Many teams bring in outside help for the design and first build, then run operations in-house. Before deciding, list who will be on call, how many people can cover an upgrade at the same time, and which parts of the design nobody on the team has built before. A DeployOpen engagement can cover design, build, hardening, backups, documentation and handover, with ongoing support available through a retainer.
Which hardware choices matter most for Rook and Ceph?
The biggest choices are drive layout, metadata storage, memory and networking. The Ceph hardware recommendations advise a dedicated drive per OSD. For HDD-based OSDs, they suggest moving the WAL and DB to shared SSDs. They also recommend enterprise-class SSDs for metadata and bonded network links across two switches. For memory, the guidance is to size for peak use rather than quiet periods, so plan for recovery and rebalancing load, not just normal traffic. Because no two clusters behave alike, the same guidance says to benchmark before buying. Illustratively, a team that buys hardware for average load may find that a single failed node triggers heavy recovery traffic that the network and memory cannot absorb. Confirm each choice against your own workload profiles and benchmark results before the purchase order is placed.
How should failure domains be planned for Ceph on Kubernetes?
Failure domains tell Ceph where to place replicas or erasure-coded chunks, so that losing a host, rack or other physical unit does not take data offline. Ceph uses CRUSH rules to express these domains. Start with your physical layout: how nodes map to racks, power feeds, network switches and any availability zones. Then decide which level must survive a single failure and which must survive a larger one. The answer also affects how many nodes you need, because a domain cannot be protected if too few of them exist to hold the replicas. Check that the Rook version you deploy supports the CRUSH layout you plan. Also decide how recovery traffic will behave when a whole domain is lost, and test that scenario before production. Ask your design reviewer to walk through a failure of each domain on paper first.
What does a fixed-scope Rook and Ceph engagement include?
A fixed-scope engagement starts with written scope, deliverables and a timeline agreed before work begins. Typical areas include design (in-cluster Ceph or an external Ceph cluster, failure domains, sizing and network layout), hardware guidance based on your workload profiles and benchmarks, build and integration with your Kubernetes clusters, hardening, backup and recovery planning, an upgrade plan for future Ceph and Rook releases, and documentation and handover to your team. Items outside the agreed scope need a written change before they are added. Ongoing support, upgrades and continuous testing are available as retainers, so you can keep a team on call after handover. The exact contents of your engagement depend on the scope you approve, so review the deliverables list closely with the quote.
How DeployOpen scopes, builds and supports Kubernetes storage with Rook and Ceph
DeployOpen starts by working out whether Ceph should run inside your Kubernetes clusters through Rook or connect to a Ceph cluster you already operate. The written scope then covers the storage classes, failure domains, hardware, network design and operational runbooks that your platform team will own after handover.
How we scope
- Confirm whether Rook-managed Ceph runs inside the cluster or connects to an external Ceph cluster, with the reasons recorded.
- Define the block, file and object storage services the workloads need, and the Kubernetes storage classes that expose them.
- Agree sizing inputs, failure domains and hardware assumptions in the written scope, with benchmarking planned before purchase.
How we build
- Deploy Rook and Ceph into your own cloud accounts, data centers or air-gapped networks, with images sourced from your approved registry.
- Configure OSD layout, WAL and DB placement, network bonding and dedicated storage networks according to the agreed design.
- Test failure scenarios such as node loss, OSD replacement and monitor quorum loss, and record the results against the acceptance criteria.
How we hand over
- Document the cluster design, CRUSH rules, storage classes, access controls and encryption settings in a handover pack.
- Provide runbooks for routine tasks such as adding OSDs, replacing failed drives, pausing recovery and restoring backups.
- Walk your platform and storage engineers through operations and agree the open items list before the engagement closes.
How we support
- Offer ongoing support, Ceph upgrade planning and continuous testing as a retainer, with scope agreed in writing.
- Plan Ceph release upgrades against the currently maintained release lines, confirming end-of-life dates with the Ceph project before each change.
- Review capacity growth, dashboard access, Object Gateway authentication and messenger settings at intervals agreed in your support scope.
