Ceph for cloud and hosting providers

You add capacity in small, planned batches and let the cluster rebalance under controlled recovery limits. Each new OSD receives data that CRUSH maps to it, so every addition creates backfill traffic that competes with client I/O. The main levers are batch size, the number of OSDs brought in at once, and the recovery and backfill settings your team tunes before the change. Before the first production change, benchmark the expected backfill rate on a test pool with the real hardware, then compare client latency for your VM and bucket workloads while backfill runs. Confirm the current defaults and throttle options for your release in the Ceph documentation, since they change between versions.

Size memory for peak use during recovery, not for the quiet periods between incidents. The Ceph hardware recommendations advise sizing for peak use rather than calm periods. Recovery and backfill raise OSD memory consumption and CPU use at the same time that client I/O continues, so a host that looks comfortable under normal load can swap or stall during a rebuild after a drive or host failure. Drivers include the number of OSDs per host, the OSD memory target you configure, placement group counts, and how many failures your design must absorb at once. Ask your hardware vendor and your implementation partner to show the memory budget per OSD and the headroom left during a simulated recovery, and benchmark that case before you buy.

For HDD-based OSDs, the Ceph hardware recommendations advise moving the WAL and DB to shared SSDs, and this is usually the difference between acceptable and poor latency for VM disks. Each SSD holds the metadata and journal for several OSDs, so the number of OSDs per SSD, the SSD endurance and the failure domain of the SSD all become design decisions. Losing a shared SSD takes down every OSD that depends on it, which makes the rebuild larger than a single drive failure. Plan the ratio of OSDs to SSDs, use enterprise-class SSDs, and benchmark random write latency with your workload profile. Confirm the exact sizing guidance for your release in the hardware recommendations.

Rebalancing impact depends on how much data moves, how fast the cluster is allowed to move it, and how sensitive your tenant workloads are to latency. Adding a host moves a share of placement groups onto new OSDs, and that share is set by CRUSH weights and the failure domain layout, so the amount of data moved is predictable before you start. The timing is not. Drivers include the recovery throttle, the client priority your team configures, network bandwidth between hosts, and how busy the cluster is when the change begins. Run the change in a maintenance window, watch slow requests and per-pool latency, and measure the backfill rate on your own hardware. Ask for a written rollback plan that stops backfill if latency crosses thresholds your tenants have agreed to.

Benchmark the workloads your tenants run, on the hardware you plan to buy, before committing to a design. The Ceph hardware recommendations state that no two clusters are alike, so published numbers are a starting point rather than a forecast. Useful tests cover small random block I/O for VM disks, large sequential object writes for RGW, backfill throughput at the planned batch size, and latency while recovery runs. Record the results against your SLA targets and the failure scenarios your design must survive. Confirm which benchmarking tools your team will use and keep the results with the cluster documentation so that later expansions can be compared against the same baseline.

The cost of scaling has three parts: hardware, the engineering time to design and validate the change, and ongoing operations. Hardware is a one-time purchase per expansion, but SSD replacement, spares and power recur. Engineering time for design, benchmarking and the first expansion is usually one-time, while runbooks, monitoring tuning and capacity reviews recur. A fixed-scope implementation can cover the design, the benchmark plan, the first controlled expansion and the runbook. Ongoing capacity reviews and upgrades can be handled under a retainer. Ask any partner to separate hardware, one-time engineering and recurring support in the quote, and to state which of your own staff must be available during each change window.

A team can scale a Ceph cluster in-house if it has operators who understand CRUSH, placement groups, OSD memory, recovery tuning and cephadm host management. The work is less about the commands and more about judging risk: deciding when to add capacity, how large each batch can safely be, and when to stop a backfill. Before taking this on, check that your team can read cluster health output, has tested drive and host replacement, and has a benchmark baseline. If any of these are missing, a partner can plan the first expansions and train the team alongside the work. Confirm the skill gaps during a short review of your current runbooks and monitoring.

The largest cost drivers are cluster size, hardware choices, network design and how much of the work your own staff can take over. A quote is paid for in components: design and architecture, hardware and network validation, deployment with cephadm, multi-tenant RGW and block configuration, testing, documentation and handover. Some costs are one-time, such as the build and initial benchmarking. Others recur, such as support retainers, upgrades and periodic testing. Ask for each component to be priced separately, with the assumptions behind hardware counts, the number of tenants and the integrations listed in writing. The Ceph hardware recommendations advise benchmarking before buying because no two clusters are alike, so ask whether the quote includes that benchmarking step.

Duration depends on the scope agreed in writing, not on a fixed template. The main factors are the number of hosts and OSDs, whether the network and hardware are already in place, how many tenant models and integrations must be built, and how much acceptance testing the provider requires before handover. Hardware lead times and change windows in your data centers often set the critical path more than software work does. Ask for a schedule with named milestones, dependencies on your hardware delivery, and the test evidence each milestone produces. Because most DeployOpen engagements are fixed-scope, timelines are agreed in writing before work starts, and the schedule should state what would move it.

You supply the physical and network foundation, the people who will run the cluster, and the time to change production systems safely. In practice that means servers that match the hardware recommendations, a dedicated drive per OSD, enterprise-class SSDs for metadata and WAL and DB on shared SSDs for HDD-based OSDs, and bonded network links across two switches. You also supply rack space, power, cooling, IP addressing, DNS, certificates, access to cloud or hypervisor hosts that will consume the storage, and approved change windows. Confirm in writing who owns each item, the delivery dates and the staff named for design reviews, runbooks and acceptance tests.

Yes, many teams run Ceph in-house, and the question is whether they can staff the skills it needs every day. Running a multi-tenant cluster involves cephadm host management, placement group and capacity planning, health warning triage, drive replacement, RGW account and quota administration, Dashboard access control and upgrade rehearsals. A partner can design the cluster, build it, document it and train your staff, then hand over a runbook. Decide which of those activities stay with your team and which go to a retainer. Ask any partner to show which tasks their staff will perform, which your staff will perform, and how knowledge transfer is verified.

Upgrades are planned as a sequence of tested steps, and the release you target determines the path. The Ceph releases page lists Tentacle (20.2.x) and Squid (19.2.x) as the actively maintained releases, with estimated end-of-life dates for each, so confirm which release a build will target and when it reaches end of life. cephadm is the deployment tool for supported upgrades, and it is not available for releases older than v15.2.0 (Octopus). Ask the provider to rehearse the upgrade in a staging cluster, define the rollback criteria, schedule maintenance windows with your tenants and verify client compatibility for RBD and RGW before production. Request the upgrade plan as a written deliverable, not a verbal assurance.

A useful quote lists the scope, deliverables, assumptions and exclusions in writing before work starts. It should name the Ceph release, the cluster topology, the hardware and network design that the design assumes, the RGW and RBD tenancy model, the integrations with hypervisors or orchestration platforms, and the security configuration including client authentication, encryption in transit and Dashboard access. It should state the acceptance tests, the documentation and training included, and what happens to change requests. Separate the one-time build from any recurring support, upgrade or testing retainer. Ask for the assumptions about your hardware and staff, because those assumptions are what most often change the price and schedule.

Ceph's source is licensed under LGPL-2.1 or LGPL-3, and the project does not sell a paid edition, so there is no software license fee to budget for the core product. Your costs are the hardware, network, staff time and any services you buy. Confirm with the provider whether a quote includes any third-party components, such as hypervisor or Kubernetes tooling, that carry their own terms. DeployOpen is not an official partner or reseller of Ceph, so ask any provider to state its relationship to the project in writing.