Ceph HEALTH_WARN and HEALTH_ERR checks explained
What the common health codes mean and how urgent each is: OSD, PG, monitor and BlueStore checks, which ones need action now and which can wait, and muting warnings safely.
Ceph operations and troubleshooting
Running Ceph day to day: health warnings, full OSDs, slow requests, scrub errors, replacing drives, adding capacity, monitoring and logging.
Recovering from nearfull and full OSDs in Ceph
What to do when writes stop: nearfull, backfillfull and full ratios, temporarily raising ratios, rebalancing, deleting data safely and adding capacity before the next incident.
Troubleshooting Ceph slow ops
Tracing slow requests to a cause: overloaded or failing drives, network problems, recovery load and bugs, and the commands that show which OSDs are behind.
Repairing inconsistent placement groups and scrub errors
Handling PG_DAMAGED and scrub errors: finding the inconsistent objects, deciding when to repair, checking the underlying drive and avoiding repair loops.
Replacing a failed OSD drive in a cephadm cluster
Swapping a dead drive: removing the OSD with --replace, keeping the OSD ID, zapping and redeploying through the service spec, and watching recovery.
Adding hosts and OSDs to a Ceph cluster without hurting clients
Expanding capacity smoothly: adding hosts with cephadm, letting the balancer move data in phases, recovery profiles and timing expansion before nearfull.
Ceph monitoring with Prometheus, Grafana and Alertmanager
Using the monitoring stack cephadm deploys: dashboards, alert routing to webhooks, custom alert rules, and integrating with an existing Prometheus.
Centralized Ceph logging with Loki and Alloy
Collecting daemon logs from every host: enabling the Loki and Alloy services in cephadm, retention, and shipping logs to an existing log platform instead.
Fixing Ceph monitor clock skew
Why monitors need clocks within 0.05 seconds, chrony and NTP setup on Ceph hosts, and what else breaks when time drifts.
Fixing BLUESTORE_SPILLOVER warnings
When the DB device fills and metadata spills to HDD: confirming spillover, compaction, expanding or migrating the DB device and sizing DB devices for new OSDs.
