CephFS snapshot schedules and retention

How CephFS snapshot schedules work and when teams need them

Teams need CephFS snapshot schedules when people working in shared directories delete or overwrite files and want to recover an earlier version without restoring from a backup. The snap_schedule manager module answers that need by creating read-only point-in-time snapshots of a directory on a repeat interval, then pruning older snapshots according to a retention policy. Intervals can be set in hours, days, weeks, months or years, and retention policies such as 24 hourly and 4 weekly tell the module how many snapshots of each frequency to keep. The result is a rolling set of recent versions that users or administrators can browse and copy from, with no restore job and no manual snapshot commands.

A snapshot is not a backup on its own. It lives in the same Ceph cluster as the data it protects, so a cluster failure, a storage pool loss, a deleted filesystem or a ransomware event that reaches the cluster can remove the snapshots along with the live files. Plan a separate copy outside the failure domain, such as a second cluster or an off-site target, and treat the schedule as the fast path for accidental changes. Before you rely on either mechanism, confirm with your own runbook how long a restore from each copy takes and who is authorized to perform it. Check the scheduling documentation for the module's current options and the exact retention syntax for your release before you write any policy, since those details are the reference for your build.

Schedule and retention options and when each fits

This table compares snapshot interval units and retention counts against common recovery needs, from accidental deletes to ransomware. Read each row as a recovery window: the retention count sets how far back you can go, and the per-directory cap sets how many snapshots can exist at once. Plan the schedule and the counts together, because a generous schedule can hit the cap before it reaches the window you need.

Schedule option	What it covers	Planning considerations
Hourly snapshots with a short retention count	Recent user file mistakes, such as overwritten or deleted documents discovered the same day	Fine-grained recovery points consume the per-directory budget quickly; confirm the count with the cap before enabling
Daily snapshots with a multi-week count	Application rollbacks after a bad release or data load within the past few weeks	Usually the best balance for most shared directories; confirm the intended rollback window with the application owner
Weekly snapshots with a multi-month count	Longer-running errors, slow corruption or a recovery point before a known change window	Low snapshot volume, but recovery points are coarse; pair with a daily tier for recent work
Monthly or yearly snapshots	Retention for audit, legal hold or long-term reference copies	Confirm the retention period with your compliance owner; the module's counts must fit under the cap alongside shorter tiers
Mixed tiers on one directory	Layered recovery: recent hourly points, daily points for weeks, weekly points for months	Each tier adds to the total snapshots kept per directory, so the sum of counts must stay below the cap
Ransomware recovery window	Restoring data to a point before encryption spread, which may be days before detection	Snapshots are not an immutable backup on their own; confirm the window against your detection times and store copies off the file system
Shallow schedule on very busy trees	Limited protection for large or frequently changing directories	Check snapshot counts per directory and consider splitting trees so each has its own schedule and cap

Why the 100 snapshots per directory limit shapes your retention plan

The snapshot limit is counted per directory, not per schedule, so your retention plan has to be built from the directory outward. The MDS setting mds_max_snaps_per_dir defaults to 100, which means the default maximum kept is 99 snapshots per directory. The snap_schedule module also keeps at most 50 snapshots per directory on its own. Those two numbers are the ceiling you design against, and they apply to every schedule that touches the same tree.

Retention counts add up faster than most teams expect. Illustrative example: a directory with an hourly schedule retaining 24 snapshots, a daily schedule retaining 7 and a weekly schedule retaining 4 already holds 35 snapshots if every schedule is pointed at it, and any extra schedule or manual snapshot consumes the same budget. Because the module caps the count at 50, a schedule with a short interval and a long retention window will prune older snapshots sooner than the policy text suggests. Ask your team to write down, for each directory, every schedule and manual snapshot pattern that applies to it, then total the retained counts before you approve the plan.

Before going live, confirm the values on your own cluster with the Ceph version you run, since the limit and the module behavior are the things to check rather than assume. Review the snap_schedule retention specs against the per-directory total, decide which directories need a shorter hourly window, and document who may add manual snapshots. Plan for the case where a directory fills up, because snapshot creation then fails until older snapshots are removed, and the Ceph snapshot scheduling documentation is the natural place to confirm the exact pruning behavior.

How to set up snapshot schedules on a CephFS directory

This sequence takes a CephFS directory from a healthy file system to a scheduled snapshot policy that you have checked against actual snapshot output. Each step ends with a check, so you can fix a problem before retention or pruning hides it.

Confirm file system and MDS health
Start from a workstation with cluster administrator credentials. Run ceph fs status to confirm that the CephFS file system is online, that an active metadata server is serving it, and that standby daemons are present. Then run ceph health detail and resolve any MDS or metadata warnings before you add a schedule. Confirm that the directory you plan to protect exists and that a client mount can reach it. This step is done when the file system reports active, the health output has no MDS warnings, and you have recorded the file system name and directory path for the commands that follow.
Enable the snap_schedule manager module
Enable the module once per cluster with ceph mgr module enable snap_schedule. Snapshot schedules run inside the manager daemon, so confirm with ceph -s that an active manager exists before you continue. Then run ceph mgr module ls and check that snap_schedule appears among the enabled modules. Next, run ceph fs snap-schedule status / to confirm the subcommands respond. This step is done when the module is listed as enabled and the status command returns output without an error.
Define the path and interval
Create the schedule with ceph fs snap-schedule add <path> <interval>, where the path is relative to the file system root, such as /projects, and the interval uses hours, days, weeks, months or years. Illustrative example: ceph fs snap-schedule add /projects 1h takes an hourly snapshot. If the cluster holds more than one file system, add the --fs option so the schedule attaches to the correct one. Review the flags for your release in the Ceph snapshot scheduling documentation. This step is done when ceph fs snap-schedule list <path> shows the path and the interval you entered.
Set the retention policy
Add retention rules with ceph fs snap-schedule retention add <path> <count><unit>, using unit letters such as h for hourly, d for daily, w for weekly, m for monthly and y for yearly. The documented example keeps 24 hourly and 4 weekly snapshots. Keep the total within the module's limit of 50 snapshots per directory, and within the default ceiling set by mds_max_snaps_per_dir, which is 100 with a maximum kept of 99. Run ceph fs snap-schedule retention list <path> to confirm the rules. This step is done when the listed retention matches the policy you documented.
Verify the first snapshots appear
Wait for the first scheduled run to pass, then check the directory from a client mount with ls -la <mount>/<path>/.snap. The snapshot names should carry the timestamp format used by the module. Run ceph fs snap-schedule status <path> to confirm that the schedule is active and shows a next run time. If no snapshot appears after the scheduled time, check manager logs and the schedule status before changing anything else. This step is done when at least one snapshot is visible under .snap and the status shows the schedule running normally.
Confirm pruning behaves as expected
After the snapshot count passes your retention limits, check that the oldest snapshots are removed. Count entries with ls <mount>/<path>/.snap | wc -l and compare the result with the retention rules you set. Confirm the active ceiling with ceph config get mds mds_max_snaps_per_dir. Record the baseline count, the schedule, and the retention rules in your runbook, and add an alert for MDS health warnings. This step is done when the snapshot count holds steady at the level your retention policy predicts and the pruning log or status output confirms removal of older entries.

Access, deletion and operational risks to control

Snapshot schedules create recovery points automatically, but they also create new ways to lose them. Access rights decide who can remove snapshots, retention rules decide when older points disappear, and scheduling depends on a module that can stop without an obvious failure. The cards below cover the controls to set and verify before you rely on the schedule.

Client capabilities for snapshot actions

Snapshot creation and deletion in CephFS are governed by the MDS authorization capabilities granted to each client, not by the schedule itself. A client with write access to a directory may be able to create or remove snapshots under it, depending on how its caps are scoped. Review each client key against the directories it mounts, and confirm that only the administrative identity that manages schedules can remove snapshots that serve as recovery points. Test this with a non-admin key before go-live.

Listing snapshots with the right identity

Teams often need to confirm which recovery points exist before a restore, and that visibility should not require broader rights than the restore requires. Decide which users can list snapshots in a directory and which can only read data from them. Snapshot directories are visible to clients with read access to the parent tree, so confirm that the visibility matches your data classification policy for protected directories.

Retention pruning that removes recovery points

A retention spec such as 24 hourly and 4 weekly keeps a fixed set of snapshots, and older snapshots are pruned as new ones arrive. If a schedule runs more often than expected, or a retention count is set too low for your restore window, the oldest recovery point can disappear sooner than the business assumes. Write the restore window in days or weeks, then calculate the retention spec that meets it and check the result after the first full cycle.

Snapshot limits and pruning order

CephFS enforces a per-directory snapshot limit through mds_max_snaps_per_dir, and the snap_schedule module applies its own cap. When a directory approaches either limit, new snapshots can fail or older ones can be pruned in an order that does not match your expectations. Confirm the effective limits in your release, alert on snapshot counts per directory, and avoid stacking many schedules on one large tree.

Detecting schedules that stop running

A schedule that stops silently is more dangerous than one that fails loudly, because the last good snapshot ages without anyone noticing. Monitor the age of the newest snapshot per scheduled directory, alert when it exceeds the expected interval plus a margin, and check Ceph health for MDS or manager issues that could block scheduling. Include a restore test in routine operations so the alert is tied to a real recovery path.

Keeping snapshots outside the primary failure domain

CephFS snapshots live in the same cluster as the data they protect, so a failed pool, a cluster-wide fault or an administrative error can affect both. Snapshots help with accidental deletion but do not replace a separate copy. Decide which copies must sit in another failure domain, such as a second cluster through CephFS snapshot mirroring, and document how asynchronous replication affects the recovery point you can promise. Encryption of the underlying OSDs is a separate control covered in the encryption at rest topic.

How snapshot retention differs by workload

Home directories change slowly and are often the first place users look when they delete or overwrite a file by mistake, so they usually get the longest retention and the most frequent recent recovery points. Because each user's data is small relative to the whole tree, the usual approach is one snapshot schedule on the top of the home tree with retention that favors recent days and a few weeks of history. Point-in-time recovery for individual users depends on how the directories are split, so plan the layout before the first schedule runs.

Interval: Hourly snapshots during working hours are a common starting point, with a daily snapshot for longer history. Confirm the interval against how often users actually lose work.
Retention: Keep a short run of hourly snapshots plus several daily and weekly copies, and check the total against the 50 snapshot cap in the module.
Layout: Schedule at the level of the home tree you want to restore from, so one schedule covers all users rather than one schedule per user.
Count control: Set the retention so the combined total stays under the per-directory limit of 100 snapshots, remembering the default effective maximum is 99.
Restore path: Test a restore from a user's .snap directory before go-live.

Before finalizing, confirm with users and the service desk how far back restores are requested in practice.

Limits to confirm in your environment

Snapshot schedules look simple until a retention spec meets a per-directory ceiling, a cluster upgrade or a quota. The cards below list the limits to confirm on your own cluster, against the snapshot scheduling docs, before you commit a retention policy to production directories.

Confirm the Ceph release in use

Confirm whether your cluster runs Squid (19.2.x) or Tentacle (20.2.x) and whether that release is still within its estimated end of life. Snapshot scheduling behavior and defaults should be checked against the documentation for the exact release you run, not assumed from the current docs. Record the version in your change plan so the limits you verify are the ones that apply to your cluster.

Check mds_max_snaps_per_dir

The default ceiling is 100 snapshots per directory, and the default maximum kept is 99. Read the current value on your cluster, since it may have been changed by an operator. Compare it with the sum of your retention counts for each directory, because a schedule that keeps more snapshots than the ceiling allows will fail to create new ones.

Plan around the module's 50 snapshot cap

The snap_schedule module keeps at most 50 snapshots per directory, which is lower than the MDS ceiling. Confirm this cap in the docs for your release before you design retention, because it limits the total of hourly, daily, weekly, monthly and yearly counts you can apply to one path. Where a directory needs longer history, plan for a separate policy or a snapshot mirror rather than a larger count.

Test behavior across upgrades

Schedules, retention and snapshot names should be tested before and after any move between Ceph releases. Run the upgrade in a staging cluster with a copy of your schedule definitions, then confirm that existing snapshots are still listed, pruned and counted the way you expect. Keep the upgrade plan and rollback steps with the schedule configuration.

Check clones and quotas

Snapshots that back clones or subvolume quotas may hold space or count against limits in ways that do not match your retention table. Confirm with a test directory how snapshots interact with clones, quota settings and the space they consume, and how the schedule handles snapshots it did not create. Document the result so capacity reviews do not rely on file counts alone.

Questions buyers ask about CephFS snapshot schedules

What drives the cost of setting up CephFS snapshot schedules?

The main drivers are how many directories need snapshots, how deep the retention policy goes, who will own the schedules after handover, and how much restore testing you want evidence for. Each directory with its own schedule and retention rule adds design and review work, while a single standard policy applied to a small tree is much simpler. Ask any partner to price these four items separately in the written scope so you can see which ones move the total. Hardware and licensing costs are not driven by this work, because Ceph's source is licensed under LGPL-2.1 or LGPL-3 with no paid edition from the project itself.

Is setting up snapshot schedules a one-time task or does it recur?

Configuring the schedules, retention specs and any restore runbooks is one-time work. Recurring work includes reviewing snapshot counts as directories grow, adjusting retention when business needs change, checking that schedules still run after Ceph upgrades, and rehearsing restores on a regular cadence. Snapshot retention also needs ongoing attention because the number of snapshots per directory is capped, so growth in a tree can quietly push a schedule toward its limits. Ask for the recurring items to be listed as a separate line in any quote so you can decide whether to keep them in-house.

How long does a CephFS snapshot schedule rollout take?

The duration depends mostly on your review cycles rather than on the technical steps themselves. Typical stages are agreeing the directory list and retention targets, reviewing the schedule design with your storage and security teams, applying the configuration in a test cluster, running restore tests, and then promoting the change to production with an approval. Each approval gate adds calendar time that is set by your change board, not by the module. Ask a partner to write the review gates into the timeline so that the schedule reflects your governance rather than an assumed pace.

What do our storage and security teams need to provide?

Storage engineers need access to the Ceph cluster and its CephFS file systems, the list of directories in scope, and the Ceph version in use, since the snap_schedule module is part of the manager tooling your cluster already runs. Security teams should define who may read, create or delete snapshots, how long data must be kept, and whether snapshot contents fall under retention or legal hold rules. Both teams should approve the restore test plan and confirm where snapshot activity is logged. Agree these inputs before work starts so the design does not need rework.

Can an internal team set up CephFS snapshot schedules without outside help?

Yes, a team with working Ceph administration skills can enable the module and define schedules from the official documentation. The harder parts are sizing retention against the per-directory snapshot limits, deciding ownership for schedules that multiple teams share, and proving that restores work under pressure. Internal teams often skip the restore test and the handover documentation, which is where most later problems appear. Consider outside help if your team lacks capacity to own those steps, or if you need an independent record of the design for compliance evidence. Confirm your Ceph release status with your vendor or the release page before you begin.

What are the main ways a snapshot schedule goes wrong?

The most common problems are retention numbers set without checking the snapshot cap per directory, so older snapshots are pruned earlier than expected or new snapshots fail to be created. Schedules set on parent directories can also overlap with child directories in ways that multiply the snapshot count. Restore procedures that were never tested often reveal missing permissions or unclear paths only during an incident. Snapshots are also not a backup on their own, because they live in the same cluster as the data they protect, so teams should confirm where a separate copy is kept. Ask any partner to show how each of these risks is checked during the design phase.

Which Ceph version and directory limits should we check before we design retention?

Start with the snapshot limit that applies to each directory. The default value of mds_max_snaps_per_dir is 100, so the default maximum kept is 99, and the snap_schedule module keeps at most 50 snapshots per directory. Your retention specs, such as 24 hourly and 4 weekly, must fit within these caps once you add the snapshots for every schedule that touches the same directory. Confirm your current cluster setting with your storage team, and check the Ceph snapshot scheduling documentation for how the module interacts with the setting in your release.

How to prove a snapshot can be restored

A snapshot schedule that runs is not the same as a restore that works. The module can report that every hourly and weekly snapshot was created and pruned on time, yet the files inside them may be unreadable, owned by the wrong user, or missing the directory you actually need. Prove restores with a recurring test plan, and treat a passed restore test as the only evidence that a retention policy protects your data.

Start by picking a small set of sample directories that represent your risk: one home or project tree with heavy churn, one shared directory with several owners, and one path that holds data you would be asked to produce in an incident. For each, confirm which snapshot is the newest one the schedule has written by listing the .snap directory of the path, then copy a few known files out of it into a scratch path outside the live tree. Record how long the copy took, compare file hashes against the live copy where the data should be unchanged, and check that numeric owner, group and mode bits survived the copy. Ownership checks catch problems that a simple file count will miss.

Repeat the test after any change that could affect snapshots: a new snap_schedule or retention spec, an upgrade of the cluster or the MDS, a change to mds_max_snaps_per_dir, new quotas, or a move of a directory that changes which snapshots it inherits. Confirm the restore test still matches the retention you documented, and keep the timing records so you can see whether restores are getting slower as the snapshot count grows. Ask your own team or your partner to define pass criteria in writing before the first run, so a restore counts as proven only when the defined checks succeed.

How DeployOpen scopes and supports CephFS snapshot schedules

DeployOpen plans, builds, documents and hands over CephFS snapshot schedules inside your own Ceph cluster, starting with the directories that need protection and the retention each one requires. Engagements are fixed-scope, with the schedule design agreed in writing before any configuration changes are made.

How we scope

We list the CephFS directories to protect and the recovery points each owner needs.
We confirm the snapshot interval, retention specs and expected snapshot count per directory.
We check the mds_max_snaps_per_dir setting and the 50 snapshot module cap against the plan.

How we build

We configure snap_schedule entries for each directory and test the schedule on a non-production path first.
We verify that pruning removes older snapshots as the retention specs predict.
We validate client access to snapshot directories for the users who need self-service restores.

How we hand over

We document each schedule, its retention rule, its owner and the directories it covers.
We provide restore runbooks showing how users and administrators find and recover a snapshot.
We walk your operations team through monitoring checks and the steps for changing a schedule.

How we support

Ongoing support, schedule reviews and upgrades are available as a retainer after handover.
We can review retention and snapshot counts when directory layouts or data volumes change.
Snapshot mirroring to a second cluster is scoped as a separate piece of work if you need it.