Securing cephadm SSH access for Ceph

Why securing cephadm SSH access matters for a production cluster

Anyone who can SSH into a cephadm host can effectively administer the cluster on that host. cephadm manages a Ceph cluster as containers, and it reaches the hosts that run those containers over SSH to deploy and manage daemons. That makes SSH access to these hosts an administrative control path, not just a login convenience. If you run a cephadm-managed cluster, the first question to answer is a simple one: who can reach every host, with which account, and through which key? Answer it for every node, including hosts that are only used as monitors, managers or clients, because each one holds a path into the cluster's management plane.

Hardening this access matters most when the cluster is in production, holds regulated or sensitive data, or is administered by more than one team. In those settings, a shared key, a root login or an unreviewed host list can turn one compromised workstation or contractor account into control over storage for every application that depends on it. Confirm with your own change and access policies who is authorized to add or remove hosts, who may hold the cluster SSH key, and how those decisions are recorded. The cephadm documentation describes the SSH and host management model, so start there and check each control against the release you run.

The rest of this page works through the controls that limit this exposure: non-root management users with sudo, protection of the cluster SSH key, restrictions on where admin keyrings are placed, and a deliberate choice of which hosts receive the _admin label. Each one reduces the number of places where a single credential grants full cluster control. Before changing anything, inventory your current hosts, SSH keys and keyring locations so you know what you are narrowing. Ask your team, or a partner you engage, to confirm the exact commands and settings for your Ceph release before applying them in production.

Access model options and when each fits

This table compares the main ways to give cephadm SSH access to cluster hosts, from the default root login to per-host restrictions. Read each row against your own host inventory and security policy, and confirm the exact user and sudo requirements in the cephadm documentation for your release before you commit to a model.

Access model	When it fits	What it protects against	What to confirm
Default root login over SSH	Lab or proof-of-concept clusters where speed matters more than audit trails and no production data is at stake	Little beyond keeping the key itself private; a leaked key gives full control of every host	Whether your release still defaults to root and what cephadm needs on each host
Dedicated non-root user with passwordless sudo	Most production clusters that need a clear audit trail of which account changed which host	Direct root login, and makes it possible to trace administrative actions to a named account	The exact sudo commands cephadm requires for your release and whether your hardening baseline accepts the sudo rule
Sudo rules scoped to required commands	Environments with a strict change-control model that want the cephadm account to run only what orchestration needs	Broad privilege escalation if the cephadm SSH key is compromised, limiting damage to the listed commands	That the command list matches the cephadm documentation for the release in use, since a too-narrow list can break deployments and upgrades
Per-host SSH and firewall restrictions	Clusters where management hosts sit on a separate network segment from OSD and monitor hosts	Use of the cephadm key from unexpected source addresses, and exposure of SSH on storage networks	Which source addresses the orchestrator connects from and how your bastion or jump host is placed
Limiting _admin label to specific hosts	Clusters where only a few hosts should hold cluster admin keyrings and client configuration	Admin credentials spreading to hosts that do not need them, which widens the blast radius of a host compromise	Which hosts carry the _admin label today and whether any workload hosts receive admin keyrings by default

Setting up a non-root cephadm management user

This sequence replaces direct root SSH from the cephadm manager with a dedicated, non-root user on every host. Each stage ends with a check, so you can confirm the cluster can still reach every host before you remove the old access path.

Create the management user on each host
On every host in the cluster, create one dedicated local account for cephadm, such as a cephadm user with a home directory and a locked password. Use the same name on all hosts so the cluster configuration stays consistent. Do this through your normal configuration management or by hand, and record the account in your host inventory. Confirm the account exists on each host with id cephadm and that you can log in to it from a bastion host using key-based SSH. Rollback: remove the account with userdel -r cephadm on any host where the check fails, since nothing in the cluster depends on it yet.
Grant sudo limited to cephadm needs
Give the management user passwordless sudo through a drop-in file in /etc/sudoers.d/, validated with visudo. Keep the rule as narrow as the cephadm documentation for your Ceph release allows, and confirm the exact command list there before you write it, because it can change between releases. Avoid a blanket ALL=(ALL) NOPASSWD: ALL rule unless your security team has approved it in writing. Verify the rule with sudo -l -U cephadm on each host and check that a test command runs without a password prompt. Rollback: delete the drop-in file, which returns the account to having no elevated access.
Generate the cluster SSH key
From a node with the admin keyring, generate the SSH key pair that cephadm will use for all host connections with ceph cephadm generate-key. Keep the private key inside the Ceph configuration store and do not copy it to a workstation or a file share. If your policy requires a specific key type or an external key store, confirm that option in the cephadm documentation for your release before you change the default. Verify the key exists by running ceph cephadm get-pub-key and confirming that it returns a public key string. Rollback: if the key is wrong, generate a new one and repeat the distribution step, because no hosts have trusted it yet.
Distribute the public key to every host
Append the public key from ceph cephadm get-pub-key to the authorized_keys file of the management user on each host. Set the file permissions so only that user can read and write it, and confirm that the directory ~/.ssh has the expected ownership. Add the key through your configuration management where possible so that new hosts receive it automatically. Verify each host by connecting with the cluster key as the management user from the manager node, using a command that returns without a password prompt. Rollback: remove the key line from authorized_keys on the affected host, which stops cephadm from connecting there until you fix it.
Set the cephadm SSH user
Tell cephadm to connect as the new account by running ceph cephadm set-user cephadm, replacing the name if you chose a different one. Before you change the setting, record the current value with ceph cephadm get-ssh-config so you have a known-good state to return to. Confirm the change by reading the configuration back and checking that the reported user matches what you created on the hosts. Rollback: run ceph cephadm set-user with the previous user, such as root if that was the setting, and confirm that cephadm reaches the hosts again before you go further.
Verify with a host check
Run ceph cephadm check-host <hostname> for every host in the cluster and read the output for each one. A healthy result shows that the host passes the cephadm prerequisite checks under the new user, including container runtime, time synchronization and the required commands. Also run ceph orch host ls to confirm every host shows as online and that no host reports an offline or connection error. Record the output as your baseline for this change. Rollback: if a check fails, return the user to the previous setting, fix the host, and rerun the check before you remove any older access path.

Protecting the cluster SSH key and admin keyrings

The SSH key that cephadm uses to reach managed hosts is a private key held by the orchestrator inside the cluster, and its public half is authorized for the cephadm user on every host it manages. If that private key leaks, anyone holding it can log in as that user on each host in the cluster. Because the user can start containers and typically has sudo rights, the leak is effectively control of every host, not just one service. Treat the key as a fleet-wide credential. Confirm in the cephadm documentation how your release generates, stores and replaces this key, which commands expose the public key, and what a rotation changes on each host before you plan one.

The client.admin keyring grants full administrative access to the cluster, and cephadm places a copy in /etc/ceph on every host that carries the _admin label. Keep that label on a small, tightly controlled set of hosts, such as hardened administration hosts, so that a compromised OSD or general compute host does not also hold full cluster credentials. Removing a label with ceph orch host label rm stops future distribution, but it may not delete a keyring file already on disk, so check each host directly and remove stale copies. Record which hosts were labeled at each change, because the set tends to grow quietly during incidents and maintenance.

Audit the admin footprint on a fixed schedule and after every host addition, replacement or staff change. Start with ceph orch host ls to see which hosts carry the _admin label, then list /etc/ceph on each host and confirm that ceph.client.admin.keyring appears only where the label says it should. Review ceph auth ls for other users with broad capabilities, since those can be as damaging as the admin keyring. If the SSH key or an admin keyring may have been exposed, plan a rotation in a maintenance window, and ask your vendor or partner to confirm the exact steps for your release, including how the old credentials are revoked on every host.

Security and operational risks to control

cephadm holds SSH access to every host in the cluster, so a weak SSH setup gives an attacker a path to orchestrate every daemon. The six risks below connect through one idea: limit who can reach each host, what they can run there, and whether anyone can see it happening. Each card names the control and the check a reviewer can repeat.

Overly broad sudo rules

The cephadm SSH user usually needs passwordless sudo to run the cephadm binary, and it is easy to grant full root shell access instead. Limit the sudoers entry to the specific commands cephadm needs on each host, and keep it in a file you manage. Check the sudoers configuration on a sample of hosts and confirm the user cannot run arbitrary commands with sudo.

Direct root SSH login

If the cluster SSH key is accepted for the root account, any compromise of that key yields root on every managed host without an audit trail of which administrator acted. Use a dedicated non-root account for cephadm and disable root login in the SSH daemon configuration. Confirm that the PermitRootLogin setting is off on every host and that the cephadm user is the only account the cluster key can use.

Admin keyrings on many hosts

A copy of the client.admin keyring on every host multiplies the places an attacker can read full cluster privileges. Keep the admin keyring only on hosts that run administrative tasks, and give other hosts only the keys they need. Periodically list which hosts hold ceph.client.admin.keyring and remove copies from hosts that no longer require them.

Unrotated SSH keys

The cluster SSH key that cephadm uses can outlive the staff who created it, and nobody may know which systems still trust it. Plan a rotation schedule, replace the key through cephadm, and remove the old public key from every host's authorized keys. Confirm the rotation by checking that hosts accept only the current key and that old keys are gone from each account.

Unmanaged _admin label assignments

Hosts labeled _admin receive copies of the admin keyring and configuration, so the label is a privilege decision and should be treated as one. Assign the label only to a small set of administrative hosts, and record who approved each assignment. Review the label list against your host inventory and remove it from any host that is no longer an administrative workstation or node.

Access logging gaps

If SSH sessions and sudo commands are not logged centrally, a misuse of the cephadm account leaves little evidence. Forward SSH and sudo logs from every host to a central store that administrators cannot quietly edit, and keep the retention period your compliance team requires. Test the pipeline by running a known sudo command on one host and confirming the event appears in the central store within your expected window.

Limiting _admin hosts by environment

In a small lab, the main risk is that a convenient setup becomes permanent. Labeling every host _admin is quick, and that habit often survives after the lab grows into a shared environment. Keep the label set narrow even here so the pattern carries forward. The cephadm documentation describes how labeled hosts receive the admin configuration, so check that behavior on your release before you rely on it.

Start with one administrative host. Apply the _admin label to the host where operators run ceph commands, and keep the label off OSD-only nodes.
Use a non-root user for SSH. Let cephadm connect as a dedicated account with sudo rights limited to what cephadm needs.
Keep the admin keyring on the labeled host only. Confirm with ls -l /etc/ceph that the file is readable by root only.
Plan the second admin host early. Record which host takes over if the first is rebuilt, so the lab does not relabel every node in a hurry.

Before relying on the lab's setup, run ceph orch host ls to list which hosts carry labels, and record the result in the lab notes.

In production, the _admin label should follow a documented list of hosts that operators actually use. Each labeled host holds cluster-wide administrative credentials, so the set should be the smallest one that still lets the team recover from a failed admin node. Place admin keyrings on that set only, and treat every other host as a consumer of its own scoped client keys.

Keep the set small and named. Record each _admin host in the change inventory with its owner and the reason it carries the label.
Maintain at least one spare. Choose a second labeled host in a different failure domain so administration survives a single host outage. Confirm the placement against your own CRUSH layout.
Separate admin hosts from workload hosts. Keep client workloads off hosts that hold the admin keyring, and limit interactive logins to named operators.
Check labels on every host change. Adding, replacing or reimaging a host should trigger a review of its labels and keyring placement, because the host's previous role can carry over.

Confirm with your change board who may add or remove the label, how each change is logged, and whether a review is required before a keyring moves to another host.

Limits to confirm in your environment

The cephadm SSH model depends on the release you run, the sudo rules on each host and the tools that record access. Confirm these against your own environment and the vendor documentation before you commit to a user model, because each one changes what the hardening steps can actually enforce.

Installed cephadm release

cephadm was introduced in v15.2.0 (Octopus) and does not support older releases, so the first check is the exact cephadm and cluster version you run. Confirm which non-root user model that version supports, since the options for the SSH user and sudo behavior can differ between releases. Plan your hardening against the release you actually operate, not the one in a blog post.

Sudo configuration on each host

A non-root cephadm user relies on passwordless sudo, and the exact rules on every host decide what that user can run. Check whether sudo is scoped to the commands cephadm needs or grants broad root access, and confirm that the same rules are present on every cluster host, including hosts added later. Record the sudoers files so that drift between hosts is visible during audits.

Cluster SSH key rotation

The key cephadm uses to reach hosts is a high-value credential, and its rotation procedure is part of the design, not an afterthought. Confirm how the key is replaced, how the new public key reaches every host, and how you verify that old keys were removed from authorized_keys. Test the rotation on a non-production cluster before you rely on it in an incident.

Admin keyring placement

Hosts labeled _admin receive the admin keyring, which grants broad cluster control, so the number of those hosts should be a deliberate decision. Confirm which hosts hold admin keyrings, who can log in to them, and whether those hosts sit in a separate management network. Review the placement after every host addition or removal.

Identity and host access logging

Your SSH logins, sudo commands and cephadm actions are only useful if they reach a central store you control. Confirm that your identity provider or PAM stack issues the SSH logins, that sudo logs are forwarded off the host, and that cephadm operations appear in the cluster logs. Decide who reviews these records and how long they are kept, based on your own retention policy.

Questions teams ask before securing cephadm SSH access

What does securing cephadm SSH access cost, and what drives the price?

Cost depends on how many hosts the cluster has, how many admin accounts and automation paths already touch those hosts, and how much of the existing access model has to change. Most of the work is inventory and sequencing: finding every place the cluster SSH key lives, confirming which hosts hold admin keyrings, and changing sudo rules without interrupting orchestration. The one-time work is the design, the migration and the verification. The recurring part is reviewing access when staff or automation change. A DeployOpen quote follows from a written scope that names the host count, the existing access tooling and the change windows. Ask any provider to itemize discovery, design, change execution and handover separately.

How long does the work take in general terms?

Duration is set by the number of hosts, the change control your organization requires, and how many teams must approve access changes. Work usually runs in phases: discovery of current SSH users and keys, a design for a non-root management account with scoped sudo, a rollout on a non-production cluster, then production changes in agreed windows. Each phase waits on approvals and on a verification pass after every host group. Ask for the phases and the approval points in writing so you can see which steps depend on your teams rather than on the work itself.

What must our team provide before the work starts?

Plan to provide access to the cluster hosts and the cephadm admin nodes, a current host inventory, and a named owner for each host group. You also need a change window for each environment, a rollback owner who can approve reverting an access change, and a contact who knows which CI jobs or monitoring tools connect over SSH. Security or identity staff should confirm the approved key types and where keys may be stored. Without these owners, the design will stall at the first host that behaves differently from the rest.

Can our own platform team secure cephadm SSH access in-house?

Yes, if the team has Linux administration experience, understands how cephadm uses SSH to reach hosts, and can test changes on a non-production cluster before touching production. The steps themselves are not exotic. The risk comes from partial changes: a sudo rule that works for one host but not another, or a key removed before orchestration has a replacement. Teams often do this in-house for a small cluster and bring in a partner when the host count is large, when several environments must match, or when the change needs an independent review for an assessment.

How does securing cephadm SSH access differ from least-privilege cephx work?

SSH hardening controls who can log in to a host and what they can do there, as well as which key cephadm uses to manage the host. Least-privilege cephx work controls what each Ceph client may do inside the cluster, such as which pools a user can read or write. The two are separate layers and both are needed. A tightly scoped cephx user does not protect a host whose root SSH key is copied to every workstation, and a locked-down SSH setup does not limit a client application that holds a broad keyring. Plan the two reviews side by side, but do not treat one as covering the other.

Which parts of the cluster does the work touch, and what does it leave alone?

The work covers the SSH account that cephadm uses, the sudo configuration on each host, the location and permissions of the cluster SSH key, the hosts designated as _admin hosts, and where admin keyrings are placed on disk. It does not change Ceph client capabilities, dashboard accounts, or network encryption. Those belong to separate reviews. Ask the provider to list the files and host roles it will change so your change board can approve the scope before work begins.

How do we check that the controls are working after the change?

Verification should show three things: that cephadm still reaches every host with the new account and key, that only approved users can log in as administrators, and that admin keyrings exist only on the hosts you named as _admin hosts. Engineers should test each claim on each host group and record the results. Keep the evidence with your change records. An external auditor may later ask for this evidence, but DeployOpen prepares controls and records for assessments and does not issue audit reports or certifications.

How DeployOpen scopes and delivers cephadm SSH hardening

DeployOpen agrees the scope in writing before any change is made to a cluster. For cephadm SSH hardening, the written scope names every host cephadm manages, the account each host uses for SSH, which hosts are allowed to hold admin keyrings and _admin labels, how the cluster SSH key is generated, stored, rotated and revoked, and what evidence the client receives at the end. Most engagements are fixed-scope, so the scope document also states deliverables and timelines. Ask for the host inventory and the current SSH and sudo configuration as part of the quote, because the effort depends far more on the number of hosts and how many people already have root access than on the hardening steps themselves.

The work is delivered inside your own infrastructure. DeployOpen makes the changes through your cloud accounts, data centers or air-gapped networks, under your change control and with your staff able to observe, approve and roll back each step. Evidence for review typically includes the before and after host access configuration, the list of accounts and sudo rules, the location and ownership of the SSH key material, and the results of checks run against the live cluster. Ongoing review of access, key rotation and new hosts is available as a retainer. DeployOpen prepares this evidence for your auditors, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make those determinations, so confirm early which artifacts your assessor expects.

Securing cephadm SSH access