What losing the FreeIPA CA renewal server means for your certificates
When the FreeIPA server that renews certificates and publishes the certificate revocation list (CRL) is gone, your logins may keep working while your certificate lifecycle quietly stops. The FreeIPA backup documentation states that the first server handles certificate renewal and CRL publication by default, so this loss hits the certificate authority (CA) even when Kerberos and LDAP look healthy. Service certificates that come due, sub-CA signing and revocation checks all depend on that role, so the problem belongs on your certificate calendar before it appears in a user ticket.
The core question is whether another CA replica can take over that role. Check which surviving servers run the Dogtag CA, which one holds the renewal role in your topology, and whether each surviving CA replica has a healthy replication agreement with the rest of the deployment. That answer leads to three decisions: promote a surviving CA replica to renewal master, rebuild a replacement server and re-enroll it into the topology, or restore from a backup taken while the lost server was still authoritative. Each path has different prerequisites, and the choice depends on what survived, not on what you would prefer.
Treat the decision as a documented change, not a quick fix. Confirm the current state with your own inventory and replication checks before changing any role, and ask your implementation partner or internal team to write down the agreed path, the owner of each step and the rollback point. Restoring an outdated backup into an active replica topology can break replication, so the restore option needs the most scrutiny. Whichever path you take, plan to verify that certificates are renewing and the CRL is being published from the new renewal server before you consider the incident closed.
Recovery paths and when each one fits
This table compares four routes for replacing a lost CA renewal server: promoting a surviving CA replica, installing a new CA host, restoring a backup into a new environment, and the case with neither. Read each row for its precondition, main risk and what to confirm first, because the first question is always whether a working CA replica still exists in each datacenter.
| Recovery path | Precondition | Main risk | What to confirm |
|---|---|---|---|
| Promote an existing CA replica | A surviving replica runs the PKI role and holds a current copy of the CA data | Other replicas may still point at the lost server for certificate renewal and CRL publication | Which replicas run PKI, that each site has one, and that the chosen replica is healthy and replicating |
| Install a replacement CA on a new host | At least one CA replica survives, or the CA keys and data are available from a valid backup | Adding a CA role to a topology with stale agreements can leave renewal and CRL tasks split or broken | Which agreements reference the lost server, the replication agreement limit per replica, and the new host's name and DNS records |
| Restore from backup into a new environment | A usable backup exists and the outage affects every server holding the data | Restoring an outdated backup into an active replica topology can break replication, and the default first server handles renewal | The backup date against the last certificate issuance, whether the restore target is isolated, and the sequence for re-initializing replicas |
| No surviving CA replica and no usable backup | No PKI-capable replica remains and no backup can restore the CA data | Certificates issued by the lost CA may not be renewable through FreeIPA, and the CA signing material may be unrecoverable | Whether any copy of the CA key material exists elsewhere, and whether your auditor and service owners accept a new trust chain |
| Plan the topology before the loss | Each datacenter has at least one replica running PKI, with two to three replicas per datacenter recommended | Without a second PKI replica per site, every recovery path depends on a single surviving server | How many PKI replicas each site runs today, and where backups are stored and tested |
Risks that turn a CA promotion into a wider outage
Promoting another CA replica is rarely the hard part. The damage usually comes from what surrounds it: an old backup restored into a live topology, dead servers still listed in agreements, certificates nearing renewal during the gap, and a single server carrying the renewal role. Each risk below has a symptom you can spot early and a check to run before you change anything.
Restoring an outdated backup
If an older backup is restored into a topology that is still replicating, the restored server can push stale data to the replicas that never went down. The docs warn that this can break replication, so the restore step is where a recovery turns into a second outage. Before any restore, confirm which backup you have, its date, and whether the other servers already hold newer data. Use restore only into a planned rebuild, never into a live topology.
Stale replication agreements
When a server disappears without being removed from the topology, its replication agreements stay in place on the surviving servers. Replication errors, failed topology checks and warnings about a missing peer are the usual signs. Before promoting a replacement CA, list every agreement from each remaining server and confirm which ones point at the lost host. Remove only the agreements tied to that server, then verify the topology is consistent again.
Certificates and CRLs during the gap
While the renewal server is gone, certificates that are close to expiry will not be renewed, and certificate revocation lists may not be republished on schedule. Services that depend on those certificates can start failing without any obvious change on the servers themselves. Build a list of certificates with their expiry dates and the service that uses each one, and confirm which host publishes the CRL before you declare recovery complete.
One server holding the renewal role
Renewal and CRL publication default to the first server in the deployment. If that is the only CA-capable server, losing it stops all certificate lifecycle work until a replacement is in place, and a promotion done in a hurry can leave no second CA at all. Check which servers run the CA role, how many there are per site, and whether a replacement path has been tested. Plan for at least two CA-capable replicas before the next incident.
Promoting a surviving CA replica and cleaning up the lost server
This sequence moves certificate renewal and CRL publication to a surviving replica, removes the lost server from the replication topology, and confirms that the CA works before anything else depends on it.
Confirm the lost server is gone for good
Before changing anything, the platform owner confirms that the lost host is unreachable from every surviving server and that it will not return with its old data. Check network reachability, hypervisor or cloud console state, and whether anyone holds a copy of its disk. A server that comes back later with stale data can re-enter replication and cause conflicts, so record who signed off on the decision and when. Done when the decision is written down and every surviving server reports the same unreachable host.
Identify the surviving replica that runs the CA
Run the topology and CA state checks from a surviving server to see which replicas still run the Dogtag certificate authority and which one holds the renewal role. Confirm the status of the CA on each candidate, its replication health, and its time synchronization. Compare the results with your records, since the lost server may have held more than one role. Done when you have one named surviving replica with a running CA that you intend to promote, and you have documented the roles the lost server held.
Move the renewal and CRL role to the chosen replica
Using the FreeIPA command-line tools on the chosen replica, assign the CA renewal master role and the CRL publication role to it, as the FreeIPA documentation describes for your version. The first server normally handles certificate renewal and CRL publication by default, so this step matters if that server was the one lost. Confirm exact commands against the current FreeIPA documentation for your version before running them. Done when the role query returns the chosen replica as the renewal master and the CRL publisher.
Remove the lost server's agreements and topology entries
From a surviving server, delete the replication agreements that point to the lost host and remove its server entry from the topology. Work through each segment that referenced it, and check that no remaining replica still lists it as a partner. Keep the replica count and agreement limits in the deployment recommendations in mind, because removing a server can leave a site with too few replicas or too few agreements. Done when the topology listing shows only live servers and every surviving replica reports healthy agreements to its remaining partners.
Check that replication is consistent after cleanup
Verify that the directory replicates normally between the surviving servers after the removals. Check replication status for each agreement, confirm that users and host entries are current on each replica, and review the server logs for errors that mention the removed host. Repeat this check after a short period of normal activity, not only right after the change. Done when replication shows no errors and a test change made on one replica appears on the others.
Confirm certificate renewal works on the new role holder
Check that the promoted replica can renew certificates. List the certificates tracked by the CA, confirm that their tracking requests are present, and review the renewal status and the CA's health output. Where a certificate is close to renewal, confirm with your team how the renewal will be triggered and observed, rather than waiting for a deadline to find a fault. Done when the CA reports healthy status and tracked certificates show the expected renewal state.
Confirm CRL publication works
Check that the promoted replica publishes the certificate revocation list and that clients and services can fetch it from the published location. Compare the published list with the CA's own record, and confirm the publication location is reachable from the networks that depend on it. Record the outcome in your change log, along with the roles now assigned and the removed host. Done when a client on a relevant network retrieves a current CRL from the promoted replica and the change record names the new role holder.
When recovery is impossible and what to do instead
Recovery in place is off the table when no CA replica survives and no backup can be restored cleanly. At that point the Dogtag CA signing key, its certificate state and the renewal and CRL publication role cannot be rebuilt from the remaining systems, and trying to force a restore can make things worse. The FreeIPA backup documentation warns that restoring an outdated backup into an active replica topology can break replication, so an attempt to bring back a stale CA on top of surviving servers is a risk to avoid. Before choosing a path, confirm with the FreeIPA backup and restore documentation and with your own records exactly what still exists: which backups are intact, whether any server still holds a copy of the CA data, and which certificates each service and host depends on.
Once the CA is confirmed unrecoverable, the realistic options narrow to three. The first is rebuilding the identity domain in a new environment, which means installing a fresh FreeIPA topology and recreating users, groups, hosts and policies from the best available source, such as an earlier backup or an LDAP or FreeIPA export. Migration tooling can move objects between deployments, but it does not restore a lost CA key, so the new CA will be a new trust anchor. The second is reissuing service certificates for web servers, LDAP clients, Kerberos services and anything else that relied on the old CA, which is usually the largest effort. The third is planning client re-enrollment, since enrolled Linux hosts may need to be re-enrolled and their SSSD state reset to trust the new realm and CA.
The right choice depends on what data is available, how many services and hosts depend on the old certificates and whether your organization can accept a change of trust anchor. Ask the vendor or your support channel to confirm the supported procedure for your FreeIPA version before any destructive step, and document the decision with a dated inventory of certificates, hosts and open dependencies. Build the plan with a rollback point, test the rebuilt topology in an isolated network, and treat the change as an identity event that needs sign-off from the owners of every system that trusts the domain.
How the recovery differs by topology
In a single datacenter, the recovery path depends on whether another server in the same site runs the CA. The deployment recommendations advise two to three replicas in each datacenter and at least one replica running PKI or DNS when those services are used, so a well-built site should have a second CA-capable server to take over renewal. If it does, the work is promotion, cleanup and replacement inside one network.
- Promotion target: Confirm that a surviving replica runs the CA role before you change anything. A replica without the CA cannot take over renewal.
- Agreement cleanup: Remove the lost server's replication agreements once the survivors are confirmed healthy, so stale entries do not linger in the topology.
- Replacement install: Rebuild a server in the same site and restore the CA role so the site returns to its intended replica count.
- Concentrated risk: If only one server held the CA, the site had a single point of failure, and recovery falls back to backups and a rebuild.
Check the CA role on every remaining replica and compare the replica count against the recommendations before you start any change.
With a CA replica in each site, the lost server is one of several certificate-capable hosts, so the main risk shifts to the timing of promotion and to the site that remains. Each replica should carry no more than four replication agreements, so the cleanup and any replacement must be planned against a current topology map. Cross-site links are the part most likely to be overlooked during a rushed recovery.
- Promotion in the surviving site: Choose a CA replica in a healthy site and confirm it holds the renewal and CRL publication duties.
- Agreement limits: Count the surviving replicas' agreements against the four-agreement guidance before adding a replacement, since the new server also needs connections.
- Cross-site links: Verify that the remaining sites still replicate to each other after the lost server's agreements are removed.
- Replacement placement: Install the new CA replica in the affected site so each datacenter keeps a CA-capable server.
- Backup timing: Confirm a recent backup exists for each site, because restoring an outdated backup into an active topology can break replication.
Verify replication status in both sites after each step, and record which server holds the renewal role at every point in the process.
A single-server deployment has no replica to promote, so the recovery question is whether a usable backup and a rebuild path exist. The deployment recommendations describe multi-server layouts with dedicated servers, so a single host sits outside that guidance and carries identity, Kerberos, certificate and DNS duties together. Recovery usually means restoring onto a new host, and the outcome depends on the backups and on the certificates already issued.
- No promotion path: No surviving CA can take over renewal, so the CA role must return from a backup or be rebuilt.
- Backup dependence: Confirm the backup type, its age and whether it captures the CA data, since full and online data-only backups hold different content.
- Host and name reuse: Confirm the replacement can use the same host name and realm, because clients and issued certificates depend on them.
- Certificate exposure: Identify which issued certificates would need replacement if the CA cannot be restored, and plan for the client and service impact.
- Impossible case: If no usable backup or CA key material survives, the outcome is a new environment, with clients re-enrolled and certificates reissued.
Test the backup restore on an isolated host before you decide whether recovery is possible.
Security and operational considerations during recovery
A CA renewal server loss is also a security event. The controls below keep the rebuild from widening access, exposing CA material or breaking logins while the topology is changed. Work through them alongside the promotion and cleanup steps.
Restrict who can change CA roles
Promoting a replica to hold the CA renewal role and cleaning up the old server are administrative changes with direct effect on certificate issuance. Limit these actions to a short list of named administrators for the duration of the incident, and remove temporary access afterward. Confirm which accounts can run topology and CA role commands on each server before you start, so that an emergency does not become a general permissions review.
Protect backups that contain CA material
A full backup taken from a FreeIPA server holds CA keys and certificate data, not just directory entries. Store archives with the same access restrictions as the live CA, encrypt them at rest, and record who copied them and where they went. Do not leave recovery copies on the replacement host or on shared scratch storage while the rebuild runs.
Log every administrative change
Record each role promotion, agreement removal, server deletion and reinstallation with the operator, time and command used. Keep these records outside the FreeIPA servers that are being rebuilt, so the evidence survives if a server is lost again. These logs give auditors and your own team a sequence of events to reconcile against the directory afterward.
Verify client authentication after each step
Before and after each change, test Kerberos login, SSSD lookups and certificate-based services from a representative set of enrolled clients. Include one client per site and one service that depends on a renewed certificate. A passing result on the new CA server alone does not show that clients still trust the realm, so record the test results with the change log.
Confirm the FreeIPA version is current
FreeIPA 4.13.4 fixed two CVEs, including LDAP access control flaws that could let unauthenticated clients obtain credentials, and hardened authentication checks across the API. Before you rebuild, confirm which version runs on every surviving server and plan to bring them to a current release. Do not restore a server onto an older version than the rest of the topology.
Questions teams ask before recovering a lost CA renewal server
Is promoting another CA replica much less work than rebuilding the CA from scratch?
Usually yes, when a healthy replica already runs the certificate authority and holds current data. Promotion moves the renewal and CRL publication role to a server that is already in the topology, then removes the lost server's agreements so replication stays clean. A rebuild means installing a new server, re-creating the CA and re-enrolling every role that depended on the old one, which adds installation, verification and client-facing checks. The gap grows when the surviving replica is out of date, has no CA role, or the lost server held roles that no other server can take over. Confirm which servers hold the CA role and whether each one is current before you compare the two paths.
What drives the time and cost of recovering a lost CA renewal server?
The main drivers are the number of servers in the topology, how many services and clients depend on the lost CA, the age and health of the remaining replicas, and whether the environment has custom certificate profiles or external CA chains. Each cost component is labor: assessment of the surviving topology, the promotion or rebuild itself, agreement cleanup, verification of replication and certificates, and documentation. Some work recurs after recovery, such as renewing service certificates that were reissued and updating backup procedures. Ask any partner for a quote that lists each component separately, names which servers are in scope and states what evidence of a working topology you will receive at the end.
What does the recovery work need from our team?
Expect to supply access and decisions, not hands-on configuration alone. Your team should provide administrative access to the remaining FreeIPA servers, the current topology layout, the list of services and clients that rely on certificates from the CA, and the names of people who can approve changes during a maintenance window. You also need someone who knows the DNS, network and change-control rules for your environment. Confirm early who owns decisions about certificates that expire during the work, because clients may need to trust a changed CA chain.
Can our own team recover a lost CA renewal server without outside help?
Yes, if your team has run FreeIPA replication and PKI before and can read the topology and certificate state accurately. The documented steps for promotion and agreement cleanup are well defined, but errors in the order of operations can break replication across the whole deployment. Teams with little recent FreeIPA experience usually benefit from a review of their plan before they touch production. Decide in advance whether you want an internal runbook only, a second pair of eyes or a fixed-scope engagement covering the full recovery. Written scope lets you compare those options on equal terms.
Is a backup enough to recover the CA after losing the renewal server?
A backup is necessary but not always sufficient. An FreeIPA backup can restore data, but the backup documentation warns that the first server handles certificate renewal and CRL publication by default, and that restoring an outdated backup into an active replica topology can break replication. If the backup is older than the current replicas, restoring it may create conflicts rather than fix them. Check when each backup was taken, which server it came from, whether it includes the CA data you need and whether a restore procedure has been tested in a separate environment. A backup you have never restored is an assumption, not a recovery plan.
What happens to clients and services while the CA recovery is underway?
Clients keep working as long as their existing certificates remain valid and the services that check them can still reach a working KDC and certificate chain. Problems appear when certificates need renewal during the work, when CRLs are not published, or when DNS and Kerberos lookups point to a server being removed. Plan for the window by listing which certificates expire soon, which clients rely on the CA chain and which services must be checked after each step. Ask for a verification plan that covers client enrollment, Kerberos logins and certificate issuance before and after the change.
When is recovery impossible, and what should we do then?
Recovery is not possible in the normal sense when no surviving server holds a usable copy of the CA data, or when the surviving copies are too old or inconsistent to merge safely. In that case the realistic options are a rebuild of the identity environment from a known-good backup or snapshot, or a new deployment with migration of users and policies. Both options change which certificates clients trust, so they require a planned cutover. Before a loss happens, confirm which servers hold the CA role, test restores and keep at least one current replica with the CA role in each datacenter.
Limits to confirm in your own environment
Recovery steps for a lost CA renewal server depend on facts about your own deployment that no general guide can settle. The five checks below connect the version you run, the surviving topology, your backups, your certificate dependencies and any signing hardware, and each one can change which recovery path is safe.
Confirm the running version and renewal role
Identify the exact FreeIPA version on every server with the package manager, then confirm which server currently holds the CA renewal role. The backup documentation states that the first server handles certificate renewal and CRL publication by default, but your installation may have moved that role. Compare what your version's documentation says about the renewal role with what your servers actually report before you promote anything.
Verify a surviving PKI replica exists
Recovery by promotion needs another server that runs the Dogtag CA and has a working replication agreement. The deployment recommendations advise at least one replica per datacenter running PKI when it is used, so check whether your topology meets that. Record which replicas run the CA, which agreements connect them, and whether replication is currently healthy, because a stale or broken agreement will block a clean promotion.
Test that a backup is current and restorable
Find the newest ipa-backup archive, note when it was taken and which servers it covers, and confirm it can be read on a test host before you depend on it. Restoring an outdated backup into an active replica topology can break replication, so the age of the archive determines whether it helps or harms. Treat an untested archive as unknown until a restore has been rehearsed.
Map certificates that depend on the lost CA
List every certificate issued by the FreeIPA CA, including web servers, service principals, VPN gateways and client certificates used for smart card or certificate-based login. Certificates issued by this CA may be trusted by systems outside FreeIPA, and those systems need to know about any change to the chain. Confirm which of these certificates must be reissued or replaced if the CA signing material has to be rebuilt, and who owns each one.
Identify any HSM or external signing setup
Determine whether the CA signing key sits in a hardware security module, a key store or an external signing service. If it does, recovery depends on access to that device or service, its credentials and its own backup procedure, none of which FreeIPA can restore. Ask the vendor or the team that operates the device what happens to signing keys if the host that runs FreeIPA is destroyed, and document the answer before an incident.
How DeployOpen scopes and delivers CA recovery
DeployOpen treats CA renewal server loss as a scoped recovery engagement inside your own infrastructure. We start by establishing which servers still hold the Dogtag CA and which replicas remain, then agree the recovery path in writing before any change is made to the topology.
How we scope
- Inventory every FreeIPA server, its roles and which one held certificate renewal and CRL publication
- Confirm whether a CA replica remains and whether a usable backup of the data exists
- Agree in writing whether the work is promotion, agreement cleanup, replacement install or assessment
- Record the decision points where recovery may be impossible and what evidence will show it
How we build
- Promote a surviving CA replica to renewal master following the documented procedure for your version
- Remove stale replication agreements and verify the remaining topology before adding any server
- Install and join a replacement server only after replication and certificate state are confirmed healthy
- Test certificate renewal and CRL publication on the rebuilt topology before closing the work
How we hand over
- Written runbook of the actions taken, commands run and the state of each server afterward
- Replication and certificate health results captured before and after each change
- Notes on which backups remain usable and which must be refreshed for the new topology
- Named owners and a schedule for the next renewal and backup checks in your environment
How we support
- Fixed-scope engagements are agreed in writing with deliverables and timelines before work starts
- Ongoing support, upgrades and continuous testing are available as retainers
- Confirm with us which certificate and CRL checks will run after handover and who acts on failures
- Independent auditors and certification bodies issue any audit reports; we prepare the evidence
