When expired certificates stop FreeIPA and where ipa-cert-fix fits
When certificates in the FreeIPA CA or in service certificate stores expire, the symptoms usually appear at the edges: users cannot log in, the Web UI shows certificate errors, or replication between servers stalls. ipa-cert-fix is one recovery path among several, and the right choice depends on which certificates expired and on which server holds them. FreeIPA combines 389 Directory Server, MIT Kerberos and the Dogtag certificate authority in one product, as described on the FreeIPA About page. Dogtag issues the certificates that services such as LDAP, the Web UI and certificate-based logins rely on, so one expired certificate can stop authentication even when the directory data is intact. Before you choose a path, list the certificates on each server and record which ones have passed their expiry date.
Renewal and certificate revocation list (CRL) publication matter as much as issuance. The backup documentation notes that the first server in a topology handles certificate renewal and CRL publication by default, so recovery work often starts there before other servers can follow. Confirm which server holds that role in your topology and check its logs for failed renewal attempts. Then take a full backup with ipa-backup on every server you plan to touch, and store the copies off those servers. The backup and restore guide also warns that restoring an outdated backup into an active replica topology can break replication, so a backup applied in the wrong place can turn a certificate problem into a directory problem.
Bring in outside help when the CA certificate itself has expired, when the CA chains to an external authority, when replicas disagree about the state of their certificates, or when no recent backup exists. Each of these changes the order of work and the evidence you need afterward, and an error can affect every login in the domain. Ask any partner for a written scope that names the servers, the certificates, the backup and rollback steps and the verification you will receive. DeployOpen plans, implements and supports FreeIPA inside your own infrastructure, but the recovery path should follow your own inventory of expired certificates and your own record of which server holds each role.
Which recovery path fits which symptom
This table compares four ways to recover FreeIPA when certificates have expired, showing the symptom each one addresses, where it runs, its main risk and why a team might pick it. Read it by matching your observed symptom to a row first, then checking the risk column before running anything. Confirm exact behavior against the documentation for your FreeIPA version before acting.
| Recovery path | Symptom it addresses | Where it runs and main risk | Why a team might choose it |
|---|---|---|---|
| ipa-cert-fix | Core IPA services fail after internal CA and server certificates expire, and the CA itself still works | Runs on the server being repaired. Main risk: applied without a verified backup or on the wrong server in a multi-server topology | Fastest in-place path when the internal CA chain is intact and the failure is limited to expired certificates |
| Restore from a current backup | Certificates, directory data or the CA are damaged, or in-place repair has already failed | Runs on a rebuilt or chosen server. Main risk: restoring an outdated backup into an active replica topology can break replication | Gives a known-good state when in-place repair is uncertain, provided the backup is recent and tested |
| Renew an externally signed CA certificate | The FreeIPA CA certificate was signed by a corporate or external CA and has expired or is about to | Runs on the CA-managed servers with a signed request from the external CA. Main risk: renewal cannot happen automatically, so a missed date stops issuance | Keeps the corporate trust chain intact when the root of trust sits outside FreeIPA |
| Renew individual service certificates | One web, LDAP or service certificate expired while the rest of IPA still runs | Runs on the affected service host through certificate tracking. Main risk: services on other hosts still hold old certificates and fail after a partial renewal | Narrowest change with the smallest blast radius when only one service certificate has lapsed |
Why backups come before any ipa-cert-fix run
Take a backup before you run anything else on an expired-certificate incident. FreeIPA provides ipa-backup with two modes: a full backup, which captures configuration, data and certificate material, and an online data-only backup, which can run while services stay up but does not replace a full backup for recovery. The backup you take before recovery is your rollback point. If ipa-cert-fix produces a result you did not expect, that backup is the only clean state you can return to, so confirm that it is a full backup and that it completed without errors before you touch certificates.
Do not restore an older backup into a live topology to fix the problem. FreeIPA's backup documentation warns that restoring an outdated backup into an active replica topology can break replication, because the restored server carries data that no longer matches its peers. Before any restore, check the age of the backup against the last change you made, list every server in the topology, and confirm which server holds the CA renewal and CRL role. Restores belong on a planned path that accounts for every replica, not on the first server you can reach.
Confirm the backup procedure and retention on your own servers before the incident starts. Check where backups are written, whether they are copied off the host, how long they are kept, and who can read them, since backups contain private keys and directory data. Run a test restore in a non-production environment if your team has not done one recently. The exact commands, options and retention periods depend on your FreeIPA version and environment, so verify them against your own runbook and the official backup documentation before relying on them.
Running ipa-cert-fix safely, step by step
This sequence takes a FreeIPA deployment with expired certificates from diagnosis through a backed-up, renewal-master-first repair to a verified state where the CA, LDAP, web and Kerberos services and replication all work again.
Identify expired certificates and their tracking state
The administrator on each FreeIPA server lists which certificates have expired or are close to expiry, along with the services and certmonger tracking requests that depend on them. Record the certificate subjects, expiry dates and the nicknames used by each server, so you can compare the state after the repair. Confirm that certmonger is running and that tracked requests show their current status. This step is complete when you have a written inventory of every expired certificate, the server it sits on and which services are down as a result. If the inventory is incomplete, stop here and confirm the list with the documentation for your installed version before changing anything.
Confirm which server is the renewal master
Identify the CA renewal master for the deployment, which is the server that handles certificate renewal and CRL publication by default. Query the topology from a working server and check the role assignment, because the fix must run there first. Note any server that is offline or unreachable, since it can complicate the repair later. This step is done when you have one named renewal master and have confirmed it is reachable over the network. If the role has been moved, or you cannot tell which server holds it, do not proceed until the current assignment is confirmed from the documentation for your version.
Take and verify a full backup
Create a full backup of the IPA data and the CA material on the renewal master, using the backup tool for the installed version. Copy the archive off the server to storage that the broken host cannot affect, and record the checksum. Restore the backup into a test environment or verify its contents with the documented listing or inspection method, so you know it is readable. This step is complete when a backup exists off-host and you have confirmed it can be read. Do not restore an older backup into a live replica topology as a shortcut, because the backup documentation warns that this can break replication.
Freeze changes to the topology
Announce a change freeze for the FreeIPA deployment and pause any work that adds, removes, renames or promotes servers, changes replication agreements, or edits CA configuration. Disable automated jobs that touch IPA servers, such as configuration management runs, until the repair is finished. Record the current replication agreements so you can restore the same layout if needed. This step is done when the team has confirmed the freeze in writing and the topology snapshot is saved. Any server that is still being added or removed should be completed or cancelled before the repair starts.
Run the fix on the renewal master first
Run ipa-cert-fix on the renewal master before any other server, using the exact command and options listed for the installed version. Follow the prompts carefully, keep the session log, and do not interrupt the process. Work out whether the tool needs a directory manager or administrator credential, and have it ready. The step is complete when the tool reports success and the expired certificates are shown with new validity dates. If the tool reports an error or leaves the server in a partial state, stop and consult the documentation and your backup before attempting a second run.
Verify CA, LDAP, HTTP and Kerberos health
On the renewal master, check that the Dogtag CA responds and that it issues and reports certificates correctly. Confirm that the directory service accepts LDAP connections over TLS, that the web UI loads with a valid certificate, and that Kerberos authentication succeeds for a test account. Check the service status of each component and review the logs for certificate or trust errors. This step is done when every component passes the checks and the test account can obtain a ticket. Any failure here should be resolved before you move to other servers.
Check replication and bring other servers in line
Verify replication from the renewal master to every replica, confirming that agreements are healthy and that changes made on one server reach the others. Apply the certificate updates to each replica according to the documented procedure for the installed version, and check each one for the same CA, LDAP, HTTP and Kerberos results. Confirm that clients trust the renewed CA chain. The step is complete when every server reports healthy services and replication shows no errors. Lift the change freeze only after this check passes, and record the final state for your audit file.
How recovery differs on the renewal master and other servers
The first server in a FreeIPA topology handles certificate renewal and CRL publication by default, according to the FreeIPA backup documentation. That makes it the first place to work during recovery, because the shared CA and service certificates it holds drive what the rest of the topology trusts. Confirm which server currently holds the renewal role on your deployment before you start, since the default can be changed and an assumption here can send work to the wrong machine.
- Confirm the role: Check which server is the renewal master in your topology and which CA replicas can take over if it is down.
- Take a backup first: Make a full backup of this server before running any repair, so you can return to the starting state.
- Run the repair here: Apply ipa-cert-fix on the renewal master first, then review which certificates were renewed and their new expiry dates.
- Watch the CRL: Confirm the CRL is being published again from this server, since the renewal master handles that role by default.
- Re-synchronize: Confirm the other servers pick up the renewed material before you touch them.
Before moving on, check that the renewal master shows healthy certificates for every service it hosts and that its replication agreements are working.
Other replicas that run the Dogtag CA hold their own copies of CA material and service certificates. They should not be repaired independently of the renewal master, because their state has to match the renewed material that the renewal master publishes. The safe approach is to verify their state after the renewal master is repaired, then bring them into line with the topology's current certificates.
- Check expiry first: Compare certificate expiry dates on each CA replica against the renewal master to see which ones are behind.
- Avoid parallel repairs: Do not run ipa-cert-fix on several CA replicas at once, because you may end up with mismatched certificates.
- Restore from the renewal master's state: Confirm the replica receives the renewed certificates through replication rather than by hand-editing files.
- Confirm CA health: Check that the CA service starts cleanly and answers requests after the update.
- Verify replication: Look for replication errors in the directory and CA logs after the repair window.
Confirm with the FreeIPA documentation for your release which steps apply to a replica holding CA material, because the exact sequence depends on your version and topology.
Replicas without a CA do not issue or renew CA-signed certificates, but they still hold service certificates for LDAP and HTTP and depend on the Kerberos and directory state that the renewal master maintains. Their recovery is mostly verification and resynchronization. Treat them as followers: confirm they reflect the repaired state and do not attempt a standalone CA repair.
- Check service certificates: Confirm the LDAP and HTTP certificates on the server are current and match what the renewal master now serves.
- Confirm the web UI and directory: Test that the web UI and directory answer on this host after the repair.
- Check Kerberos: Confirm the KDC on this replica issues tickets, since expired certificates can break PKINIT and related checks.
- Review the topology: Make sure this replica still has enough agreements to receive changes and has no stale state.
- Re-synchronize where needed: If the server remains out of step, follow the documented procedure for your release rather than copying files from another host by hand.
Confirm with your own topology which replicas hold service certificates for which services, since that mapping determines what you verify here.
Enrolled Linux clients use SSSD and trust the FreeIPA CA through the certificate store installed at enrollment. Their certificates are normally issued through certmonger, so a client with expired certificates may need its own renewal step after the servers are healthy. Clients are repaired last, and only after the server side is confirmed stable, because a client update before the servers are ready will fail or fetch the wrong material.
- Check trust material: Confirm the client has the current CA chain, since a renewed or replaced CA certificate has to reach every machine.
- Check tracked certificates: Review certmonger tracking on the client and confirm renewals are queued and succeed.
- Test authentication: Log in with Kerberos and SSSD after the server repair to confirm the client path works.
- Re-synchronize trust: Run the CA update on the client when the documented process for your release calls for it.
- Record exceptions: Note any client that cannot be reached or renewed, so you can plan its repair separately.
Confirm with the FreeIPA documentation for your release how client trust updates are applied in your environment, since the method depends on the version and the distribution in use.
What ipa-cert-fix cannot fix
ipa-cert-fix renews the certificates that FreeIPA's own Dogtag CA and services track on the server. The cards below cover what sits outside that scope, so you can plan the follow-up work before an expired certificate stops logins again.
Externally signed CA certificates
If your FreeIPA CA certificate was signed by a corporate root or an external CA, the tool cannot renew it on your behalf. That certificate needs a new certificate request sent to the external signer, then installation and distribution to every server and client. Before the outage, confirm which CA signed the IPA CA chain, who holds the signing authority, and how long the external process takes. Start that request early, because the external signer controls the timeline.
Damage from outdated backups
Restoring an old backup into a live replica topology can break replication, and FreeIPA's backup documentation warns about this risk. If your recovery plan involves ipa-restore, confirm the backup date, the server it came from, and whether other replicas hold newer data. Keep a fresh full backup before any change, and treat restore as a last resort with a tested runbook.
Clients with stale trust stores
Enrolled Linux clients keep a local copy of the CA trust store, so a renewed CA certificate may not be trusted until each client refreshes it. Expired or replaced trust material can make some hosts fail while others work. Inventory your enrolled hosts, plan a trust refresh on every machine, and confirm which hosts are unreachable from your change window, such as laptops and remote sites.
Services outside certmonger
Certificates that were issued by hand, or installed on services that certmonger does not track, will not renew through FreeIPA's recovery steps. Examples include appliances, network devices, and third-party web servers with copied key files. Build an inventory of every certificate in use, record its issuer and expiry, and assign an owner. Services that are not tracked need a manual request and installation plan.
Corrupted directory data
Expired certificates are a different fault from damage to the 389 Directory Server data behind FreeIPA. If users, groups, or policies are missing or inconsistent after the certificate work, the tool has not fixed the underlying problem. Check directory health on each replica, compare replication status, and confirm that a verified backup exists before you change any server state. Involve your directory specialists early when the symptoms do not match certificate errors.
Security and operational controls during the recovery window
Set the decision owner and the rollback point before anyone touches the renewal master. Recovery from expired certificates often happens under time pressure, and the first server in a FreeIPA topology is the one that handles certificate renewal and CRL publication by default, according to the FreeIPA backup documentation. Name one person who approves the change, one who executes it, and one who can stop it. Write down the backup you will restore from if the work goes wrong, and confirm that it is recent and taken from the correct server before the window opens.
Limit who can reach the renewal master during recovery. Use named administrator accounts with Kerberos authentication, close other sessions, and record who logged in and from where. Treat the directory data, the CA signing material and the backup archives as sensitive: keep the backup files on storage with restricted access, control who can copy them, and avoid moving key material through chat or ticket attachments. Confirm your backup and restore procedure in a test environment first, because the documentation warns that restoring an outdated backup into an active replica topology can break replication.
Capture evidence as you go. Keep the terminal session output, the exact commands run, the version of FreeIPA on each server, and the time of each step in the change record, and export the relevant FreeIPA and system logs once recovery is complete. Confirm with your own environment which logs your deployment retains and for how long. Compliance teams can use this record to show how the change was controlled, and an auditor will judge whether it is sufficient. DeployOpen prepares controls and evidence for compliance assessments, but auditors and certification bodies issue the audit reports and attestations.
Questions teams ask before recovering FreeIPA certificates
What does recovering FreeIPA from expired certificates cost in effort?
Recovery effort depends on how many servers are involved, how the certificates were issued, and whether a verified backup exists before any change is made. The main cost components are planning the outage window, taking and checking backups, running the repair on the renewal master, confirming each replica and client picks up the new state, and documenting what was done. Effort rises when the topology has several replicas per site, when services such as the web UI, LDAP or Kerberos depend on certificates that expired at different times, or when the team must reconstruct evidence for an auditor. A quote follows from a written scope that lists the servers, certificates, backup state and change window involved. DeployOpen provides that quote after the scope is written.
How long does recovery take for a FreeIPA deployment?
Duration depends on topology and backup state more than on the repair command itself. A single server with a current, tested backup is simpler than a multi-replica deployment where renewal, CRL publication and replication must be checked in order. Ask your team to confirm the number of servers, which one is the renewal master, when the most recent full backup was taken, and whether any replica has been offline during the outage. If a backup is missing or outdated, the work expands into recovery planning, which is the riskiest part. Any timeline should come from the written scope rather than from a general estimate.
What must our team provide before recovery starts?
Recovery needs a small set of inputs that your team controls:
- Administrative access to every FreeIPA server and the renewal master, with a named contact who can approve changes
- A current inventory of which servers run the CA, DNS and other services, and which certificates have expired
- Recent full backups taken with ipa-backup, stored off the affected hosts and verified for readability
- A change window agreed with the business owners, including a rollback plan and a way to reach users during the work
- Access to affected clients, load balancers and any services that consume FreeIPA certificates
The backup documentation warns that restoring an outdated backup into an active replica topology can break replication, so backup state must be confirmed before anything else.
Can our own team recover FreeIPA from expired certificates?
Yes, a team with strong Linux and FreeIPA operations experience can run ipa-cert-fix itself, provided it reads the tool's documentation for the installed release, takes verified backups, and tests the procedure outside production when possible. The questions to answer honestly are whether someone has done a recovery before, whether the team knows which server holds the renewal role, and whether it can work through replication and client checks during the window. If any of those answers is uncertain, a fixed-scope engagement reduces the chance of a second outage. Your team keeps ownership of the decision either way.
What ongoing work follows a certificate recovery?
Recovery is usually followed by work that prevents the same failure. This includes setting up tracking so each certificate has a known renewal path, checking expiry dates on a schedule, confirming that the renewal master and CRL publication behave as expected, and reviewing how service certificates are requested and renewed. Some teams also document a runbook for the next recovery and test restores on a schedule. These tasks recur, so ask whether the team will own them internally or whether a support retainer should cover health checks and upgrades. DeployOpen offers ongoing support and upgrades as retainers.
When should we choose a fixed-scope engagement for FreeIPA recovery?
A fixed-scope engagement suits teams that have a defined set of servers, known certificates and a backup they trust, and that want the work, deliverables and timeline agreed in writing before the window opens. It is also a sound choice when the internal team is small, when regulated evidence of the change is needed, or when a previous attempt has failed. Scope is agreed in writing before work starts, so the boundaries of the change are clear to both sides. If the environment is still uncertain, a short assessment of the topology and backup state usually produces a better quote than a guess made from a description.
How DeployOpen scopes and delivers FreeIPA certificate recovery
DeployOpen approaches expired-certificate recovery in FreeIPA as a controlled, written-scope job inside your own infrastructure. We first establish which certificates have lapsed, which services they stop, and whether ipa-cert-fix is the right tool, then work through backups, the renewal master and each replica in a planned order.
How we scope
- We inventory the CA, LDAP, HTTP and service certificates on every FreeIPA server and record their expiry dates before any change.
- We confirm which server is the renewal master and whether any shared certificates depend on it, since this sets the recovery order.
- Engagements are fixed-scope, so the written scope states which servers, services and certificate types the recovery covers and what it excludes.
How we build
- We verify a complete ipa-backup exists and is restorable on a separate host before running ipa-cert-fix on production servers.
- We run the recovery on a non-production copy of your topology first, then document each step, output and rollback point for your team.
- We check replication health, Kerberos logins, Web UI access and certmonger-tracked service certificates after each stage of the work.
How we hand over
- You receive a written record of the certificates that were renewed, the servers touched, and the verification results for each.
- We document what ipa-cert-fix cannot fix in your environment, such as external CA chains or clients that need a manual trust update, with owners for each item.
- We agree a runbook for the next renewal window, including monitoring for certificate expiry dates, so the same lapse is caught earlier.
How we support
- Ongoing support, health checks and upgrades are available as a retainer after the recovery is complete.
- A retainer can include periodic certificate expiry reviews, so renewal work is planned instead of discovered during an outage.
- Confirm during scoping whether your team or DeployOpen will handle future renewals, and what response arrangements apply.
