Why Backstage catalog entities go missing or stale
Backstage catalog troubleshooting starts when a team's service is absent from the catalog, shows outdated data, or carries a warning flag on its entity page. Catalog processing errors are problems that stop Backstage from turning a descriptor file into a usable entity. You need to troubleshoot them when a new service never appears after registration, when a change to catalog-info.yaml does not show up, or when an owner sees an error badge they cannot explain. The first question is not whether the file is valid YAML but where in the pipeline the entity stopped.
Backstage reads descriptor files through locations, which point at a file or a directory, and providers that discover and load them, often from a source control platform. Processors then parse each entity, validate it against its kind and apply rules such as relationship checks and default values. Each step can record an error on the entity itself, so the entity page is usually the first place to look. A missing entity usually means the failure happened before any entity was created, such as an unreachable location or a provider that never ran, while a stale entity usually means processing ran but stopped on a later step or was never scheduled again.
Start by separating three situations: the location was never read, the file was read but rejected, or the entity exists but its relationships or ownership are broken. Confirm which location the team registered and whether the provider's credentials and read permissions still work, because access changes are a common cause of silent gaps. Then check the processing errors on the entity and any error logs from the catalog backend. Record which descriptor kind is involved, since kinds the instance does not accept or cannot resolve will fail even when the YAML is well formed. Confirm these details against the catalog documentation for your Backstage version, as behavior around providers and processors has changed over the project's monthly and weekly release lines.
Which catalog error class you are looking at
A missing, stale or broken entity usually traces back to one of a small number of failure classes. Each class shows a different symptom in the catalog UI and points to a different place in the configuration, the processing pipeline or the source system. Identify the class first, then open only the logs and settings that belong to it.
Processing errors on the entity
The entity exists in the catalog but shows an error attached to it, often in the entity's overview or processing status. The error describes a step that failed while the catalog was validating or enriching that specific entity. Start with the error text on the entity itself, then check the descriptor fields the message names. Because the error is tied to one entity, the rest of the catalog usually keeps working, which helps you separate this class from broader failures. Check whether the most recent change to that descriptor introduced the problem.
Location and read failures
A location is a source the catalog reads from, such as a URL or file path that points at one or more descriptor files. When the catalog cannot fetch or parse that location, every entity it would have produced is missing, not just one. Look for a single failed location rather than many scattered errors. Check that the URL or path is reachable from the backend process, that the credentials for the source host are present in the environment the backend runs in, and that the file at the path actually parses as YAML.
Provider sync failures
Entity providers discover entities from an external system on a schedule, such as a repository scanner or a directory sync. A provider sync failure leaves previously ingested entities in place but stops new ones from appearing and can leave removed ones behind. Check the provider's configuration block, its schedule, and whether the token or service account it uses still has read access. Compare the last successful run time with the time you expect the next run, because a silently stopped schedule looks like a missing entity.
Rejected kinds and invalid descriptors
The catalog only ingests kinds and API versions it knows, and a descriptor that uses an unregistered kind, a misspelled apiVersion, or a required field that is absent is rejected before it becomes an entity. Nothing appears in the catalog, so there may be no entity-level error to read. Check the apiVersion and kind at the top of the file against the entity kinds your installation supports, then confirm that the metadata.name and spec fields match the schema for that kind. A kind added by a plugin will be rejected until that plugin is installed in the backend.
Orphaned entities
An orphan is an entity that was once ingested but no longer has a source that produces it, so the catalog keeps it in place rather than deleting it. Orphans typically follow a moved, renamed or deleted descriptor, or a provider that no longer returns the entity. They show up as entities that look correct but no longer change. Find the source that last defined the entity, confirm whether the file or repository still exists, and decide whether to remove the entity or restore the source. Confirm the orphan handling settings in your configuration before deleting anything in bulk.
Processing loops and stale entities
A processing loop happens when processing an entity keeps re-triggering processing for the same entity, which consumes backend time and can leave other entities waiting. A stale entity is one whose data no longer matches its source because refresh has not run or has been failing quietly. Both show up as entities whose last-updated time does not move, or as a backend that stays busy with few visible changes. Check the backend logs for repeated processing of one entity name, then look for relations or annotations that reference each other in a cycle. Verify the refresh interval before assuming the source is at fault.
How to find the cause of a missing or broken entity
This sequence moves from the entity's source location to its processing state, the file itself, the registered provider and finally competing references, so you isolate the failing stage instead of guessing. Each step ends with a check you can record before moving on.
Confirm the entity's source location
Start by identifying where the entity should come from: the location or provider that declared it, usually a catalog-info.yaml file in a Git repository or a location entry in the app-config file. Ask the owning team for the exact repository, branch and path, and compare them with what is registered in the catalog. In the catalog UI, search for the component by name and open its Overview to see the location annotation. Done when you can name one source URL that matches the file in the repository.
Check processing errors and last refresh
Open the entity page and look for processing errors or warnings reported against it, then note the last refresh time shown with the entity. Errors show the processor that failed and the message it returned, which tells you whether the problem is in fetching, parsing, validation or relations. If the entity has no errors and its refresh time is recent, the problem is probably upstream of the catalog, such as a file that was never committed. Done when you have written down the error text, or confirmed there is none and a recent refresh time.
Verify the file is readable and valid YAML
Fetch the descriptor exactly as Backstage would, either from the Git host or from the file path on the backend, and confirm the request succeeds without authentication errors. Parse the file as YAML, then check that apiVersion and kind are present and match a kind your catalog accepts, and that metadata.name is set and unique within its namespace. Watch for tabs, duplicated keys and multiple documents in one file, since these produce parse or validation errors. Done when the file parses cleanly and carries a valid apiVersion, kind and name.
Check the provider or location is registered and reachable
Confirm that the location or entity provider that reads the file is registered in the backend configuration and that it started without errors. Check the backend logs for the provider's name at startup and for repeated fetch failures, which usually show a timeout, an expired token or a missing permission on the repository. Confirm the integration credentials can read the specific path and branch, not only the organization. Done when the provider appears in the logs as running and a test read of the file succeeds from the backend host.
Check for orphans and conflicting references
Look for entities that exist but have no location pointing to them, which the catalog marks as orphans, and for a second entity with the same kind, namespace and name registered from another file. Also check spec.owner, spec.system and dependsOn references for entities that do not exist, since broken references leave the entity visible but incomplete. Remove duplicate registrations and correct misspelled references, then trigger a refresh of the affected location. Done when the entity shows a single source, no orphan flag and no unresolved references in the catalog UI.
Record the outcome and confirm the refresh
After the fix, trigger a refresh of the entity or its location and wait for the next processing pass. Confirm in the catalog UI that the entity appears with the expected owner, type and lifecycle, that the processing error list is empty, and that the last refresh time has moved forward. In the backend logs, confirm there are no new errors for that location in the same window. If the error returns, compare the logged processor name with the step where it failed to decide whether the cause is the file, the provider or a conflict. Done when the entity is correct and the logs are quiet for that location.
Fixes by cause and when each fits
Most processing errors come from the catalog-info.yaml file itself: a missing required field, a malformed metadata name, an owner reference that does not match an existing Group or User, or a dependsOn target that was renamed. The fix lives in the source repository, so it goes through the normal pull request and review path and takes effect when the catalog next reads that file. Because no service restart is needed, this is usually the cheapest and most auditable change, and the commit history records who changed the entity and why.
- Read the error first: the processing error attached to the entity names the failing field, so confirm the message before editing anything.
- Validate locally: check the YAML and the entity kind against the schema your Backstage version expects before opening the pull request.
- Fix references in the same change: if you rename a System or Component, update every dependsOn, consumesApi and providesApi reference that points to it.
- Expect a delay: the catalog re-reads the file on its refresh cycle, so a successful commit may not clear the error immediately.
- Watch for drift: if the same file keeps regressing, the template or generator that produces it needs the fix, not the individual repository.
Before closing the ticket, confirm the entity shows no processing error and that its relations render on the graph.
When no entity appears at all, or a whole group of entities goes stale together, the problem is usually in app-config.yaml rather than in any single descriptor. Location targets, catalog provider settings, integration tokens and the schedule that polls them are all configured on the backend. Changing them means a configuration change, a deploy and often a credential rotation, so the change path is slower and touches more people than a descriptor edit. Treat it as a platform change with its own review.
- Check the provider first: confirm the provider's filters, branch and path patterns still match where the descriptors actually live.
- Check the integration credential: an expired or under-scoped token causes failures that look like missing files.
- Keep secrets out of the file: reference environment variables or your secret store rather than committing tokens.
- Plan the deploy: the configuration takes effect after the backend restarts with the new file, so schedule it and confirm the rollback.
- Reconcile the blast radius: a provider change can add or remove many entities at once, so review the expected count before deploying.
After the change, confirm the expected entities reappear and that no entities were removed unintentionally.
An entity with a kind the catalog does not accept is rejected during processing. The choice is between correcting the descriptor to use an accepted kind and extending the catalog so the kind is allowed. Extending the catalog is a design decision, because every new kind needs an owner, a schema, a meaning that teams agree on and handling in templates, search and plugins. Adding kinds to satisfy one team's file tends to spread inconsistency across the catalog, so the decision should be recorded before the configuration is changed.
- Prefer an existing kind: most rejections come from a Component or Resource being labeled with an unsupported kind, which a descriptor edit fixes.
- Confirm the kind is intentional: agree the meaning, owner and lifecycle values for any new kind with the platform team.
- Check plugin support: a kind that loads may still be invisible to the catalog views and plugins that expect the standard kinds, so confirm what it affects.
- Document the decision: record the new kind in the catalog model guidance so later authors use it consistently.
- Confirm the version: check the kinds supported in your Backstage release, since this varies between versions.
After a change, confirm the entity processes without error and that the kind appears in the catalog filters you expect.
An orphan is an entity whose source file or provider no longer supplies it, so the catalog keeps it without a current origin. Orphans appear after repositories are archived, files are moved, or providers are narrowed. The two fixes are to restore ownership by reattaching the entity to a live source, or to remove it. Which one fits depends on whether the service still runs. Removing a live service's entity hides its ownership and dependencies from everyone who relies on the catalog, so confirm the state of the underlying system before deleting anything.
- Confirm the service state: check whether the system still runs and who is responsible before choosing between reattaching and removing.
- Reattach through the source: if the service moved, add its descriptor to the new repository so the entity has a current origin.
- Remove deliberately: deleting an entity is simplest when its source is gone and nothing depends on it, so check its relations first.
- Handle ownership gaps: an orphan often has an owner that no longer exists, so reassign ownership before the entity is kept.
- Review on a schedule: a periodic check of orphaned entities catches decommissioned services before they accumulate.
After the fix, confirm the entity either has a live source with a valid owner or no longer appears in the catalog.
A processing loop happens when entities keep re-triggering each other's processing, typically through relations, location references or generated entities that point back to their origin. The symptoms are repeated processing activity, entities that never settle, and load on the backend. The fix is to remove the cycle at its source, so the change usually belongs in the descriptor or generator that creates the reference, not in the catalog settings. Stopping the loop by disabling processing treats the symptom and leaves the cause in place.
- Trace the chain: follow the references from the entity that keeps re-processing to find the point where the cycle closes.
- Break the cycle in the generator: if a template or provider creates the back reference, change it to stop emitting the link.
- Use one direction for relations: keep a single authoritative side for each relationship and let the other side be derived.
- Verify the processing count: after the change, confirm the activity settles instead of only checking that the error is gone.
- Plan for the backend load: a loop can slow the whole catalog, so schedule the fix when the impact is visible to users.
Before closing the issue, confirm the affected entities stop re-processing and that their relations still render as intended.
Which configuration changes affect catalog processing
This table compares the four configuration areas you are most likely to change when an entity is missing, stale or broken. For each area it shows what the setting controls, the trade-off it introduces and when to reach for it. Confirm exact configuration keys against the catalog documentation for your installed Backstage version before editing app-config files.
| Configuration area | What it controls | Trade-off | When to change it |
|---|---|---|---|
| Catalog locations | Which descriptor files or URLs the catalog reads at startup and on refresh, such as a specific catalog-info.yaml path in a repository | Explicit locations are easy to audit but must be updated by hand when repositories move or rename | An entity never appears, or a repository's descriptor path changed |
| Entity providers | How entities are discovered in bulk, for example by scanning a source control organization or an identity directory on a schedule | Discovery finds more entities automatically, but a broad scan can ingest unowned or test repositories and increase processing load | Entities are missing across many repositories, or stale because no provider refreshes them |
| Processing rules for kinds | Which entity kinds the catalog accepts and how each kind is validated and processed; unsupported kinds are rejected with an error on the entity | Enabling more kinds widens what teams can register, while restricting kinds keeps the model consistent but turns some valid descriptors into rejected entities | An entity is rejected with an unsupported kind error, or a custom kind needs to be accepted |
| Read access to source repositories | The credentials and integration settings the backend uses to fetch descriptor files and check file existence | Least-privilege read tokens limit exposure but a token scoped too narrowly produces location and provider failures that look like missing files | Processing errors report authentication, 404 or access denied failures for a location that exists |
Setting up a repeatable catalog troubleshooting routine
This sequence makes catalog processing errors visible to the people who own entities, assigns one group to triage them, blocks bad descriptors before deploy, and records expected entity counts so drift shows up early.
Confirm which catalog errors reach entity owners
The platform team checks how processing errors are exposed in the Backstage UI today, including the errors shown on each entity page, and decides which teams can see them. Owners need to see errors for the entities they own without relying on someone reading backend logs. Confirm the exact view and permissions with your Backstage version, since catalog error display and access rules depend on the plugins and authorization setup you run. Done when a test entity with a deliberately broken descriptor shows its error to a user from the owning team and to no unrelated team.
Assign a named triage group for processing errors
The platform team names a rotating triage owner, plus a backup, who receives catalog error reports and decides whether the fix belongs to the entity owner, the integration owner or the platform team. Write the routing rules down: which error types go to which group, and what counts as a blocking error. Include the team that owns each entity provider, since provider failures often come from a source control or directory setting outside the catalog. Done when the rota is written in your team runbook and a sample error has been routed to the correct group in a dry run.
Add descriptor validation to CI before deploy
The platform team adds a check in the pipeline that validates catalog descriptors in changed repositories before they merge or deploy, so malformed YAML, missing required fields and unknown kinds fail in review rather than as errors after release. Use the validation tooling that matches your Backstage version and catalog setup, and confirm the exact command with the project documentation. Pair it with the configuration check used for the portal itself. Done when a pull request containing a deliberately invalid descriptor fails the pipeline with a message that names the file and field.
Record a baseline of expected entity counts
Before tuning anything, the platform team exports the current count of entities by kind, owner and source location, using the catalog API or the catalog UI filters. Store the snapshot with the date and the Backstage version, and agree which counts should stay stable, such as the number of Components registered from each repository. A count that drops sharply after a deployment or provider change usually points to a failed location or a rejected kind. Done when the baseline file is committed next to the runbook and a second export made later can be compared against it.
Set alerts on sudden changes in counts and errors
The platform team sets an alert or a scheduled check that compares current entity counts and error totals with the baseline and notifies the triage group when they move beyond the agreed tolerance. Define the tolerance with the owners who know normal churn in their repositories, and decide whether to alert on errors, on count drops or on both. If your monitoring stack already collects Backstage metrics, reuse it, and confirm which catalog metrics your version exposes before building dashboards on them. Done when a deliberate removal of a test location triggers the alert in the triage channel within the check interval you set.
Run a drill with a planted fault
The platform team introduces one controlled fault in a test location, such as a renamed file or an unreachable repository path, and follows the routine end to end: the owner sees the error, the triage group receives the alert, and the fix restores the expected count. Record how long each handoff took and where the process stalled. Use the drill results to adjust the routing rules or the alert tolerance. Done when the drill ends with the entity restored, the baseline matching again, and the runbook updated with any gaps the drill exposed.
Review the baseline after each release
After every Backstage upgrade or change to an entity provider, the platform team re-exports the entity counts and compares them with the stored baseline before closing the change. Differences are either expected, such as new services onboarded, or they are investigated as possible processing regressions. The team updates the baseline only with a recorded reason. Done when the post-release comparison is attached to the change record and every unexplained difference has an assigned owner.
Security considerations when fixing catalog ingestion
Fixing a broken catalog entity often means changing how Backstage reads source repositories, tokens and descriptor files. Each change widens or narrows what the catalog can reach, so the fix needs the same scrutiny as any other access change. The cards below cover the controls to check before and after you repair ingestion.
Use read-only source credentials
Locations and entity providers read descriptor files from source repositories, so the credential they use only needs read access to the paths they ingest. A read-only credential limits the damage if the Backstage backend is compromised or a token leaks in logs. When a fix involves a 403 or missing file error, check whether the token lost access or never had it, and grant read access to the specific repositories rather than the whole organization.
Avoid broad provider token scope
Provider tokens often get created with wide scopes because a single error message was hard to interpret. Scope them to the minimum set of organizations, groups and read permissions your providers query, and record which token serves which provider. A narrow scope also makes a failing provider easier to diagnose, since a permission error then points at one identified credential instead of a shared one.
Review descriptor changes like code
A changed catalog-info.yaml can alter ownership, lifecycle or the annotations that drive plugins, so treat each descriptor change as a code change. Require a pull request, a named reviewer from the owning team, and a check that the file still validates before it merges. When a processing error is fixed by editing a descriptor directly in the catalog, move that edit back into Git so the next refresh does not overwrite or reintroduce the problem.
Limit who registers locations
Registering a new location tells the catalog to ingest whatever a URL points to, which can pull in untrusted or unowned content. Restrict registration to a small set of platform administrators or to the catalog permission rules you have configured, and confirm which users can add locations through the UI. Before you register a location to fix a missing entity, confirm its owner and the repository it points to.
Keep secrets out of descriptors
Descriptor files are read by many users and plugins, and they are stored in Git history, so a password or token placed in an annotation or spec field is exposed to everyone with read access. Reference secrets through the backend configuration instead, and scan descriptors in CI for credential-like strings. If a fix requires a secret value to appear in a descriptor, stop and redesign the integration so the value lives in a secret store.
Audit catalog registration changes
When an entity disappears or a location is removed, you need to know who made the change and when. Record location registration and removal, keep the Git history of descriptor files, and check the Backstage authentication documentation so each user's identity maps to a single catalog user. Without a consistent identity, an audit trail shows a provider account instead of the person who acted.
Questions teams ask before fixing catalog errors
How long does it usually take to fix Backstage catalog errors?
No fixed duration applies to every portal, so the answer depends on what is broken and how it is found. The main drivers are the number of entities affected, whether errors come from one location or many providers, how quickly your team can read the processing errors attached to each entity, and whether a custom processor or plugin is involved. Access to the Git repositories that hold the catalog files also matters, because a fix in a descriptor usually needs an owning team to merge it. Ask any partner to state the diagnosis and remediation steps in the written scope, with the assumptions behind the timeline listed, so you can see which factors would extend it.
Can our own platform team fix catalog processing errors without a partner?
Yes, many teams can, provided they have someone who knows the Backstage catalog model and can read the processing errors shown on each entity. Most cases come down to invalid descriptors, locations that point to moved or deleted files, or provider settings that no longer match the source. Harder cases involve custom processors, entity providers written in-house, or errors that appeared after an upgrade. Those usually need a staging instance that matches production, the backend logs, and a clear record of which change started the problem. Your team should confirm it can reproduce the error outside production before changing anything.
What does outside help for catalog errors cost, and what drives the price?
DeployOpen does not publish a price list, and a quote follows from the written scope. The cost comes from the people who do the work: diagnosis of processing errors, repair of custom processors or providers, validation in a staging instance, and documentation of the fix. Costs rise with the number of distinct failure causes, the size of the catalog, the amount of custom code in the processing path, and how many teams must review changes to their own repositories. Prevention controls, such as validation in CI, are a separate line item. Ask for the quote to split one-time remediation from any recurring support or retainer, and to state which tasks stay with your staff.
What access and information does a catalog repair need from our team?
The repair needs enough access to reproduce the problem and confirm the fix. Typically this includes:
- Read access to the catalog repositories and the locations that feed them, with write access held by your owning teams
- A staging Backstage instance on the same release as production, with the configuration files and secrets handled outside the shared repository
- Backend logs and the processing error messages from the affected entities, exported or captured with timestamps
- A list of configured entity providers, locations, and any custom processors, and the change history for each
- A named contact for each team that owns affected entities, to approve descriptor changes
Ask for these items to be listed in the written scope, along with who provides each one and by when.
How do we keep catalog errors from coming back?
Recurrence drops when errors are caught before they reach the running portal. Validate descriptors in the same CI pipeline that merges them into the catalog repositories, so malformed files fail before merge. Assign an owner to each system and component so processing errors have a named team to act on them. Track processing errors and orphaned entities on a regular schedule, not only when a developer complains. Keep Backstage on a supported release line, since the project's versioning policy describes a monthly main line and a weekly next line, and apply upgrades with the version bump tooling rather than ad hoc edits. Document the expected behavior of each location and provider so future changes can be judged against it.
Why do entities go missing or become orphans after a Git change?
An entity usually disappears because the source that produced it no longer emits it, or because the descriptor that defined it fails validation. Common causes include a location path that changed when a repository was renamed or restructured, a provider filter that no longer matches the new layout, a deleted file that still has a registered location, or a descriptor with an invalid kind or missing required field. Orphan entities are typically ones whose source was removed while the catalog still holds the record. Check the processing errors on nearby entities first, then compare the location configuration with the current repository layout, and confirm the change in staging before editing production settings.
Does fixing catalog errors add a license or vendor fee for Backstage itself?
Backstage is an open source framework released under the Apache License 2.0, and the project does not offer a paid edition. Your costs therefore come from the staff who operate it, the infrastructure that hosts it, and any external help you choose to buy. Plugins are a separate matter, because some third-party plugins carry their own licenses or service terms. Before adding a plugin to fix a catalog problem, confirm its license and maintenance status with its publisher. Your legal or procurement team should review the license of any component that enters the processing path.
How DeployOpen scopes catalog troubleshooting and recovery
DeployOpen approaches catalog processing errors as a scoped engagement: we first establish which entities are missing, stale or broken, then trace each failure to its location, provider or processor. The work ends with a documented, tested recovery path your own team can run. Engagements are fixed-scope, with the written scope agreed before work starts.
How we scope
- Agree in writing which catalog locations, entity providers and kinds are in scope for the investigation.
- Inventory the entities currently showing processing errors, orphans or stale timestamps in your portal.
- Confirm which Git repositories, identity sources and processors feed the catalog so the trace has clear boundaries.
How we build
- Reproduce each failing entity in a non-production instance and record the exact processing error message.
- Correct descriptor files, location URLs, provider settings and rejected kind definitions in version-controlled changes.
- Test that affected entities re-process cleanly and that the processing loop no longer repeats on the same failure.
How we hand over
- Document each error class, its root cause and the fix so your team can diagnose similar failures later.
- Walk your platform engineers through the triage steps, log locations and catalog views used during the investigation.
- Deliver a runbook for orphan cleanup and stale entity checks that fits your ownership and review process.
How we support
- Ongoing support, upgrades and continuous testing for your Backstage catalog are available as a retainer.
- Retainer scope is agreed in writing and can cover periodic catalog health reviews after major Backstage upgrades.
- Ask for a quote based on the written scope, since DeployOpen does not publish a price list.
