Why Backstage search on Elasticsearch 7 needs a plan now
A platform team running Backstage search on Elasticsearch 7 has to change search engines before the upgrade can finish. The supported paths are to move to an Elasticsearch 8 line at version 8.19 or later, to move to OpenSearch, or to switch to the PostgreSQL search engine. In each case the index must be rebuilt in the new engine, so the work is a reindex and a configuration change, not a version bump that a routine monthly upgrade can absorb. Start by confirming which search engine and which version your deployment currently points to, and who in your organization owns that cluster today.
The licensing facts shape the decision. Elasticsearch and Kibana source code is available under three licenses: the Elastic License 2.0 (the default distribution), SSPL 1.0 and AGPLv3, with AGPLv3 added in September 2024 before the 8.16 release. Elastic states that ELv2 and SSPL are not OSI-approved open source licenses. Elastic's self-managed subscriptions page lists single sign-on, field- and document-level security and anomaly-based alerting in paid tiers, and says the Platinum tier is no longer available to new customers. Elastic's current release lines are 9.5.5 and 9.4.8, with 8.19.23 still maintained, and the EOL page gives end of maintenance for 8.x on January 15, 2027. OpenSearch is released under the Apache License, Version 2.0, and its FAQ describes no paid edition. OpenSearch is a fork of Elasticsearch 7.10, so an Elasticsearch 7 cluster is the closer starting point for it.
Because these facts pull in different directions, the choice is a platform decision. It affects which license your legal team reviews, which security features you must buy or build, how your identity and audit controls map to the search tier, and who patches the cluster on what schedule. Confirm the current license terms and tier entitlements directly with the vendor, since these details change. Then compare the three options against your security requirements, your operating model and the effort to reindex your catalog, TechDocs and search sources. Record the decision before the upgrade plan is scheduled.
Comparing the search engine options for Backstage
This table compares Elasticsearch 8.19 or later, OpenSearch and the PostgreSQL search engine across license, support lifecycle, free-tier security, operating load and migration effort. Read each row against your own security requirements and confirm current Backstage search plugin support for each engine before you choose.
| Criterion | Elasticsearch 8.19 or later | OpenSearch | PostgreSQL engine |
|---|---|---|---|
| License | Default distribution under Elastic License 2.0, with AGPLv3 added as an OSI-approved option before 8.16 | Apache License 2.0 for all project software, with no paid edition described in the FAQ | PostgreSQL License, a permissive open source license similar to BSD or MIT |
| Support lifecycle | 8.19.23 is still maintained; 8.x end of maintenance is January 15, 2027 and support ends July 15, 2027 per Elastic | Versions stay in maintenance until the next major enters maintenance or one year passes; new minor releases about every eight weeks | Each major is supported for 5 years; version 14 ends November 12, 2026 and 15 ends November 11, 2027 |
| Security features in free tiers | Basic tier includes TLS, file and native realms and role-based access control; SSO, LDAP and field or document-level security sit in paid tiers | Security plugin includes SAML, OpenID Connect, LDAP, role-based access, document and field-level security, audit logs and TLS | Depends on your PostgreSQL authentication setup, including certificates, LDAP, GSSAPI and OAuth methods |
| Operational load | Separate cluster to run; a highly available cluster needs at least three master-eligible nodes | Separate cluster to run with similar node and sizing planning; installable with Helm or the Kubernetes Operator | Uses the PostgreSQL instance you already operate for Backstage, adding index and storage load to it |
| Migration effort from Elasticsearch 7 | Requires moving from 7.x indices to 8.x, with reindexing and client and plugin changes to confirm | Wire-compatible with Elasticsearch 7.10 and rolling upgrades from supported versions; confirm the Backstage engine setting | No search cluster to build, but index data must be rebuilt in the new engine and results compared |
| Backstage plugin support | Confirm current search plugin support for your Backstage release before choosing | Confirm current search plugin support for your Backstage release before choosing | Confirm current search plugin support and behavior for your Backstage release before choosing |
| Best fit | Teams already running Elastic who accept its license and tier model | Teams wanting an Apache-licensed search cluster with security features in the free distribution | Smaller catalogs where one fewer cluster to operate outweighs search features |
Which search engine fits your organization
If your organization already runs Elasticsearch under a paid subscription, moving to Elasticsearch 8.19 or later keeps the engine, the client behavior and most of your operational runbooks. The change is mainly a version and index migration, so the work is scoped mostly by how many clusters, indices and integrations depend on the current setup. Confirm the subscription tier before you plan, because some security features depend on it.
- Version path: Elasticsearch 8.19.23 is still maintained according to the Elastic releases page, and 8.x is listed with end of maintenance on January 15, 2027, so plan a move to a later line if your timeline runs past that.
- Tier-dependent security: single sign-on, attribute-based access control, field- and document-level security and anomaly-based alerting depend on the subscription tier, so check what your current contract actually includes.
- Platinum status: Elastic states the Platinum tier is no longer offered to new customers, so a renewal or new contract may point to a different tier.
- Operations stay familiar: node roles, hot, warm and cold data tiers and a three-master-eligible-node minimum for high availability continue to apply.
- Kubernetes option: Elastic Cloud on Kubernetes is a self-managed Elastic product offered in Basic and Enterprise tiers, which may matter if you already run clusters on Kubernetes.
Before committing, ask your Elastic account team or licensing contact to confirm the tier and feature list in writing.
If your policy requires an OSI-approved license, OpenSearch is the usual path. All OpenSearch software is released under the Apache License, Version 2.0, and the FAQ does not describe a paid edition. OpenSearch is a fork of Elasticsearch 7.10 and is wire-compatible with it, so Backstage search moves with fewer application changes. You take on the operation of a different project with its own release cadence and support windows.
- License clarity: Elastic's default distribution is under the Elastic License 2.0, and the AGPLv3 option was added later, so the Apache 2.0 path is simpler to explain to legal and procurement.
- Release cadence: OpenSearch targets a new minor version about every eight weeks, and versions move to maintenance on a defined schedule, so pin a version and plan upgrades.
- Security features in the core: the OpenSearch Security plugin provides SAML, OpenID Connect, LDAP and Active Directory, role-based access control, document- and field-level security and audit logs. Confirm which of these your chosen distribution enables in your configuration.
- Governance: the OpenSearch Software Foundation is a Linux Foundation project, but it is not involved in technical oversight of the project.
- Deployment choices: Docker, Helm, the OpenSearch Kubernetes Operator and tarball or package installs are documented.
Confirm with your security team that the Security plugin configuration meets your audit and access requirements before you build on it.
For a smaller catalog, the PostgreSQL search engine can avoid running a separate search cluster altogether. You already operate PostgreSQL for the Backstage catalog and other plugins, so the added operational surface is smaller. The trade-off is that search capacity and query behavior depend on your database sizing, and you should test it against your real catalog volume before relying on it. Confirm current engine support in the Backstage search documentation for your release.
- Fewer moving parts: no additional cluster, node roles, index lifecycle or search-specific certificates to maintain.
- Shared backup and recovery: PostgreSQL backups, point-in-time recovery and replication already cover your portal data, which simplifies the recovery plan.
- Database sizing: search queries share resources with the catalog and other plugins, so monitor connections and load, and consider connection pooling with PgBouncer.
- Supported PostgreSQL line: PostgreSQL 14 is supported to November 12, 2026, while 15 to 18 are later lines, so check your version before you plan.
- Growth point: if the catalog grows or search demand increases, you may need to revisit the choice, so define the threshold in advance.
Ask your Backstage implementer to test search latency and index behavior on a copy of your catalog before you retire the Elasticsearch cluster.
How to move Backstage search from version 7 to 8
This sequence moves Backstage search off Elasticsearch 7 onto a supported engine, rebuilds the index from the catalog and TechDocs sources, and leaves the old index in place so you can return to it if results are wrong after cutover.
Confirm the target engine and Backstage release
Your team decides whether the new search backend will be Elasticsearch 8.19 or a later 8.x release, or OpenSearch, and confirms that your Backstage release supports that backend and its search plugin version. Elasticsearch 8.x is listed as maintained until January 15, 2027, with support ending July 15, 2027, so the choice should account for that window. Confirm the supported engine versions in the current Backstage search documentation and in the Backstage release notes for the version you run. Done when the engine, its version and the Backstage release are written down and agreed by the platform owner and the security reviewer, who must also sign off on the license choice (Elastic's default distribution uses the Elastic License 2.0, while OpenSearch is Apache 2.0).
Stand up the new search cluster
The platform team builds the new cluster in infrastructure you control, separate from the Elasticsearch 7 cluster. For a highly available Elasticsearch cluster, plan at least three master-eligible nodes. Apply TLS, authentication and role-based access before any Backstage traffic reaches it, and confirm which security features your chosen tier or distribution includes, since some Elastic features sit in paid tiers. Keep the Elasticsearch 7 cluster running and untouched. Done when the new cluster passes a health check, accepts an authenticated test write from the Backstage backend network segment, and rejects unauthenticated requests.
Update the search plugin configuration
Engineers add a second search engine configuration to the Backstage backend, pointing at the new cluster's endpoint, credentials and TLS trust chain, while the existing configuration still points at version 7. Store credentials in your secrets mechanism, not in the repository. Confirm the exact configuration keys and the search engine module names in the current Backstage search documentation for your release, because these change between versions. Done when the configuration builds in a development environment and the backend starts with the new target defined, without yet serving users from it.
Build the new index with a full reindex
Run a full reindex of the search collections (catalog entities, TechDocs and any other enabled collections) against the new cluster. The backend's indexing schedule and the search index names are set in configuration, so confirm them before the run. Watch the backend logs for indexing errors per collection and note the document counts for each collection. Do not rely on copying Elasticsearch 7 indices, because search indices are rebuilt from the sources. Done when each collection reports a complete run and the document counts in the new index match the catalog and documentation sources within an explained tolerance.
Compare results in a staging Backstage
Point a staging Backstage instance at the new index and run a fixed set of queries that your users rely on: known entity names, owners, tags, TechDocs page titles and phrases. Compare the top results and the total hit counts with the production instance on Elasticsearch 7. Record differences in ranking, missing documents and permission filtering, and check that users only see results their catalog access allows. Done when the query set has a written pass or fail result for each item, and every failure has an owner and a fix or an accepted explanation.
Cut over to the new index
Schedule the switch during a window your users have been told about. Change the production backend configuration so search reads and writes go to the new cluster, deploy it, and keep the Elasticsearch 7 configuration in version control as the rollback target. Confirm that search returns results from the new cluster, that the indexing schedule runs against it, and that catalog changes made after the reindex appear in search. Done when the production search queries pass the same checks used in staging and the backend logs show no search errors for a full scheduled indexing cycle.
Keep the old index for rollback
Do not delete the Elasticsearch 7 cluster or its indices at cutover. Keep them running, read-only if your operations allow, until your team has accepted the new search results over an agreed period. Document the rollback procedure: revert the backend configuration to the Elasticsearch 7 target, redeploy, and trigger a reindex if documents changed while the new index was live. Confirm the retention and decommissioning decision with the security and compliance owners, since old clusters still hold indexed catalog data. Done when the rollback procedure has been tested in staging and the decommissioning date is recorded with an owner.
What to prepare before touching the search cluster
Most search upgrade problems start before any node changes version. The items below fix your rollback position, reveal the custom code that depends on the search engine, and give you a safe place to rehearse the move. Work through them in order, because the inventory decides what the staging test has to cover.
Snapshot the existing cluster
Take a snapshot of every index Backstage search uses on the Elasticsearch 7 cluster and copy it to storage outside the cluster. Confirm you can restore it into a separate test cluster before you rely on it. This is your rollback position if the reindex or the new engine misbehaves, so check that the repository and its credentials are reachable from the place you would restore to.
Inventory custom search extensions
List every custom collator, decorator and plugin search extension in your Backstage app and backend. For each one, record which index it writes to, which document fields it adds, and whether it uses Elasticsearch-specific query syntax. These are the parts a default upgrade will not check for you, so they need an owner and a test case before the cutover.
Build a production-like staging environment
Stand up a staging Backstage instance with a copy of your production catalog, at a similar entity count and with the same plugins enabled. Point it at a separate search cluster. Staging is where you measure reindex duration and find mapping or query errors, so a small sample catalog will hide the problems you need to see.
Set credentials and TLS for the new engine
Decide how Backstage authenticates to the new search engine before the first connection attempt. Create the service account or API key with the narrowest index permissions the search plugin needs, and issue the TLS certificates the client will trust. Record the secret names and rotation owner so the configuration change is reviewable and does not depend on a shared admin login.
Check capacity for reindex load
Estimate the extra CPU, memory, disk and network your cluster needs while the old and new indices both exist and the reindex runs. Ask your platform team to confirm headroom on the target cluster and on the Backstage backend that drives the job. Plan the reindex window around normal catalog update traffic, and agree who can pause it if search latency rises.
How reindexing changes the cutover plan
The search index is rebuilt from its sources, not copied wholesale, so the cutover plan has to allow time for a full ingest before users can rely on the new engine. Catalog entities, TechDocs pages and other indexed documents are read again from their providers and written into the new cluster. Plan for the catalog to be re-ingested and for TechDocs to be rebuilt where they are indexed, and confirm with your team how long a full pass takes against your entity count and document volume. Results for recently changed entities may lag until the ingest catches up, so tell catalog owners that search freshness will be reduced during the window.
Keep the Elasticsearch 7 index serving queries until the new index is verified. Point the search backend at the new cluster only after comparing result counts and spot-checking known entities, owners and documentation pages against the catalog. The OpenSearch FAQ states that OpenSearch can use indices from Elasticsearch 6.0 through 7.10, but that does not by itself tell you whether Backstage's search plugin will read or migrate your existing indices. Confirm with the Backstage search plugin documentation and your own test cluster whether a reindex path is supported for your setup, or whether a fresh build from sources is the only route.
Decide before cutover how you will roll back. Retain the Elasticsearch 7 cluster and its data until the new index has passed review, and record which configuration key switches the backend so the change can be reverted in one step. Test the rollback on a staging instance, since a rollback that depends on an index you have already deleted is not a rollback.
What makes a search upgrade take longer
This table lists the factors that expand or shrink an Elasticsearch 7 to 8 search migration for Backstage, with the direction each one pushes effort. Use it to size the work before asking for a quote, and to see which answers about your environment matter most.
| Driver | Why it changes effort | How to keep it down |
|---|---|---|
| Catalog entities and TechDocs pages | More documents means a longer full reindex and more validation of search results after cutover | Clean up orphaned entities and unused docs before reindexing, and confirm the document count per index |
| Custom search collators and plugins | Each custom collator or plugin that writes to search must be checked against the new engine and mapping | Inventory every collator and plugin, and retire those nobody uses |
| Security requirements such as TLS and SSO | Features like single sign-on and field-level security may need a paid tier or a different engine, which changes the target choice | Confirm the licensing needed for each control before choosing between Elasticsearch 8 and OpenSearch |
| Field-level or document-level access rules | Access controls on search results must be re-created and tested on the new engine | List which entities are restricted and test those queries by role |
| Number of environments | Each environment needs its own reindex, configuration change and test pass, often in sequence | Rehearse on a non-production copy first, then reuse the same steps for each later environment |
| Self-managed cluster in your infrastructure | You own node sizing, master-eligible node count, upgrades, backups and monitoring of the cluster | Confirm the node layout, backup approach and who is on call before the build starts |
| Team familiarity with the target engine | Teams new to Elasticsearch 8 or OpenSearch need more time to learn its security and operations model | Assign an owner who can run the cluster after handover |
Questions teams ask before upgrading Backstage search
Does Elasticsearch 7 still work with Backstage search?
Backstage search no longer supports Elasticsearch 7, so an existing 7.x setup needs a planned move to Elasticsearch 8.19 or later, or to OpenSearch. Check the Elastic end-of-life page for the dates that apply to your release. The 8.x line also has a finite life: Elastic lists end of maintenance for 8.x on January 15, 2027 and end of support on July 15, 2027. Choosing 8.19 buys time, but it also starts a clock toward a second move. Confirm which Elasticsearch versions your Backstage release supports before you pick a target.
Is OpenSearch truly open source under the Apache 2.0 license?
Yes. All software in the OpenSearch project is released under the Apache License, Version 2.0, and the OpenSearch FAQ does not describe any paid edition. OpenSearch was forked from Elasticsearch 7.10, and OpenSearch Dashboards derives from Kibana 7.10.2. The OpenSearch Software Foundation is a Linux Foundation project, but it is not involved in technical oversight of OpenSearch, so review the project's own release and maintenance pages when you plan. Under that project's policy, a version in maintenance receives bug fixes and security patches but no new features, and new minor versions arrive about every eight weeks. Confirm that your Backstage release supports the OpenSearch line you choose.
Is PostgreSQL search enough to replace Elasticsearch?
It can be enough for some portals, but the answer depends on your catalog size, query patterns and ranking needs, so test it before you commit. Backstage already runs on PostgreSQL in production, so a Postgres search engine adds no new service to operate, which is the main reason teams consider it. Ask whether the results, filters and index freshness your users expect hold up on a copy of your production catalog and TechDocs content, and compare the outcome with what your current search delivers. Also confirm your PostgreSQL version is supported. Version 14 is supported until November 12, 2026, and version 15 until November 11, 2027, so pick a supported major version before the search work starts.
What cost and effort drivers should we ask a partner to quote?
Ask for a quote built from these drivers, each of which changes effort and cost:
- The volume of indexed content and the number of indices, since reindexing time grows with both.
- The target engine: Elasticsearch 8.19 or later, OpenSearch, or Postgres, and where it runs (your cloud, data center or air-gapped network).
- Cluster design, including node roles and whether you need at least three master-eligible nodes for high availability.
- Security scope, such as TLS, authentication, roles and audit logging, which depend on the Elastic subscription tier or the OpenSearch plugins you use.
- Testing, cutover and rollback steps, plus runbooks for your operators.
Design, migration and testing are one-time work. Hosting, backups, operations and any Elastic subscription recur. Ask the partner to show these lines separately and to name any dependency on your Backstage upgrade.
How long does a Backstage search engine upgrade take?
No researched figure sets a duration, so the timeline depends on your environment. The main drivers are data volume and the number of sources feeding search, the cluster topology and target engine, plugin and configuration changes, and how many test and staging cycles your change process requires. Ask for a schedule split into inventory, target build, reindex and validation, cutover and decommissioning, with an estimate for each stage. The most reliable number comes from timing a full reindex of a copy of your production data in staging, because reindex speed depends on your hardware and content. Under a fixed scope, the agreed timeline is written into the scope before work starts.
What must our team supply before a search upgrade starts?
Before the work starts, your team should provide:
- Read access to the current Elasticsearch cluster, including index mappings, settings and document counts.
- The Backstage version you run and the search configuration in your app.
- Cloud, data center or network access for the new cluster, plus a staging environment.
- A decision on the target engine and a named owner for that choice.
- Named owners for catalog, search and identity work, and an agreed change window.
- Your backup and rollback expectations, and any evidence your change process needs.
Confirm these items in the written scope so that gaps appear early rather than during cutover.
Can our own team upgrade Backstage search without a partner?
Yes, if your team already runs Backstage backends and operates Elasticsearch or OpenSearch clusters, and has time to test in staging. The parts that carry the most risk are cluster sizing, security configuration including TLS and roles, reindex validation and a tested rollback. A scoped review is a practical middle path: a short written assessment of your current cluster, your Backstage configuration and the target options, which your team can then carry out. DeployOpen plans, implements, hardens and supports self-hosted open-source platforms in infrastructure you control. Most engagements are fixed-scope and agreed in writing, and retainers are available for ongoing support.
How DeployOpen scopes and delivers this search upgrade
DeployOpen plans the move of Backstage search off Elasticsearch 7 as a written, fixed-scope engagement inside your own infrastructure. We agree the target engine, the reindexing plan, the test evidence and the handover before any change reaches production.
How we scope
- We confirm which Backstage release and search plugin versions you run, and which Elasticsearch 7 deployment serves them today.
- We compare Elasticsearch 8.19 or later with OpenSearch and the Postgres engine against your licensing, hosting and team skills.
- The written scope lists the indices, reindex windows, search filters and dashboards covered, plus what stays out of scope.
How we build
- We stand up the target search engine in a non-production environment that mirrors your access controls and network paths.
- We reindex a copy of the catalog and TechDocs content, then compare result counts and sample queries against production.
- We update the Backstage search configuration and test sign-in, permission filtering and catalog results before cutover.
How we hand over
- You receive the configuration, the reindex runbook, rollback steps and the test results used to approve cutover.
- We document the search engine version, its support dates and the owner for each credential and index.
- Your team runs a walkthrough and a supervised reindex so that the next routine upgrade does not depend on us.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after the handover.
- A retainer can cover search engine patching, Backstage version bumps that touch search, and periodic reindex checks.
- Before any retainer starts, we agree its written scope, response expectations and the systems it covers.
