Temporal advanced visibility with Elasticsearch or OpenSearch

Temporal keeps workflow history and execution state in the persistence store, and it keeps a separate set of visibility records that describe each workflow execution so operators and clients can list and filter them. When a workflow starts, changes state or closes, the server writes a visibility record to the configured visibility store. List and count requests from the Temporal CLI, the Web UI and SDK clients then read from that store, not from workflow history. Search attributes, including the custom ones you register, are indexed as part of these records, so the filter fields available to operators depend on what is registered for each namespace and on the index schema. Confirm the write and read path for your server version in the visibility documentation before you design dashboards or runbooks around it.

Moving search off the primary persistence store gives you richer filtering over large workflow populations and custom search attributes, and it keeps heavy list queries from competing with history reads and writes on the database. The cost is a second stateful system. You take on its sizing, backups, upgrades, access control and retention, and you must coordinate deletion across both stores so that search results do not point at history that no longer exists. Visibility records are also updated after the persistence write, so a workflow may take a short, variable time to appear in list results. Ask Temporal to describe the expected lag for your version, then measure it under your own load during testing.

Search latency and index health now affect the Temporal service directly. When an operator or a client lists workflows, the request depends on the search cluster's response time, so slow queries or an overloaded cluster degrade those calls. If the cluster falls behind on writes, visibility data goes stale and workflows can appear missing or in the wrong state, which is an operational problem even when history is healthy. Plan to watch cluster health signals such as disk pressure, shard allocation, index growth and indexing backlog, and to alert on them alongside Temporal service metrics. The sibling monitoring page covers the metrics side, and you should confirm which visibility metrics your server version emits before you set thresholds.

Choose the visibility store against your query volume and retention needs, not against what the platform team already runs. Cassandra visibility was removed in Temporal Server 1.24, so Cassandra can no longer serve as the visibility store on current releases. MySQL, PostgreSQL and SQLite were documented visibility options from server version 1.20, and the documentation recommends Elasticsearch or OpenSearch for anything beyond a few workflow executions. Elasticsearch 7 and 8 are supported, and OpenSearch 2 or later is supported from Temporal Server 1.30.1. Estimate how many list and filter calls your operators and services make, how long records must stay searchable, and whether your team can operate a search cluster. Then request the supported version matrix for your target server release in writing.

Plan index schema changes and Temporal Server upgrades together, because a visibility change that looks safe on its own can fail during the next server step. Temporal's upgrade path moves one minor version at a time, and it moves to the latest patch of the current minor first, since backward compatibility is guaranteed only between two successive minor versions. When the release notes call for a schema upgrade, run it before the server upgrade. Write the sequence down with the server version, the visibility store version and the Helm chart version. Confirm the chart as well, because Helm chart versions below 0.73.1 do not work with server images 1.30 and later.

A new custom search attribute adds a field to the visibility index that existing workflows do not populate. Queries on that attribute will miss older executions unless you backfill them, so confirm how your Temporal version handles historical executions before you rely on it. A retyped attribute is the more disruptive change. Elasticsearch and OpenSearch generally do not let you change an existing field's type in place, so you may need a new attribute name or a rebuilt index. A change to the index mapping can affect the same way. Dashboards, saved searches and alerts that filter on the old name or type can return wrong results without raising an error, so inventory every consumer of each search attribute before the change.

Test each change on a non-production namespace before it reaches production. Use workflow executions that resemble production in volume and shape, and run the same list and count queries your dashboards and operators use. Confirm that new attributes appear on executions started after the change, that the queries return the results you expect, and that the schema and server steps run in the planned order. Keep the test namespace on the same server and visibility versions as production so the result carries over. Record what you tested and what changed, so the production change can be repeated exactly if it has to be rolled back or run again.

The search store needs its own upgrade track. OpenSearch sends maintenance releases that carry bug fixes and security patches but no new features, so you will apply patch updates on a regular basis. OpenSearch also targets a new minor version about every eight weeks, with 3.9.0 released on September 29, 2026. Before you move the search cluster to a new major version, check Temporal's compatibility documentation for that OpenSearch version, because Temporal supports OpenSearch 2 or later from server 1.30.1. OpenSearch versions also reach end of life. Version 1 reached end of life on May 6, 2025, so keep an OpenSearch calendar next to your Temporal calendar and review both at the same time.

Run two visibility stores during a move only when operators and automation must keep searching workflow history while the new store fills in. The usual cases are moving from another supported visibility store to Elasticsearch or OpenSearch, or moving between Elasticsearch and OpenSearch. The overlap protects search continuity and gives you a path back if the new store misbehaves. It also costs more: both stores must be provisioned, secured, backed up and monitored, and writes must reach both. Before you start, ask whether a shorter planned cutover with a documented search gap would be acceptable to operations and compliance teams. If it would, the overlap may not be worth its cost.

During the overlap, verify the same behavior in both stores. First, list results: sample workflow executions from each namespace and check that workflow ID, run ID, workflow type, status, and start and close times match between stores. Second, custom search attribute queries: confirm that each attribute your workers write is defined in both stores, has the same data type, and returns the same results for the same query strings. Third, retention behavior: confirm how long closed workflows stay searchable in each store and whether the new store's retention settings match the old store's policy. A workflow that is still searchable in one store but already gone from the other is a gap that must be recorded and explained. Use test workflows with known attributes (illustrative) to make these checks repeatable.

Write down, for every stage, which namespaces and workflow types are readable from which store. A simple matrix works: one row per namespace and workflow type, and columns for the stage, the read store, the write stores and the named owner. Stages usually include initial dual writes with reads from the old store, dual writes with reads moved to the new store after validation, and final single-store operation. Confirm the exact dual visibility configuration and its supported combinations in the current Temporal visibility documentation before you plan the stages, because supported store versions depend on your Temporal Server version. Also check the upgrade rules: move the server one minor version at a time and run any schema upgrades the release notes call for before the server upgrade.

Cut over only when the evidence in the matrix supports it: list and attribute query results agree across an agreed test window, retention behavior matches the policy your compliance owners approved, and the new store has been stable under real query load. Keep the old store readable and writable long enough to roll back, and take backups or snapshots of it before retiring it. Retire it only after the agreed read-only period ends, the data you must keep is migrated or exported, and access to the old store is removed from service accounts, dashboards and automation. Record the final retirement date, the owner who approved it, and the evidence used, so auditors can trace how search history was preserved.