How OpenSearch OpenTelemetry trace analytics works in practice
The supported path for trace analytics is OpenTelemetry instrumentation, then the OpenTelemetry Collector, then Data Prepper, then OpenSearch and OpenSearch Dashboards. OpenTelemetry generates and exports telemetry, but it does not store or visualize traces, so OpenSearch serves as the storage and analysis layer, and Dashboards is where your teams search spans and view service maps. Keep that division of labor in mind when you assign ownership, because each stage has its own configuration, its own failure modes and its own upgrade cadence.
In practice, each instrumented service exports spans over OTLP to a Collector. The Collector can run as an agent beside each workload or as a gateway that receives data from many sources, and most teams start with a gateway tier they can scale independently of the applications. From the Collector, spans move to Data Prepper, which shapes trace data into the documents that OpenSearch indexes and the trace views in Dashboards expect. Before you write any pipeline configuration, confirm the exact trace source and sink plugin names, their options and their supported versions in the Data Prepper documentation for the release you plan to run, because these names change between releases and examples copied from older posts often fail.
Several decisions shape the build before anyone writes configuration. Decide where sampling and attribute filtering happen, in the services, in the Collector or in Data Prepper, because that choice controls index growth and retention cost. Decide who may send spans to the Collector and how the Collector authenticates them, since Collector authenticator extensions can check incoming connections, for example with OIDC, and add credentials to outbound exporter requests. Decide whether Data Prepper runs as a separate tier or alongside the Collector, and plan index naming and retention for trace data separately from application logs so that trace retention can change without affecting other data.
Security and version planning belong in the design, not in a later hardening pass. Use TLS for every hop between services, the Collector, Data Prepper and OpenSearch, and give the Data Prepper pipeline a dedicated OpenSearch user with write access only to its trace indices. Give Dashboards users read roles scoped to those indices, and review whether span attributes contain personal or regulated data that needs field-level security or masking. Pin the Collector, Data Prepper and OpenSearch versions together, check each one's compatibility with the others in its release notes, and confirm how your chosen OpenSearch release is supported under the project's maintenance policy before you commit to a version.
What OpenSearch OpenTelemetry trace analytics needs before you start
Trace analytics depends on a chain of components, and each one has its own version, install method and stability profile. Pin these choices early, because the Data Prepper release depends on the OpenSearch version and the Collector configuration depends on the services you instrument.
OpenSearch and Dashboards versions
Choose the OpenSearch minor version and the matching OpenSearch Dashboards build first, because the Data Prepper release depends on them. OpenSearch 3.9.0 was released on September 29, 2026, and each version stays in maintenance for a limited period, so check the lifecycle on the releases page. Pick one install method now: Docker, Helm, the Kubernetes Operator, tarball, RPM, Debian, Ansible or Windows. That choice shapes how you upgrade the cluster and where you place certificates later.
Matching Data Prepper release
Data Prepper receives trace data from the Collector and writes it into OpenSearch, so its release must match the OpenSearch version you deploy. Confirm the pairing in the OpenSearch documentation for your target version instead of assuming the newest Data Prepper works with an older cluster. Record the chosen version in your change plan, and re-check the pairing whenever you upgrade either side.
Collector release and distribution
Choose the OpenTelemetry Collector release and distribution you will run, then pin it. Collector v0.163.0 was released on October 16, 2026, and each release ships a contrib distribution with signed artifacts, SHA256 checksums and SBOM files, along with the builder and OpAMP supervisor. Build a custom distribution only when the standard builds lack a component you need, because every added component adds a review and update obligation.
Instrumented services and SDK languages
List every service that will emit traces, along with its language and framework. Trace APIs and SDKs are stable in most languages, including C++, .NET, Go, Java, JavaScript, PHP, Python, Ruby and Swift, but confirm the status of each language you use on the OpenTelemetry status page before you rely on it. Note which services already emit spans and which need manual instrumentation, since that difference drives the effort and the test plan.
Collector component stability
Core Collector components have mixed stability levels, and each component's README documents its own status. List every receiver, processor, exporter and extension you plan to use, and record the stability level of each in your design. Components with less mature levels may change between releases, so add a compatibility review to every Collector upgrade. Include authenticator extensions, such as OIDC for receivers or oauth2client for exporters, in the same review.
Network paths and certificates
Map each hop before deployment: services to the Collector, the Collector to Data Prepper, Data Prepper to OpenSearch, and Dashboards to OpenSearch. Confirm which ports and firewall rules each hop needs in your network, and whether traffic crosses a zone boundary. Issue a certificate for every TLS endpoint, record who issues and renews it, and confirm that each client trusts the issuing chain.
How traces move from services into OpenSearch Dashboards
Trace data passes through five hand-offs between the application and a dashboard, and each one needs a named owner. The instrumented application creates spans through the OpenTelemetry SDK for its language. The SDK batches and exports them to a Collector. The Collector receives, processes and forwards them to Data Prepper, which shapes trace records into the documents OpenSearch stores. OpenSearch writes them to trace indices, and OpenSearch Dashboards reads those indices to draw service maps and span detail views. Before build work starts, record which team owns each hop. A missing attribute or a changed field name usually shows up first in a dashboard, far from the point where the data was lost.
The Collector can run in two roles, and the choice affects failure handling and operations. An agent runs beside each service, often as a sidecar or a node-level daemon. It keeps export close to the application and adds host or pod metadata where that information is known. A gateway is a shared Collector run centrally, which accepts data from many agents or directly from applications. Teams often start with agents where they need host-level enrichment, then add a gateway when they need one place to set sampling, routing and credentials. Some deployments run both, with agents forwarding to a gateway. Choose the layout based on your Kubernetes or virtual machine operations model, because each extra tier is another component to patch, scale and monitor.
Many teams place a gateway in front of Data Prepper so that application agents do not connect to it directly. The gateway can batch spans into fewer, larger requests, which eases load on the pipeline during traffic bursts. It is also a practical place to terminate TLS and enforce authentication on incoming connections. The Collector supports authenticator extensions that check credentials such as OIDC tokens on receivers, and the oauth2client extension can attach credentials to outgoing exporter requests. Confirm how your chosen Data Prepper version authenticates incoming trace requests before deciding which tier holds that control, so the check is neither duplicated nor skipped. Queue and retry settings on the gateway also determine what happens when Data Prepper is briefly unavailable, so test that behavior explicitly.
On the OpenSearch side, plan the trace indices before the first span arrives. Decide who may read span data, because spans often carry URLs, user identifiers and query text. The OpenSearch Security plugin provides role-based access control, document-level and field-level security and field masking, which can limit what each team sees in the same index. Dashboards views depend on the index mappings that Data Prepper writes, so a mismatch between the Data Prepper version and the index templates can leave service maps empty while raw spans still appear. Test one request path end to end: the span appears in Dashboards, the service map shows the expected edge, and a restricted user cannot see fields they should not. Confirm the exact pipeline topology and index mappings against the Data Prepper trace documentation for the version you deploy.
Configuring the OpenSearch OpenTelemetry trace analytics pipeline in sequence
This sequence connects one test service to OpenSearch Dashboards through the OpenTelemetry Collector and Data Prepper, with a check after each step. Each check confirms that data has moved one hop further, so a fault is found at the hop where it occurs.
Deploy or confirm the OpenSearch cluster and Dashboards
The platform team deploys or confirms an OpenSearch cluster and OpenSearch Dashboards with the Security plugin enabled, TLS on node and client traffic, and an authentication backend that matches your identity source. Confirm that dedicated cluster manager nodes and replicas on separate nodes are planned as the cluster documentation recommends. The step is done when an administrator signs in to Dashboards through the chosen backend and the cluster reports a healthy state.
Create the service account and index permissions
The security team creates a dedicated service account for Data Prepper and maps it to a role that only allows the index, template and write actions the trace pipeline needs, scoped to the trace index pattern. Confirm the exact permission names with the current OpenSearch security documentation for your version before you write the role. The step is done when the account authenticates, can write a test document to the trace index, and is denied writes to an index outside its scope.
Configure and start Data Prepper with a trace pipeline
The platform engineer configures Data Prepper with a trace pipeline whose sink is the OpenSearch cluster, using TLS and the service account credentials. Keep the password in a secrets store or environment variable rather than the plain pipeline file. Confirm the option names and endpoint formats in the current Data Prepper documentation. The step is done when Data Prepper starts without configuration errors, reports the trace pipeline as running, and authenticates to the cluster.
Configure the Collector receiver and OTLP exporter
The observability engineer configures the OpenTelemetry Collector with an OTLP receiver for incoming telemetry and an OTLP exporter that sends traces to the Data Prepper endpoint defined in the previous step. Check the receiver and exporter settings against the Collector component documentation for your release, since component stability levels vary. The step is done when the Collector starts and its logs show the exporter connected without repeated send failures.
Point one test service at the Collector
A developer instruments one test service with the OpenTelemetry SDK for its language and sets the OTLP endpoint to the Collector. Include a call to a second service so that a single request produces parent and child spans. Send a small set of requests from a known source so the expected traces are easy to recognize later. The step is done when the Collector logs show that it received spans from the test service.
Confirm trace documents arrive in the expected index
The platform engineer queries the trace index named in the Data Prepper pipeline and checks that the document count rises after the test requests. Confirm that documents contain the trace identifier, span identifier, parent span identifier and service name for both services. Compare the index name against the pipeline configuration rather than an assumed default. The step is done when the requests sent in the previous step are visible as span documents in that index.
Confirm service map and span views in Dashboards
The engineer opens the trace analytics area in OpenSearch Dashboards and confirms that the trace analytics app is enabled for your Dashboards version. Check that the service map shows the test service and the service it called, and that a trace opens with its spans in the correct parent and child order. Compare the trace count and durations with the requests you sent. The step is done when the service map and span views match the test traffic, which closes the pipeline.
Mapping span, service and resource attributes to OpenSearch fields
This table compares each OpenTelemetry trace concept with the field the Data Prepper trace pipeline and OpenSearch Dashboards expect, who decides its naming, and what to fix before indexing. Read it as a checklist for each attribute: confirm the field name against the Data Prepper and Dashboards trace documentation for your version, then decide whether the value needs renaming or enrichment. Most gaps appear in service names and resource attributes, so settle those first.
| OpenTelemetry concept | Expected OpenSearch field (confirm for your version) | Who owns the naming decision | Check or action before indexing |
|---|---|---|---|
| Trace ID | traceId in the trace document | OpenTelemetry SDK sets the value; platform team confirms the format | Confirm the format matches what the Dashboards trace views query on |
| Span ID | spanId in the trace document | OpenTelemetry SDK sets the value; no renaming expected | Confirm span IDs are unique within each trace |
| Parent span | parentSpanId, used to build the span hierarchy | SDK sets the value; service owners fix missing context propagation | Flag orphan spans caused by partial instrumentation or dropped headers |
| Service name | serviceName, taken from the service.name resource attribute | Service owners set service.name; platform team publishes the allowed naming standard | Flag inconsistent names such as case variants; enrich or rename in the Collector if needed |
| Other resource attributes | Resource attributes stored under the document's resource fields | Platform architecture team, with input from service owners | Decide which keys to index and avoid unbounded values such as request IDs |
| Span kind | kind field (for example, server or client values) | SDK sets the value; platform team confirms the values the pipeline expects | Confirm casing and enum values match the service map logic |
| Status | status code and status message fields | Instrumentation owners decide what counts as an error | Confirm error statuses map to the error counts shown in service maps |
| Timestamps and duration | startTime, endTime and duration fields, usually in nanoseconds | Pipeline configuration sets units; infrastructure team manages host clocks | Check time units and time zone handling, and measure clock skew between hosts |
Securing OpenSearch OpenTelemetry trace analytics end to end
Trace data passes through several components before it reaches a dashboard, and each hop widens who can see or change it. The cards cover transport encryption, authentication at the Collector, credentials on exporter requests, OpenSearch access control and audit, and removal of sensitive values before export. Work through them together, because a gap at any single hop exposes the same spans.
TLS on every telemetry hop
Encrypt each hop where trace data moves: from instrumented services to the Collector, from the Collector to Data Prepper, and from Data Prepper to OpenSearch. Each receiver and exporter has its own TLS settings, so a certificate that works on one hop can still fail on another. Check certificate chains, hostname validation and the certificate authority each component trusts, and confirm the option names against the Collector and Data Prepper versions you deploy.
Authenticating Collector receivers
Authenticator extensions can check incoming connections on Collector receivers, and OIDC is one option where the chosen distribution supports it. Decide which services must authenticate, how tokens are issued and rotated, and what happens when the identity provider is unreachable. Each authenticator must be confirmed for your Collector distribution, because extension availability differs between builds. Follow the Collector configuration best practices when wiring them in.
Credentials on exporter requests
The oauth2client extension can add credentials to requests the Collector sends to its exporters, so downstream endpoints receive authenticated traffic without a static secret copied into every service. Plan where client secrets are stored, how they are rotated, and which token endpoint and scopes the Collector requests. Confirm that the extension and exporter combination exists in your distribution, and test token expiry and refresh before production traffic depends on it.
Index roles for trace access
OpenSearch Security role-based access control decides which users and services can read which trace indices, including the indices that hold spans and service map data. Map roles to the teams that own services, and keep write permissions for Data Prepper separate from read permissions for analysts. Review the index patterns in each role, because a broad wildcard can expose spans from services that a team should not see.
Field-level limits for sensitive payloads
Span attributes can carry customer identifiers, tokens or query text. Document-level security limits which documents a role returns, and field-level security or field masking limits which fields it sees. Identify sensitive attributes before you write these rules, then test each role in OpenSearch Dashboards with sample traces, since saved views and aggregations can expose fields that a basic search would not show.
Audit logs and node and client TLS
Enable TLS for OpenSearch node-to-node and client traffic so that Data Prepper and Dashboards connections are encrypted inside the cluster as well as at its edge. Turn on audit logs, then decide where they are stored, who reviews them and how long they are kept under your retention policy. Confirm which audit categories your OpenSearch version supports and whether logs go to a separate destination or back into the cluster.
Why traces do not appear in OpenSearch Dashboards
This table lists the failures teams most often see when traces stop reaching OpenSearch Dashboards, the likely cause of each, a check to run first, and the fix. Work through the rows in the order data flows: Collector to Data Prepper, Data Prepper to OpenSearch, then indexed fields, versions and timestamps. Most problems show up in the logs at each hop, so check those before changing configuration.
| Symptom | Likely cause | Check to run | Fix |
|---|---|---|---|
| No traces arrive | The Collector exporter cannot reach Data Prepper because of a wrong endpoint, blocked port, DNS or TLS mismatch | Test connectivity from the Collector host to the Data Prepper endpoint, then check Collector logs for repeated export retries | Correct the exporter endpoint, open the network path, and confirm the Data Prepper trace source listens on that port |
| Pipeline runs but writes nothing | OpenSearch rejects the index write because the pipeline user lacks permissions or the TLS trust chain is wrong | Search Data Prepper logs for 401, 403 or certificate errors, then test the same user with a manual write to a trace index | Grant the pipeline role write and index creation rights on trace indices and configure the cluster CA in the sink |
| Trace index exists but service map is empty | Required fields such as service name, span ID or parent span ID are missing or named differently from what the service map expects | Pull a few span documents and compare field names and presence against the trace documentation for your Data Prepper version | Align resource attributes and Data Prepper processors so service name and span identifiers are populated on every span |
| Pipeline fails after an upgrade | Data Prepper and OpenSearch versions are not a supported pairing | Compare the Data Prepper release with the OpenSearch version in the release notes and compatibility information | Pin both components to a supported pairing and upgrade them in the order the release notes give |
| Collector connects but requests are refused | Authentication fails between the Collector and a receiver, often a missing, expired or mismatched credential | Check Collector logs for 401 or 403 responses and compare the authenticator extension on the receiver with the credential on the exporter | Configure a matching authenticator on the receiver and credential on the exporter, reload the Collector, and retest |
| Index grows fast and queries slow down | High cardinality attributes such as user IDs or request IDs are stored on every span | Count distinct values of each span and resource attribute in a sample and list those with very large value counts | Drop, hash or aggregate unbounded attributes in Collector processors and keep them out of indexed fields |
| Spans sort in the wrong order or durations look wrong | Clock skew between hosts, or timestamps in an unexpected format or time zone | Compare clock offsets on each service host with its NTP status, then inspect start and end times of a parent and child span | Sync all hosts to the same time source and fix timestamp handling so spans sort by start time |
Verifying OpenSearch OpenTelemetry trace analytics before dashboards are trusted
Before anyone builds a service map or a latency panel on this pipeline, run a fixed sequence of tests that proves each hop works and that access is limited the way your policy requires. The pipeline has four parts that can fail independently: the instrumented application, the OpenTelemetry Collector, Data Prepper, and the OpenSearch cluster that stores the trace indices and serves OpenSearch Dashboards. A dashboard that looks populated can still hide dropped spans, wrong timestamps or broken parent links, so the checks below should be written down and signed off before dashboards are shared.
Start with a known trace. Deploy a small test service with a distinctive service name that no production workload uses, send a single request with a known trace ID, and confirm that the trace ID appears in the trace index with the fields your dashboards depend on: service name, span name, start time, duration, status and parent span ID. Compare the stored values against what the test service emitted. Then confirm parent-child relationships by sending a request that crosses two or three services and checking that each child span points to the correct parent. Missing or reversed parent links produce misleading service maps, so this check matters more than a count of documents.
Next, run a load test at a span volume that resembles your production peak. The volume, duration and success thresholds must come from your own requirements, because no benchmark figures are supplied for this pipeline. Watch Data Prepper for backpressure, rejected writes and growing buffers, and watch the OpenSearch cluster for indexing rejections, queue growth and shard sizes. Decide in advance what counts as a pass, such as no unexplained span loss between the test service and the index, and record the Collector and Data Prepper settings used so the result can be repeated after a change.
Then test failure and recovery. Restart the OpenTelemetry Collector during a steady send and confirm what happens to spans in flight: whether the sender retries, whether any are lost, and how long ingestion takes to resume. Restart one OpenSearch node and confirm that the cluster returns to a healthy state, that no trace data is lost after replicas recover, and that Data Prepper resumes writing without manual intervention. Record which errors appear in logs during each restart, because those messages become the basis of your alert rules later.
Finally, verify access. Create a user with a restricted role that is permitted to read only specific trace indices, then sign in to OpenSearch Dashboards as that user and confirm that the user cannot see other indices, saved objects or service maps built on restricted data. Review the role mapping against your authentication backend, and confirm that document-level or field-level restrictions behave as your design says. Only after these checks pass should dashboards be shared, and the test results should be kept with the configuration so that any future upgrade of the Collector, Data Prepper or OpenSearch can be checked against the same sequence.
Questions teams ask before they connect OpenTelemetry traces to OpenSearch
What drives the cost and effort of an OpenTelemetry to OpenSearch trace pipeline?
Four factors set most of the effort. Services and languages: each service needs instrumentation, and trace SDK maturity differs by language, so a mixed estate takes longer than a single stack. Pipeline topology: agent or gateway Collectors, Data Prepper stages and the routing between them determine how many components you run and monitor. Security integration: TLS, authentication on receivers and exporters, and OpenSearch roles and tenant access add design and testing work. Cluster size: node roles, shard layout, retention and replicas drive hardware and operational work. Ask any partner to estimate each factor separately against your service count so you can see which one moves the quote.
Does running OpenSearch and the OpenTelemetry Collector require a license fee?
No license fee is described for the core software. OpenSearch is released under the Apache License, Version 2.0, and the OpenSearch FAQ does not describe a paid edition. The OpenTelemetry Collector repository is also licensed under Apache-2.0. Your recurring costs are the infrastructure that runs the Collectors, Data Prepper and OpenSearch nodes, storage for trace indices, backups, and the staff time to operate them. Check the license of any additional plugin, vendor distribution or managed service you add, because those terms are separate from the core projects. Confirm the terms of any component you plan to use with its publisher before you commit.
How long does an OpenSearch trace analytics implementation take?
Duration depends on scope, and no fixed timeline applies to every environment. A narrow pilot that instruments a few services in one language and sends traces to one OpenSearch cluster is a much smaller job than an estate-wide rollout across many languages, several Collector tiers and a security review of each data path. Schedule factors include how many services need instrumentation, whether the cluster already exists or must be built, how long identity and network approvals take in your organization, and how many environments such as development, test and production must be validated. DeployOpen works from a written scope and timeline agreed before work starts, so ask for the timeline in the proposal and confirm each dependency on your side.
What does our team need to provide for an OpenTelemetry to OpenSearch project?
Your team provides access, decisions and approvals that the work cannot proceed without. Access means read access to the services to be instrumented, deployment rights in the target environments, and a test environment that mirrors the production data path. For OpenSearch, you provide cluster capacity or the infrastructure to build it, storage and retention requirements, and the index and role model that governs who can query traces. For identity, you need a named integration owner and approval to create service accounts or client credentials. For network, you need firewall rules between services, Collectors, Data Prepper and OpenSearch, plus approval for certificate issuance. Name these owners at kickoff, because most delays come from waiting on approvals during the build.
Can an in-house platform team set up OpenTelemetry tracing into OpenSearch alone?
Yes, a team can, if it has the skills and time, although the work spans more disciplines than a single dashboard project. The build needs engineers who can instrument applications in each language in use, operate Collector configuration and pipelines, administer OpenSearch clusters including node roles, shard allocation and index lifecycle, and configure OpenSearch Security for authentication, role-based access and TLS. Someone must also own the views in OpenSearch Dashboards and check that span data supports the queries your engineers need. Compare your team against each of these areas before you start, and name who is on call for the pipeline. The build-or-partner decision depends on this skills gap and the capacity your team can commit over the project.
What security controls should we plan for trace data in OpenSearch?
Plan authentication, authorization, transport encryption and audit from the start, because traces often carry request metadata that identifies users or systems. The OpenSearch Security plugin supports SAML, OpenID Connect, Active Directory, LDAP, Kerberos and JSON Web Token authentication backends, so map your identity provider to a backend early. Role-based access control, document-level and field-level security, and field masking can limit who sees sensitive span attributes. TLS should cover node and client traffic, and audit logs record access. On the Collector side, authenticator extensions can authenticate incoming telemetry and add credentials to exporter requests. Review span attributes at instrumentation time, because removing sensitive values after they are stored is harder.
What changes when we upgrade the OpenTelemetry Collector or OpenSearch?
Each upgrade can change configuration, component behavior and compatibility, so both need their own test cycle before production. For the Collector, releases arrive often and component stability varies, so read the README for each receiver, processor and exporter you use and check for deprecated settings. For OpenSearch, new minor versions arrive on a regular cadence, and older versions move into maintenance, where they receive bug fixes and security patches but no new features. Test in a staging cluster, replay representative traces, confirm that dashboards and service maps still render, and keep a rollback path such as a snapshot taken before the change. For major OpenSearch upgrades, choose among rolling upgrades, snapshot and restore, remote reindexing and Migration Assistant, and confirm that your Collector and Data Prepper versions support the target.
How DeployOpen works on this
DeployOpen plans, builds and supports OpenSearch with OpenTelemetry for trace analytics inside your own infrastructure, whether that is your cloud accounts, data centers or air-gapped network. The work starts with a written scope that defines which services emit traces, how the Collector and Data Prepper pipelines are laid out, and who can read the resulting service maps and spans.
How we scope
- Inventory which services and languages will emit traces, and where each Collector agent or gateway will run.
- Define the Data Prepper pipeline stages, index layout and retention needs before any configuration is written.
- Agree access rules for OpenSearch Dashboards trace views, including roles, tenants and the Security plugin authentication backend.
How we build
- Deploy OpenSearch and OpenSearch Dashboards using the installation method your platform team already operates.
- Configure Collector receivers, processors and exporters, and test trace flow end to end with sample services.
- Verify that service maps and span views in Dashboards match expected service names, parent-child relationships and timestamps.
How we hand over
- Deliver the configuration, pipeline definitions, role mappings and network rules as version-controlled artifacts.
- Document the runbook for adding a service, rotating credentials, changing retention and reading pipeline health.
- Walk your operations and security engineers through the test results and the known limits of each component.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Plan OpenSearch and Collector upgrades against each project's own release stability, and confirm compatibility before rollout.
- Ask any partner to confirm the support scope, response expectations and upgrade responsibilities in the written agreement.
