OpenTelemetry Operator for Kubernetes

The Target Allocator exists to spread Prometheus scrape targets across several OpenTelemetry Collector instances so that scraping keeps up as the number of targets grows. A single Collector running Prometheus receiver scrape jobs works well until the target count, scrape interval or series volume pushes one process past its CPU and memory budget. Before enabling the Target Allocator, decide whether your scrape load genuinely needs distribution or whether one well-sized Collector, with clear scrape configurations and resource limits, will remain adequate for the planning horizon you are designing for.

The decision comes down to load and operational cost. Keeping scraping in a single Collector means one configuration to review, one set of logs to watch and one scaling boundary to monitor. Distributing scraping means the Operator assigns targets to Collector replicas, which adds a second moving part: the allocator must discover targets, compute assignments and keep them consistent as pods come and go. Ask your team to measure current target counts, scrape durations and Collector memory under peak conditions, and to project growth over the period you plan to operate the system. That data, not a general rule, should decide the topology.

Configuration complexity is the first trade-off to plan for. The Target Allocator relies on the Prometheus receiver's scrape configuration, and the Operator needs the Collector custom resource to enable the allocator and to name the Collector deployment mode that suits the workload. Teams often underestimate how much of the existing Prometheus configuration must be translated, reviewed and tested. Service discovery is the second trade-off. The allocator reads targets from Kubernetes discovery roles or from the scrape configuration, so RBAC permissions, ServiceMonitor and PodMonitor resources, and namespace scoping all affect which targets appear. Verify that the service accounts involved can list and watch the resources they need, and confirm that no target is scraped twice or dropped silently.

Support is the third concern, and it depends on your exact Collector version. Receivers and scrape configuration options are not all covered by the allocator to the same degree, and the level of support can change between releases. Before committing, confirm with the Operator and Collector release notes which Prometheus receiver features, scrape settings and target discovery methods your version supports, and test the ones your monitoring depends on in a non-production cluster. Record the version you validated so that upgrades trigger a repeat of that check rather than an assumption.

Plan for failure modes before production. If an allocator or Collector replica is lost, targets must be reassigned without a long gap in metrics, so test pod restarts, scale-up and scale-down events and observe whether assignments converge. Monitor scrape success rates and target distribution across replicas so that an uneven assignment does not hide a saturated instance. Keep the single-Collector configuration as a documented fallback, so that if distribution proves unreliable for your workload, the team can return to a known state while the cause is investigated.

OpenTelemetry Collector releases arrive frequently, so your Operator-managed Collectors will need upgrades on a regular schedule. The recent releases in the project's release history illustrate this: v0.161.0 shipped on September 16, 2026, v0.162.0 on September 29, 2026, and v0.163.0 on October 16, 2026. The gaps between those dates were not identical, so plan your review process around release events rather than a fixed calendar day. Set a recurring owner for reading release notes, and decide in advance which changes trigger an out-of-cycle review, such as a security fix or a removed component.

Pin the Collector version explicitly in the image field of the OpenTelemetryCollector custom resource, and keep that manifest in version control. A floating tag such as latest can change the running Collector without any change to your manifests, so the upgrade would take effect on the next pod restart, outside your change process. Also confirm which Operator release supports the Collector version you choose, because the Operator and the Collector are versioned separately. Ask your team to document who approves each version bump, what evidence they need, and how the pinned version is recorded in your change tickets.

Test every version bump in a non-production cluster that mirrors production receivers, processors, exporters and the Target Allocator configuration. Run representative workloads long enough to observe steady-state behavior, then compare the Collector's own health and resource metrics before and after the change. Check that traces, metrics and logs still reach the backend, that dashboards and alert rules still evaluate, and that no pipeline has silently dropped data. Review the stability level of each component you use, which the project documents in each component's README, and treat any component marked as in development as a higher-risk item during upgrades.

Each release ships signed artifacts, SHA256 checksums and SBOM files, and your supply chain process should verify them before a version enters your registry. Compare the SHA256 checksum of every downloaded artifact against the published value, verify the signatures using the method your security policy requires, and store the SBOM alongside the version record so vulnerability scanning can match its components against advisories. If you build a custom Collector with the Collector Builder, generate and verify the SBOM for that build as well. Confirm the exact signing identities, verification commands and SBOM format in the release notes for the version you deploy, because those details can change between releases.

Before you roll out the OpenTelemetry Operator, confirm the limits that the Operator and its Collector depend on instead of assuming them. The first limit is component stability. OpenTelemetry Collector components have mixed stability levels, and each component's README documents its own level, so a receiver or processor you need may be stable while another in the same pipeline is still in development. Read the README for every receiver, processor, exporter and extension in your planned configuration, and record the stability level next to each one in your design document. Treat any component below stable as a decision that needs an owner, a fallback and a plan for upgrades.

Next, confirm which components the Collector distribution you choose actually includes. The Collector releases ship a contrib distribution, and a custom build can be created with the Collector Builder so that only the components you need are compiled in. The OpenTelemetry security guidance recommends building with only the components you need, which reduces the attack surface and the number of dependencies your security team must review. Ask your platform team which distribution the Operator will reference, list the components it contains, and compare that list against your pipeline design. If a component you need is missing, decide whether to add it through a custom build or to change the design.

Confirm version compatibility across three layers: the Operator, the Kubernetes version of each cluster, and cert-manager. The Operator depends on cert-manager to issue certificates for its admission webhooks, so the cert-manager version must be one the Operator release supports. Check the Operator release notes and the Kubernetes version support it declares against every cluster in scope, including non-production clusters, since drift between environments is a common source of failed rollouts. Also confirm who owns cert-manager upgrades and how certificate renewal is monitored, because an expired webhook certificate can block changes to Collector and Instrumentation resources.

Check language support for each auto-instrumentation target before you enable it. Trace APIs and SDKs are stable in most languages, including .NET, Go, Java, JavaScript, Python and others, but log SDK maturity varies more, and several languages still have logs support in development. Auto-instrumentation support differs by language and by library version, so list each application runtime and framework in your estate, map it to the instrumentation the Operator provides, and test the instrumented services in a non-production cluster. Applications outside that mapping will need manual SDK instrumentation or will remain uninstrumented, and your rollout plan should say which.

Confirm the backend's ingestion limits and the network path to it. OpenTelemetry is not an observability backend, so the limits that matter are those of the system that stores your telemetry, such as sustained ingestion rate, burst tolerance, cardinality limits and retention. Ask the backend owner for these limits in writing and size the Collector gateway, batching and retry settings against them. Finally, verify network policy between namespaces. Application pods must reach the Collector endpoints, the Collector must reach the backend, and the Target Allocator must reach the scrape targets, while unrelated namespaces stay isolated. Test each of these paths with your actual policies applied, because a permissive default in a development cluster can hide a blocked path in production.