Sending OpenTelemetry data to ClickHouse: when the exporter fits your stack
A platform team that stores logs, traces and metrics in self-managed ClickHouse still needs a way to get that telemetry out of applications and hosts and into the database. OpenTelemetry is not a storage backend, so ClickHouse holds the data while something else collects it. The OpenTelemetry Collector fills that role. It is a vendor-agnostic service that receives telemetry, processes it (for example, batching, filtering or adding attributes) and exports it to one or more destinations. Because it is tool-agnostic, the same collection path can keep working if the storage choice changes later.
The Contrib distribution of the Collector includes the ClickHouse exporter, which writes traces, metrics and logs into ClickHouse tables. Teams need this path when they want applications to speak OpenTelemetry (OTLP) and want one pipeline to own batching, retries and routing, instead of having every service write directly to the database. The exporter also decides which tables and schemas exist, so the platform team should read what it creates before the first production write. Confirm the exporter's current table definitions, supported signals and configuration keys against the Collector release you plan to deploy, because the component's stability level is documented in its own README.
Before committing to this path, check whether your sources can emit OpenTelemetry at all, and whether your team will own the Collector fleet. An agent deployed beside workloads and a gateway that accepts traffic from many sources are different operational units, and the choice affects authentication, batching and failure handling. The Collector also supports authenticator extensions for inbound and outbound connections, which matter when telemetry crosses network zones. Ask your delivery partner or internal team to confirm these choices against your own environment before the design is fixed.
Agent, gateway or both: choosing collector roles
In an agent-only design, a Collector runs beside each workload, usually as a sidecar container or a daemon on each host, and exports directly to ClickHouse. Every agent holds its own exporter queue and batch settings, so the number of writers grows with the number of hosts and pods. That is simple to reason about for a few services but harder to tune across a fleet.
- Where it runs: on each host, node or pod, close to the application that emits the telemetry.
- Batching: each agent batches its own data, so batches are smaller and more numerous unless you raise the batch size and flush interval on every agent.
- Retries: failed exports are retried from each agent's local queue. A host that is restarted or evicted can lose data still held in memory.
- Connections: each agent opens its own connections to ClickHouse, so connection count scales with the fleet size.
- Configuration: credentials, table names and exporter settings must be distributed to every agent, which raises change-management effort.
Before you commit, confirm how many agents will write to a single ClickHouse endpoint at peak and whether your ClickHouse servers accept that many concurrent inserts.
In a gateway-only design, applications send telemetry to a small set of central Collectors, and only those gateways export to ClickHouse. The gateways become the single place where batching, retries and credentials are configured. This keeps the ClickHouse connection count predictable but makes the gateway tier a shared dependency.
- Where it runs: in a dedicated pool of hosts or Kubernetes deployments, usually behind a load balancer.
- Batching: large, consistent batches are built centrally, which generally produces fewer and bigger inserts into ClickHouse.
- Retries: gateways hold the export queues, so their memory and disk sizing decides how long an outage in ClickHouse can be absorbed.
- Connections: the number of ClickHouse connections is set by the gateway replica count and exporter settings, not by the number of applications.
- Failure scope: a gateway outage affects every sender that depends on it, so the pool needs redundancy.
Confirm with your ClickHouse team which insert rate and batch size the cluster has been sized for, then match gateway replicas and queue settings to that figure.
A layered design places a lightweight agent on each host to receive local telemetry and forward it to a gateway tier, which performs the heavier processing and exports to ClickHouse. The agent absorbs short network or gateway interruptions locally, and the gateway concentrates batching and export. This is the most common shape for larger fleets, at the cost of running and monitoring two tiers.
- Where it runs: agents on every node, gateways in a separate pool that can scale independently.
- Batching: agents send small, fast forwards, and gateways build the large batches that ClickHouse receives.
- Retries: retries happen at two hops, so you must set queue and timeout values on both tiers to avoid duplicate or lost data.
- Connections: only gateway replicas open connections to ClickHouse, which keeps the connection count stable as the fleet grows.
- Operations: you must monitor both tiers, version them together, and test a gateway restart under load.
Check that your gateway pool can scale horizontally and that each tier exposes its own queue and export metrics before you rely on the layering.
A small team with one or two services and a single ClickHouse cluster usually gains more from one tier than from a full topology. A single gateway, or agents exporting directly for a very small set of hosts, is enough to prove the schema, batching and dashboards. The question is when to add the second tier, not whether to start with a design that scales.
- Start small: one gateway deployment with two replicas gives redundancy without a fleet-wide rollout.
- Use the same configuration: keep exporter settings in version control so that moving to a layered design later only adds agents.
- Watch the triggers: add agents when you need local buffering for hosts that lose gateway connectivity, or when gateway CPU or memory limits are reached.
- Keep the ClickHouse side fixed: the table design and batch sizes should not change when you add tiers.
- Plan the move: test the new topology in a non-production environment that receives representative data.
As a planning step, ask your team to name the single condition that would trigger the move to a second tier, and record it with the expected owner.
Exporter and pipeline options and when each fits
This table compares the main configuration areas of the Collector ClickHouse exporter pipeline and the trade-off each one carries. Read each row as a decision: what you set, when that setting fits your ingest pattern, and what it costs in latency, data safety or operational effort. Confirm every default against the component README for your Collector release before you rely on it.
| Configuration area | When it fits | Trade-off to plan for |
|---|---|---|
| Exporter endpoint and credentials | Send all signals to one ClickHouse endpoint from a gateway, with credentials kept out of config files and supplied from a secret store | Hard-coded or shared database users make access review and rotation harder; confirm the exporter's supported authentication options in its README |
| Batch processor settings | Large steady ingest where fewer, larger inserts reduce part creation pressure in ClickHouse | Bigger batches add delay before data is queryable and hold more data in memory if a collector fails before flushing |
| Retry and queue behavior | Pipelines where a short ClickHouse outage should not lose telemetry, such as production gateways | Persistent queues need disk space and monitoring; unbounded retries can hide a stuck destination until the queue fills |
| Table naming and schema creation | Greenfield deployments where the exporter may create tables it expects, so you can review DDL before production | Auto-created tables may not match your ordering keys, TTL or codecs; review generated schemas before go-live and plan migrations if they change |
| Authenticator extensions such as oauth2client and OIDC | Gateways that must verify callers with OIDC on receivers or present OAuth2 client credentials on exporter requests | Each extension adds identity provider dependencies and token lifecycle work; confirm which authenticators your ClickHouse access path supports |
| Agent versus gateway role | Agents for host-level collection with light processing; gateways for central batching, auth and fan-out to ClickHouse | Gateways concentrate risk and need horizontal scaling and load balancing; agents multiply configuration to maintain |
Which tables the exporter creates and who owns the schema
The ClickHouse exporter can create its tables on first run, so the first table definitions may come from the exporter's defaults rather than from a design your team reviewed. Those defaults fix the ordering key, partitioning, TTL and column codecs once data arrives. Changing an ordering key later usually means building a new table and copying data into it, which is slow on a large log or trace dataset. Decide these four settings before the first production batch, and treat the exporter's generated DDL as a draft to approve, not as the final schema.
Your team owns the schema, and the exporter should not be the only reviewer. Before the first run against a staging cluster, capture the generated CREATE TABLE statements and check them against your query patterns. Ordering keys should match the filters your engineers use most, usually service and time. The TTL should match your written retention policy for each signal, and codecs should suit the size of log bodies and attribute columns. The sibling topic on designing ClickHouse schemas for logs and traces covers these choices in depth, so use it as the reference for the design itself.
Confirm two versions before you rely on automatic table creation. First, check the exporter's README in the Collector release you deploy, since the Collector release line and the exporter's schema behavior are documented per component, and confirm whether tables are created only when missing or are altered on later starts. Second, confirm that the ClickHouse release in use supports the column types and codecs the exporter generates, using the ClickHouse production FAQ as the upgrade reference. Ask the exporter maintainers what happens to existing tables when a newer Collector changes the schema.
How to set up the collector pipeline to ClickHouse
This sequence takes a ClickHouse cluster and an OpenTelemetry Collector Contrib release from verified artifacts to a tested pipeline that writes telemetry into tables you can query. Each step closes with a check, so the rollout to production agents and gateways starts only after the data path has been proven.
Confirm the cluster and create a dedicated ingest user
Your ClickHouse team confirms the target cluster version, the database that will hold telemetry, and the Keeper and replica layout if the tables are replicated. They create a dedicated ingest user with rights limited to the database and the statements the exporter needs, and they restrict the default user as the access control documentation recommends. Confirm the version against the ClickHouse security policy before you go further. Done when a login as the ingest user succeeds, a test insert into a scratch table works, and the default user can no longer write to telemetry tables.
Choose a Collector Contrib release and verify it
Select the current OpenTelemetry Collector Contrib release from the official releases page, since the contrib distribution includes the ClickHouse exporter. Download the binary or image and the matching SHA256 checksum file, then verify the checksum and the signature on the artifact. Keep the SBOM with the release record for your security review. Done when the checksum matches, the signature verifies against the project's published method, and the binary reports the version you chose.
Build the pipeline in a non-production environment
Write a Collector configuration with the receivers your applications use, a batch processor, a memory limiter, and the ClickHouse exporter pointed at the test cluster. Store credentials in environment variables or a secrets store rather than in the file itself, and confirm the stability level of each component in its README before you rely on it. Start the Collector and watch its logs for startup errors. Done when the Collector starts cleanly, reports its pipelines as running, and the exporter logs no connection or authentication errors.
Send representative load through the pipeline
Use a load generator or a staging service to send telemetry that matches production in shape: the same signal types, attribute sizes, cardinality and burst patterns. Include traces, metrics and logs if all three will reach ClickHouse. Watch Collector metrics for refused or failed exports and for queue growth while the test runs. Done when the load completes without sustained queue growth or dropped data, and you have recorded the batch settings that worked.
Check table creation and row counts
Review the tables the exporter created, including their engines, ordering keys, partitioning and TTL settings, and compare them with your schema standards. Compare row counts in ClickHouse with the number of items your load generator sent, allowing for signals you filtered out on purpose. Check that timestamps and key attributes landed in the expected columns. Done when the table definitions match your review, and counts reconcile with the test load within the margin you set in advance.
Confirm retention, access and recovery before rollout
Verify that the ingest user cannot read or alter tables outside its scope, that the retention rules applied to the new tables match your policy, and that the tables are covered by your backup job. Run a restore of the test tables to a spare location as the backup documentation recommends, since a backup that was never restored may not restore when needed. Done when a restore succeeds and the retention and access checks have a written record.
Roll out agents and gateways in stages
Deploy agent Collectors next to workloads and gateway Collectors that receive from agents, in a staged order that starts with one host group and one gateway. Apply the same version and configuration you tested, and scale gateways by adding instances behind the same load balancer once throughput data shows the need. Keep the previous configuration available for rollback. Done when each stage shows steady export success, queues stay within limits, and ClickHouse insert latency remains stable across the rollout.
Scaling collectors for high ingest
A single collector is rarely the limit on ingest. The bottlenecks sit in batching, memory, the path between collectors and ClickHouse, and the load you have not yet tested. The cards below cover the controls that connect those pieces and the checks to run before go-live.
Scale gateways horizontally
Run several identical gateway collectors behind a load balancer so that each one receives a share of OTLP traffic. Stateless batching and export make this the usual scaling path, but tail sampling and any component that keeps state across requests need traffic affinity. Confirm which processors in your configuration hold state before adding replicas, and check how your load balancer distributes long-lived gRPC connections.
Tune batch size and flush interval
The batch processor groups telemetry before it reaches the ClickHouse exporter, so each insert carries enough rows to keep ClickHouse parts healthy. Larger batches reduce insert frequency but raise memory use and delay data visibility. Set the batch size and timeout together, then measure how often batches close on size versus on timeout under your real traffic.
Enforce the memory limiter
The memory_limiter processor should sit first in each pipeline so the collector refuses or slows incoming data before it runs out of memory. Pair it with a receiver-side response so senders retry instead of losing data. Check the collector's own memory metrics under load, and size the container limit above the limiter threshold so the limiter acts before the operating system kills the process.
Shard collectors by signal or tenant
Separate pipelines for logs, traces and metrics, or separate gateway pools for different tenants, stop a burst in one signal from stalling the others. Each pool can then have its own batch settings, memory limits and scaling rules. The cost is more deployments to operate and a need to route telemetry consistently, so decide the split before traffic grows rather than after.
Watch ClickHouse insert pressure
Many small inserts or bursts from many collectors create many small parts, which ClickHouse must merge in the background. Watch insert latency, error responses and the count of active parts on the target tables while you scale. If pressure builds, increase batch size, add buffering or reduce the number of concurrent exporter connections before adding more gateways, since more collectors can make the problem worse.
Load test before go-live
Replay realistic telemetry volumes, including the bursts your services produce during incidents, against a staging pipeline that matches production shape. Measure dropped data, retry rates, memory peaks and end-to-end delay from sender to query. Ask your team to define pass criteria for each measure in writing, and repeat the test after every change to batching, scaling or the ClickHouse schema.
Questions teams ask before sending telemetry to ClickHouse
What drives the cost and effort of sending OpenTelemetry data to ClickHouse?
Most of the effort is design, configuration and testing, and the largest recurring cost is the infrastructure that runs the Collectors and the ClickHouse cluster. The one-time work covers pipeline design, schema decisions, Collector configuration for each source, sizing, and a test plan. Recurring work covers operating the infrastructure, upgrading the Collector and ClickHouse, and tuning as volume grows. The main cost drivers are the number of telemetry sources, which signals you collect (traces, metrics or logs), peak ingest rate, retention period, and whether authentication, TLS and high availability are required from the first release. Ask for a quote that itemizes these drivers and separates fixed-scope build work from any ongoing support retainer.
How long does a first OpenTelemetry to ClickHouse pipeline take?
The duration depends on scope, and no fixed figure applies to every environment. A single application sending traces to one ClickHouse table is a much smaller job than a multi-team rollout with agents on every host, a gateway tier, authentication, and dashboards. The main variables are how many sources need instrumentation or configuration, whether you already run ClickHouse in production, whether the schemas the exporter creates are acceptable as they are, and how much load testing you require before go-live. Ask any proposal to state the phases, the environments it covers, the acceptance tests, and the dependencies on your team's approvals and access. Those items explain the timeline better than a single number.
What does the work need from our team?
The work needs named owners for the telemetry sources, the ClickHouse cluster, and the network path between them. On the source side, your teams need to confirm which services emit telemetry, which attributes matter, and what data may be stored. On the platform side, someone must provide ClickHouse hosts or a Kubernetes namespace, storage, DNS, certificates, and firewall changes. Security or compliance staff should review where telemetry may contain personal data, how long it is kept, and who may query it. Finally, someone must approve the test results and own the Collector configuration after handover. Agreeing these owners before the build starts usually removes more delay than any technical choice.
Can an in-house team set this up alone?
An in-house team with experience in OpenTelemetry and ClickHouse can build the pipeline alone, provided it has time, access, and someone accountable for the result. The OpenTelemetry Collector is open source under Apache-2.0, and its documentation covers receivers, processors and exporters in detail. The harder parts are usually operational: sizing ClickHouse for the ingest rate, designing table ordering and retention, securing connections, and proving that no data is lost under load. Stability also varies across Collector components, as the OpenTelemetry status page shows, so each component you depend on needs a check. Teams without that experience often choose a scoped engagement for design and testing, then run the pipeline themselves.
How do we choose between agent and gateway Collectors?
Use agents for collection close to each workload, and use gateways to centralize processing and export. The OpenTelemetry Collector can run as an agent next to services or as a gateway that accepts data from many sources. Agents suit host-level collection and keep local buffering near the source. Gateways suit sampling, enrichment, batching and authentication in one place, and they limit how many clients hold ClickHouse credentials. Many production designs use both, with agents forwarding to a gateway tier that scales horizontally. The deciding questions are how many sources you have, where credentials may live, how much processing must happen before storage, and how you will scale and upgrade each tier. Write those answers down before choosing the topology.
How do we test that telemetry arrives in ClickHouse intact?
Send known test telemetry through the full pipeline and compare what arrives with what was sent. Start by generating a fixed set of traces, metrics and log records with known identifiers, timestamps and attributes. Query ClickHouse for those records and check counts, timestamps, attribute mapping and field types against the source. Repeat the test at expected peak ingest, and while a gateway instance restarts or a ClickHouse node is unavailable, because those are the conditions that cause loss or duplication. Monitor the Collector's own health and exporter failure metrics during the test. Record the results and the configuration version they apply to, so the same test can run again after each upgrade.
How do we secure the connection from Collectors to ClickHouse?
Secure the connection with TLS, dedicated database users with narrow permissions, and credentials managed outside configuration files. The Collector supports authenticator extensions that can authenticate incoming connections on receivers, for example with OIDC, and that can add credentials to outgoing exporter requests, for example with the oauth2client extension. Pair these with a ClickHouse user that can only insert into the telemetry tables and that is not the default user. Restrict which hosts can reach ClickHouse, and log access to the telemetry data. Review the Collector configuration best practices before go-live, and confirm which authentication options your ClickHouse version and exporter setup support, since that can differ by release.
Which Collector version should we run, and how do we keep it current?
Pin a specific Collector release in every environment, test it, and upgrade on a planned schedule. The OpenTelemetry Collector v0.163.0 was released on October 16, 2026, following v0.162.0 on September 29 and v0.161.0 on September 16, 2026. Collector releases ship the contrib distribution with signed artifacts, SHA256 checksums and SBOM files, which help with verification and supply chain reviews. Check the release notes for each component you use, because stability levels differ between components. Keep the ClickHouse exporter configuration in version control so an upgrade can be reviewed and rolled back. Confirm the support window for your own release channel before you commit to a cadence.
How DeployOpen scopes and delivers the ClickHouse pipeline
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, and an OpenTelemetry-to-ClickHouse pipeline is scoped the same way. The work runs in your cloud accounts, data centers or restricted networks, so the Collector fleet, the ClickHouse cluster and the Keeper nodes stay under your control. Engagements are normally fixed-scope: the written scope names the Collector roles (agent, gateway or both), the receivers and processors in each pipeline, the ClickHouse exporter settings, the tables the exporter creates, and the test and acceptance criteria. Work starts only after that scope is agreed in writing, and a quote follows from it. DeployOpen does not publish a price list, so ask for a quote tied to each line of that scope.
Before a scope is signed, confirm the facts that change the design. Check the exact exporter and Collector versions you intend to run against the release notes for the contrib distribution, since the Collector ships on a regular cadence and each component carries its own stability level. Confirm where batching and retry settings will live, how authenticator extensions will secure receivers and exporter credentials, and whether your ClickHouse cluster is already replicated through ClickHouse Keeper. Each of these answers changes effort, so ask for them to be recorded as assumptions in the scope document rather than discovered during the build.
Ongoing work is handled through a retainer when you want continuity after go-live. Typical retainer items for this pipeline include Collector version upgrades, reviews of exporter and batching settings as ingest grows, checks that backups of the ClickHouse tables still restore, and responses to changes in your telemetry sources. Retainer scope, response expectations and reporting are agreed in writing, and they should be compared with what your own team can cover. DeployOpen does not claim partner status or certifications with any vendor or standards body for this work, and it does not issue audit reports or certifications. Where you need evidence for an assessment, DeployOpen prepares the controls and documentation, and the independent assessor confirms them.
How DeployOpen works on OpenTelemetry to ClickHouse pipelines
DeployOpen plans, builds, documents and supports self-hosted OpenTelemetry Collector pipelines that send telemetry into ClickHouse inside infrastructure you control. Each engagement starts from a written scope covering collector roles, batching, table schemas and ingest capacity, so your team knows what is delivered before work begins.
How we scope
- Map your telemetry sources, signal types and expected ingest volume before choosing agent or gateway roles.
- Agree in writing which collector roles, exporter settings and ClickHouse tables are in the engagement.
- Confirm the Collector Contrib distribution version and ClickHouse server version your environment will run.
How we build
- Configure the ClickHouse exporter batching, retry and queue settings against your measured event rates.
- Verify the tables the exporter creates, and document any ordering key or TTL changes made afterward.
- Test scaling of gateway collectors under representative load in a pre-production environment with realistic data.
How we hand over
- Deliver collector configuration files, deployment manifests and the pipeline diagram under version control.
- Provide runbooks for adding collectors, rotating credentials and recovering from a failed ClickHouse endpoint.
- Document the ingest tests performed, their results and the settings your team should review on each upgrade.
How we support
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
- Ongoing support, collector upgrades and continuous testing are available as a retainer.
- Confirm with us how changes to Collector releases and ClickHouse versions will be tested before they reach production.
