Grafana with ClickHouse for logs and traces

Why teams point Grafana at ClickHouse for logs and traces

A team that already runs Grafana and has OpenTelemetry data landing in ClickHouse can query that data directly from Grafana by installing the Grafana ClickHouse plugin, which lets them build log and trace views over tables they already operate. ClickHouse documents this pattern for visualizing observability data such as logs and traces collected with OpenTelemetry, so the team keeps one storage layer and one dashboarding tool instead of running a second log or trace store just for viewing. This page covers the plugin and the Grafana side of that setup. It does not cover ClickStack, the separate observability stack with its own UI, which is discussed in its own sibling topic.

The decision in front of the team is less about whether the plugin works and more about what they are signing up to run. Grafana is a separate service with its own upgrade cadence, its own configuration database and its own authentication model, so the team should confirm which Grafana edition, database and login method they will use before they connect it to production tables. The Grafana installation documentation says SQLite does not scale for production, so a production deployment needs MySQL or PostgreSQL. The team should also confirm which Grafana release line they will pin, since the upgrade guidance describes minor and patch cadences that each carry their own support windows.

Before going further, check three things in your own environment: that the OpenTelemetry collector writes to tables the plugin can reach with a read-only ClickHouse user, that network paths from Grafana to ClickHouse are open and encrypted, and that the query role Grafana uses cannot read tables it should not see. ClickHouse access control supports users, roles and row policies, and the documentation recommends restricting the default user, which starts with all permissions. Those controls matter more once a dashboard tool sits between many viewers and the same tables.

What the plugin's log and trace views do for each audience

During an incident, the on-call engineer needs to move quickly from a symptom to the failing requests. The ClickHouse Grafana plugin, documented in the ClickHouse observability guide for visualizing OpenTelemetry logs and traces, lets that engineer run SQL-backed log and trace queries from Grafana panels and Explore. The work is mostly time-bounded filtering by service, severity and trace ID, then pivoting from a log line to its trace. Expect the queries to be written or adjusted by hand, because the views you see depend on how your tables are named and shaped.

Time-bounded log search: Filter by service, severity and a short time window before widening the range, so queries stay selective on large tables.
Log to trace pivot: Confirm whether the version in use can link a log row to its trace ID and open the matching span tree.
Span detail: Check that span attributes, durations and status fields display as expected for your OpenTelemetry schema.
Saved queries for runbooks: Store the common incident queries as panels or links so responders do not rebuild them under pressure.
Query cost awareness: Ask the platform team which columns are in the ordering key, since filters on other columns scan more data.

Before relying on these views in an incident, confirm the exact log and trace capabilities against the plugin version you run and rehearse a sample incident in a non-production environment.

How wide tables shape query performance in Grafana

Grafana panels turn every refresh into one or more ClickHouse queries, so the table design and the dashboard design share the cost. The cards below are the drivers to test against representative logs and traces before a dashboard goes to a wider team.

Column count and attribute maps

OpenTelemetry resource and span attributes can fan out into many columns or sit in a Map column. Wide schemas make each table part larger on disk, while Map keys are read through the column that holds them. Test the dashboard queries that touch the most attribute keys, and confirm which attributes the Grafana panels actually select rather than reading the whole row.

Ordering key for service and time

The ordering key decides which granules ClickHouse can skip. A key that starts with time suits range scans across all services, while a key that starts with service name suits per-service investigations. Choose the order from the filters your dashboards apply most often, and compare query plans on a copy of real data before fixing it.

Time-range filters in dashboard queries

Grafana passes the dashboard time range into every query through macros or variables. A panel that is set to a seven-day range scans far more parts than a one-hour panel on the same table. Check that each panel's time filter is on the timestamp column that leads the ordering key, and that no panel quietly ignores the range.

Dashboard refresh rates

A panel that refreshes every few seconds reruns its query at that rate for every open browser tab. Long dashboards with many panels multiply that load. Set refresh intervals per dashboard by audience, such as live incident views versus weekly review, and confirm the refresh interval is not shorter than the time it takes the slowest query to finish.

Concurrent users

Several engineers opening the same incident dashboard can run identical queries at the same moment. ClickHouse handles this with its own concurrency and memory limits, so peak use matters more than average use. Load test with the expected number of simultaneous viewers, and agree with the platform team which settings profiles and quotas apply to the Grafana service account.

Materialized columns

Materialized columns compute a value at insert time, such as a service key or a parsed status code, so that panels do not recompute it on every read. The trade-off is extra storage and a schema change to maintain. Identify the expressions that appear in most panel queries, then measure whether precomputing them removes work from the slowest panels.

Which Grafana hosting and configuration options fit your team

This table compares a single Grafana server on SQLite, a production server on MySQL or PostgreSQL, and an active-active layout behind a load balancer. Read each row against your user count, uptime needs and how many people will build dashboards and alerts over ClickHouse telemetry. The main takeaway is that SQLite is documented as unsuitable for production, so any shared investigation workflow needs an external database from the start.

Option	Fits when	Trade-off	Watch for
Single server on SQLite	A small trial or a single engineer exploring logs and traces in a non-production environment	Grafana's installation docs say SQLite does not scale for production; small deployments under 25 concurrent users are sized at 2 cores and 2 to 4 GB of RAM	Moving off SQLite later means migrating configuration, dashboards and users, so plan the database early
Production server on MySQL 8.0 or PostgreSQL 12 or later	Shared dashboards and alerting for a team, with a single Grafana node accepted for now	You operate one more stateful service (the database) with its own backups and upgrades	Confirm supported database versions against the current installation docs before you build
Active-active high availability	Several teams depend on Grafana during incidents and a node outage must not block investigations	Two or more Grafana servers behind a load balancer sharing MySQL or PostgreSQL, with database-backed auth tokens so users reach any node	Alerting needs extra configuration to avoid duplicate notifications, and Grafana Live has limitations in HA
Package installs (Debian, Ubuntu, RHEL, Fedora, SUSE, macOS, Windows)	Grafana runs on virtual machines or bare metal that your team already patches and monitors	Your team owns OS packages, service units, reverse proxy and upgrade timing	Pin the version you test and confirm the upgrade path before rolling to production
Docker image or Helm chart on Kubernetes	Your platform team already runs Kubernetes and wants declarative, repeatable deployments	Chart values, persistence, secrets and ingress become part of your cluster configuration	Check the chart version against the Grafana release you intend to run, and keep database settings in version control
Provisioned data sources and dashboards	You need the ClickHouse data source and log or trace views defined the same way in every environment	Provisioning files are loaded at startup, so changes go through a review step instead of the UI	Decide early which objects are provisioned and which teams may still edit in the UI
SSO through LDAP, OAuth or JWT (all editions)	Your identity provider is LDAP, generic OAuth, Entra ID, Okta, GitHub, GitLab or Google, or you use an auth proxy	SAML, team sync, active sync and SCIM provisioning need Grafana Enterprise or Grafana Cloud	Confirm which identity features your compliance scope requires before choosing the edition
Grafana Cloud for the Grafana layer	You want Grafana hosted by Grafana Labs while ClickHouse stays in your infrastructure	Your ClickHouse connection and query traffic cross into a vendor-hosted control plane, so network and data path reviews are needed	Check the pricing page for tier limits and fees; this option is not a substitute for a self-hosted review

When ClickStack's UI is the better fit than Grafana

Choose ClickStack's HyperDX UI over Grafana when the people who investigate logs and traces are not already Grafana users, and when your organization does not need one tool for dashboards, alerting and telemetry exploration. This is a judgment about team needs and how many systems you want to operate, not a claim that one interface returns queries faster than the other. Both read the same OpenTelemetry data in ClickHouse, so the question is who works in the tool every day and what else depends on it.

Start with the audience. Engineers who triage incidents from a trace view and a log search may find a purpose-built interface quicker to learn than a general-purpose dashboarding tool. Platform and SRE teams that already maintain Grafana dashboards, alert rules, contact points and data source provisioning will get more value from extending that estate, because their existing folders, permissions and on-call routing stay in one place. Ask each group which tool they open first during an incident, and which tool owns alerting today. Those answers usually settle the choice more clearly than any feature comparison.

Count the operational cost of each path before you commit. Grafana adds a server to run, and the Grafana installation documentation points production and high availability setups toward MySQL or PostgreSQL for configuration storage. ClickStack brings its own deployment and state components, which the sibling page on self-hosting ClickStack covers. Whichever you pick, confirm the version, upgrade cadence and authentication model with your own platform team and the vendor documentation before you standardize, and write down which tool will be the system of record for alerts. Running both for the same users usually doubles access reviews, upgrades and incident runbooks without adding capability.

Setting up the Grafana ClickHouse plugin against your cluster

This sequence connects Grafana to the OpenTelemetry log and trace tables in your self-managed ClickHouse cluster through a read-only account. Each step ends with a check, so the team knows the connection works before anyone builds dashboards on it.

Confirm the OpenTelemetry tables exist
The OpenTelemetry Collector pipeline, or the ClickHouse exporter in it, creates the log and trace tables that Grafana will query. Ask the team that owns ingestion which database and table names they use, then list the tables in that database from a ClickHouse client. Confirm that recent rows have arrived for both logs and traces within the time window you expect. The step is done when you can name every table Grafana will read and see fresh rows in each one.
Create a read-only ClickHouse user
Using the SQL-driven access control workflow, a ClickHouse administrator creates a dedicated user for Grafana with a role that grants SELECT only on the observability database. Add row policies if some services or tenants must stay hidden, and attach a settings profile that limits the account's resource use. The access control documentation also recommends restricting the default user, which starts with all permissions. Check by logging in as the new user, running a SELECT that succeeds, and confirming that an INSERT or DDL statement is refused.
Install Grafana with a production database
The platform team installs Grafana from a supported package, Docker image or Helm chart, depending on its hosting standard. Grafana stores its configuration in SQLite by default, but its installation documentation says SQLite does not scale for production, so point it at MySQL 8.0 or later, or PostgreSQL 12 or later, from the start. Plan high availability separately if you need two or more active-active servers behind a load balancer. The step is done when the service starts, the login page loads on your internal address, and Grafana reports a healthy database connection.
Install the ClickHouse plugin
Install the ClickHouse data source plugin on the Grafana servers using the method in the plugin documentation for your Grafana version. Review the plugin's release notes, confirm it supports the Grafana version you installed, and record both versions in your change record. The ClickHouse observability guide describes how the plugin visualizes logs and traces collected with OpenTelemetry. The step is done when the ClickHouse data source type appears in the Grafana connections list and a test connection can be created in the UI without errors.
Define the data source in provisioning files
Write a data source file in the provisioning directory, which Grafana loads at startup, and set the ClickHouse host, port, database and the read-only user's name. Supply the password through an environment variable or your secrets manager, and keep it out of the file itself. Provisioning is the as-code path, so keep the YAML in Git. Read the provisioning documentation for the exact file layout your version expects. The step is done when the data source passes its test after a restart and appears with a provisioned status.
Verify log and trace queries return expected rows
Open Explore, select the ClickHouse data source, and run a log query and a trace query over a window you already know contains data. Compare the row counts and a sample timestamp with the same query run directly in ClickHouse, and confirm that trace spans show their service and parent fields. Run one wide time range query to see how the cluster responds, and note which filters the team must add to keep it responsive. The step is done when the Grafana results match the direct ClickHouse results for both signals.
Store the setup as code
Commit the provisioning files, the role and user definitions, and the Grafana configuration to your Git repository, and review changes through the same process as other infrastructure. Grafana also documents Terraform and Git Sync as further as-code options, so choose the one your platform team already runs. Check by building a fresh Grafana instance from the repository alone, then confirming that the data source and the test queries return the same results as the original.

Securing Grafana access to ClickHouse telemetry

Grafana sits between your people and the telemetry stored in ClickHouse, so access has to be controlled on both sides of that connection. The cards below cover the ClickHouse account Grafana queries with, how users authenticate to each system, the Grafana web hardening settings, and the sensitivity of the log and trace attributes that both tools will display.

A dedicated read-only ClickHouse user

Create a separate account for Grafana instead of reusing the default user, which starts with all permissions. Grant it only the SELECT rights on the observability database it needs, through a role, and attach row policies where teams should only see their own services. Then check that the account cannot create, alter or drop tables. Confirm the exact grants against your ClickHouse version, since the access control documentation describes the SQL-driven workflow in detail.

ClickHouse external authenticators

Decide whether engineers reach ClickHouse through the database's own user list or through a central identity source. ClickHouse supports LDAP directories, Kerberos, SSL X.509 certificates, an HTTP authenticator and JWT authentication as external authenticators. Each option needs its own test in your environment, including how group membership maps to ClickHouse roles and what happens when a directory account is disabled.

Grafana sign-in methods by edition

Grafana's authentication options depend on the edition. LDAP, generic OAuth, Entra ID, Okta, GitHub, GitLab, Google OAuth, auth proxy and JWT are available in all editions. SAML, team sync, active sync and SCIM provisioning are available only in Grafana Enterprise and Grafana Cloud, so confirm your identity requirements against the edition you plan to license before the design is fixed.

Secure cookies and SameSite settings

Grafana's hardening guidance recommends secure and SameSite cookies, which protects the session when Grafana is reached over HTTPS behind a proxy or load balancer. Check that the cookie settings match your TLS termination point, and that a session cannot be replayed from an unencrypted path. Test sign-in, sign-out and session expiry after any change to the reverse proxy.

Content Security Policy and hidden version

Grafana's hardening guidance also covers a Content Security Policy, Trusted Types, hiding the version from unauthenticated users and domain verification against DNS rebinding. Set the policy in a test instance first, because dashboards and plugins that load external scripts or images can break. Confirm which plugins your dashboards need before tightening the policy.

Sensitive attributes in log and trace data

OpenTelemetry logs and traces often carry request paths, user identifiers, query text and headers that were never meant for wide audiences. Anyone who can open a Grafana log panel can see whatever ClickHouse returns, so filter or mask sensitive attributes at the Collector or in the table design, and restrict row policies by team. Ask the application owners which fields are personal data before you grant broad access to the telemetry.

Questions teams ask before running Grafana on ClickHouse

Is running Grafana on ClickHouse a fixed-scope project or an ongoing service?

It can be either, and the split is usually clear once the scope is written down. The initial build (data source connection, log and trace views, authentication, provisioning and handover) is normally fixed scope, with deliverables and timelines agreed in writing before work starts. Ongoing work such as Grafana and plugin upgrades, dashboard additions and continuous review is usually better handled under a retainer. Ask for the two parts to be quoted separately so you can see what is one-time and what recurs.

What drives the effort of a Grafana and ClickHouse build?

Four factors shape most of the effort. The data model comes first: how logs and traces are laid out in ClickHouse determines how many queries, column mappings and data source settings the Grafana views need. The number of dashboards and their query load comes next. Authentication integration is third, since Grafana supports several identity methods and some are limited to Enterprise or Cloud editions. High availability is last, because running two or more active-active Grafana servers requires a shared MySQL or PostgreSQL database, a load balancer and alerting changes to avoid duplicate notifications. Ask for each factor to be priced or sized separately in the quote.

Can our in-house team set up Grafana on ClickHouse without a partner?

Yes, many teams do, provided they have engineers who know ClickHouse table design and Grafana administration. The ClickHouse Grafana plugin and the observability guide cover the basic connection and log and trace views. The harder parts are usually the schema decisions, production database and HA setup for Grafana, and identity integration. A useful self-check is whether your team can answer three questions: which columns the trace and log views need, where Grafana's configuration database will live, and who owns the group mappings. If any answer is unclear, a short assessment before committing to a build is worthwhile.

What does the work need from our team?

Plan for three inputs. First, read access to the ClickHouse schema and sample queries, so the views match how your telemetry is actually stored. Second, named owners from your identity team for SSO, group mapping and any role rules, since these determine who can see which data. Third, a representative test dataset, ideally a copy of recent logs and traces with realistic volume and attribute variety, so query performance on wide tables can be measured before go-live. Confirm access routes and approval steps early, because they often set the schedule more than the build itself.

Does Grafana's 2021 license change matter for our deployment?

It matters mainly for how you distribute or host Grafana, and your legal team should confirm the position. On April 20, 2021, Grafana Labs relicensed Grafana, Loki and Tempo from Apache License 2.0 to AGPLv3, while plugins and SDKs kept other licenses. Grafana Enterprise is offered as a non-AGPL commercial option. For an internal deployment that you run for your own staff, many organizations proceed with the open-source edition, but whether AGPLv3 obligations apply to your modifications or any offering to outside users is a question for counsel. The ClickHouse plugin's license is separate and should be checked on its own. Also note that SAML and team sync are Enterprise or Cloud features.

How should we plan upgrades for both Grafana and ClickHouse?

Treat them as two release tracks with different cadences. Grafana major releases arrive once a year in April or May, minor releases every other month, and patch releases at least monthly. Each minor release is supported for 9 months, and the last minor of a major version for 15 months. ClickHouse stable releases ship roughly monthly, and its security policy lists which versions receive security updates, so check that list before choosing a line. Read the changelogs before each move, since minor backward-incompatible changes can occur. Keep a pre-production environment with representative data, and test the Grafana plugin version against the ClickHouse version you run.

When should we use ClickStack's UI instead of Grafana?

Choose ClickStack's UI when your team wants a purpose-built observability interface on top of ClickHouse with less assembly, and choose Grafana when you already run Grafana dashboards, alerting and data sources across other systems. Grafana gives you one place for metrics, logs and traces from many backends, plus a mature provisioning model. ClickStack is a narrower stack with its own deployment and state requirements. Many teams evaluate both against the same test dataset and their existing on-call workflow. Ask a partner to document the trade-off for your environment, including which team will own each tool.

What the Grafana license change and release cadence mean for your plan

On April 20, 2021, Grafana Labs relicensed Grafana, Loki and Tempo from Apache License 2.0 to AGPLv3, while plugins, SDKs and agent software kept other licenses, and Grafana Enterprise became available as a non-AGPL commercial option. For a ClickHouse log and trace setup, the first decision is which edition your organization needs. Confirm whether AGPLv3 terms are acceptable for how your staff will use and, if you modify it, distribute Grafana, and whether the Grafana ClickHouse plugin's license terms fit your policy. Check the Grafana licensing page and record the answer with your legal or procurement team before the build starts.

The edition question also drives authentication. The Grafana authentication docs list LDAP, generic OAuth, Entra ID, Okta and other methods as available in all editions, but SAML, team sync, active sync and SCIM provisioning only in Enterprise and Grafana Cloud. If your identity provider requires SAML or group-based access to ClickHouse log and trace views, that requirement may settle the edition for you. Confirm your identity needs against the current edition matrix before you plan the Grafana rollout, since changing editions later is a migration task in its own right.

The upgrade guidance sets a predictable rhythm: major releases once a year in April or May, minor releases every other month, and patch releases at least monthly. Each minor release is supported for 9 months, and the last minor of a major version for 15 months. Build a recurring window for minor upgrades on that cadence, and test each one in a staging Grafana that points at a copy of your ClickHouse data source configuration. Grafana stores its own state in MySQL or PostgreSQL for production, so back up that database before every upgrade and keep provisioned data sources and dashboards in version control so a rebuild after a failed upgrade is routine.

How DeployOpen scopes and delivers Grafana on ClickHouse

DeployOpen plans, builds, documents and supports Grafana connected to ClickHouse for logs and traces inside your own infrastructure. Each engagement starts with a written scope that names the data sources, dashboards, users and hosts involved, so the work and its boundaries are agreed before any build begins.

How we scope

Inventory the OpenTelemetry log and trace tables, their ordering keys and the query patterns your teams actually run.
Define which Grafana instance, database backend and ClickHouse user roles the deployment will use and who can reach them.
Agree in writing which dashboards, log and trace views, and alerting resources are in scope, and which are left for later.

How we build

Configure the ClickHouse data source, log and trace views and correlation settings as provisioned files so they can be reviewed and repeated.
Test query performance on wide tables with representative data and time ranges, and record which panels need tuning before go-live.
Set up Grafana high availability with a MySQL or PostgreSQL database, and confirm alerting avoids duplicate notifications across nodes.

How we hand over

Deliver the provisioning repository, configuration notes and a runbook covering upgrades, plugin updates and database backups.
Document the access model, including authentication options available in your Grafana edition and how role mapping is handled.
Walk your operators through a restore test of the Grafana database and a dashboard recovery from provisioned files.

How we support

Ongoing support, version upgrades and continuous testing are available as a retainer, scoped separately from the initial build.
Plan Grafana upgrades against the release and support windows published by Grafana, with a staging check before each change.
Confirm with your team which ClickHouse and Grafana versions are supported at the time of upgrade, since support windows change.

Using Grafana with ClickHouse for logs and traces