Self-hosting ClickStack on Kubernetes

What self-hosting ClickStack on Kubernetes involves

Self-hosted ClickStack means running the open source ClickStack observability stack in your own Kubernetes cluster, installed from its Helm chart, so logs and traces are stored in infrastructure your team controls. A team needs it when telemetry must stay in its own environment, when it wants to set retention and capacity itself, or when a hosted backend does not fit its data governance. The stack has three parts. ClickHouse holds the telemetry and answers queries over it. The HyperDX UI, along with the state it keeps for users and settings, runs alongside ClickHouse in the same cluster. OpenTelemetry supplies the data, with Collectors receiving logs, traces and metrics from applications and infrastructure and exporting them to ClickHouse. Each part has its own storage needs, failure modes and upgrade cadence, so the install plan has to cover all three.

The licensing question frames the choice. The ClickHouse server is licensed under the Apache License 2.0, according to its ClickHouse repository on GitHub, and ClickHouse Inc. sells ClickHouse Cloud as a separate managed service. Self-hosting ClickStack means your team operates the whole data plane. That includes the ClickHouse cluster, its ClickHouse Keeper nodes (the replication guide recommends at least three, on dedicated hosts in production), backups, version upgrades and the UI's state store. These duties recur, because ClickHouse stable releases ship roughly monthly. Choose self-operation only if your engineers can own that cycle, and compare the ongoing staffing cost against a managed service quote before you commit.

Which deployment options fit which team

This table compares the five choices a team makes before installing ClickStack with its Helm chart: ClickHouse topology, Keeper placement, MongoDB hosting, Collector roles and ingress with SSO. Read each row for when the option fits, what it asks of your operators, and what to confirm in the chart documentation before you commit.

Decision	Option and when it fits	Operational effort	What to confirm in the chart docs
ClickHouse topology	Single node fits evaluation and pre-production with representative data. Replicated ReplicatedMergeTree fits teams that need replica recovery for logs and traces.	Single node is lowest effort but offers no replica failover. Replicated adds Keeper, replica monitoring and upgrade coordination.	Whether the chart deploys replicas and Keeper, and how replicated table DDL is applied on install.
Keeper placement	Dedicated hosts fit production, since the replication guide recommends at least three Keeper nodes on dedicated hosts with about 4 GB of RAM each. Shared nodes fit only non-production clusters.	Dedicated hosts add machines to patch and monitor. Shared placement lowers hardware count but couples Keeper latency to ClickHouse load.	Whether Keeper pods can be pinned to separate nodes, and the node count the chart defaults to.
MongoDB state	Managed MongoDB fits teams with an existing database service and backup policy. In-cluster MongoDB fits teams that want the whole stack in one Kubernetes environment.	Managed service shifts patching and backups to the provider. In-cluster needs your own backup, restore testing and storage operations.	Supported MongoDB versions, persistence settings, and how connection strings are passed as Secrets.
OpenTelemetry Collector roles	Agent only fits small estates where each node sends data directly. Agent plus gateway fits larger estates that need central batching, filtering and credential handling.	Agent only is simpler to run. Agent plus gateway adds a second tier to scale, configure and monitor.	Which Collector modes the chart deploys, and whether the ClickHouse exporter settings are exposed as values.
Ingress and SSO for HyperDX	Without SSO fits a lab or a network-restricted pilot. With SSO fits teams whose access must follow an existing identity provider.	SSO requires TLS certificates, identity provider configuration and group mapping. Skipping it leaves local accounts to manage.	Supported authentication options, TLS secret names, and whether ingress is created by the chart or must be supplied.

Sizing ClickHouse for logs and traces in your cluster

Node sizes for ClickHouse should follow from measured ingest and query load, not from the chart's defaults. The main drivers are the ingest rate of logs and traces, the retention period, how well the log text compresses, the number of concurrent queries from dashboards and alerts, and the number of replicas that must hold each part of the data. Storage grows roughly with ingest multiplied by retention and the replica count, while memory pressure rises with query concurrency, large aggregations over wide time ranges and merge activity. Ask your ClickStack and ClickHouse specialists to show how each driver maps to disk, CPU and RAM in your cluster before you commit to node counts.

Load a pre-production environment with data and queries that resemble production before you fix node sizes. The ClickHouse production guidance recommends this step and advises checking the changelog before upgrades, because minor backward-incompatible changes can occur. Replay a representative sample of OpenTelemetry traffic at your expected peak, run the dashboard and alert queries your teams actually use, and record disk growth per day, merge behavior and query latency. Repeat the test with retention set to the period you need, since storage needs grow with it and are easy to underestimate.

ClickHouse Keeper coordinates replicated tables, so plan it separately from the data nodes. The replication guide recommends at least three Keeper nodes, on dedicated hosts in production, with about 4 GB of RAM each. Confirm that the Helm chart's resource requests and limits for the ClickHouse and Keeper pods match these figures, and that the chart does not place Keeper on the same nodes as data pods. Also confirm how the chart handles MongoDB and the HyperDX UI sizing, because those components have their own load profiles and are not covered by the ClickHouse guidance.

How the two-phase Helm install runs, step by step

The sequence moves from confirming the Kubernetes target and preparing cluster prerequisites, through a first phase that brings up ClickHouse and MongoDB, to a second phase that adds the UI and collectors, ending with an end-to-end ingest and query check. Each step closes with a check you can record before moving on.

Confirm the Kubernetes version and support line
Your platform team checks the cluster's Kubernetes minor version against the currently supported release line on the Kubernetes releases page, and confirms it sits within the version skew rules for the kubelets already running. The output is a written note of the cluster version, the supported lines and the upgrade path if the cluster is behind. Do not proceed if the control plane is on a line that is out of support, because the operator and chart may not be tested against it. Confirm the exact compatibility matrix for the ClickStack chart version you plan to install with the chart documentation.
Prepare namespaces, storage classes and secrets
Platform engineers create dedicated namespaces for ClickStack, ClickHouse, MongoDB and the collectors, define the storage classes the chart will reference, and create secrets for database credentials and the HyperDX configuration through your approved secret store. Confirm that the storage class supports the volume sizes and access modes the chart expects, and that Secrets are encrypted at rest in etcd or sourced from an external store. The check is a list of namespaces, storage classes and secret names that the install values file references, with no secret values stored in Git.
Install the operator and prerequisite components
The platform team installs the ClickHouse operator and any other prerequisite components the chart documentation lists for the version being deployed, such as custom resource definitions or certificate tooling. Install them into their own namespaces with the versions pinned in your values or Helm lock file. The check is that the operator pod is running and its custom resource definitions are registered, which you can confirm with kubectl before any ClickStack resources are created.
Run the first install phase
Using the phase one commands and values from the chart documentation for your version, the operator runs the first Helm release, which creates the ClickHouse cluster and the MongoDB state store that HyperDX uses for application data. Your team reviews the rendered manifests with helm template before applying them. The check is that the release reports deployed and the ClickHouse and MongoDB resources exist in the expected namespaces, even if they are not yet fully ready.
Verify ClickHouse and MongoDB are healthy
Before the UI is deployed, confirm that every ClickHouse pod and every MongoDB pod is ready, that ClickHouse answers a simple query over its HTTP or native interface, and that MongoDB accepts an authenticated connection with the credentials stored in your secret store. Check that persistent volumes are bound and that the ClickHouse data directories sit on the storage class you planned. The check is a short written record of readiness, query response and volume binding for each component. If any of these fail, resolve them before phase two, since the UI depends on both.
Run the second install phase for the UI and collectors
Apply the phase two release from the chart documentation, which deploys the HyperDX UI and the OpenTelemetry collector components that receive and forward telemetry. Set the UI's exposure and authentication settings according to your ingress design before this step, so that the UI is not reachable from an unintended network. The check is that the UI and collector pods are ready, the ingress or service resolves inside your network, and the collector endpoints listed in the chart output accept connections from a test client.
Confirm the UI can sign in and reach the data stores
Sign in to the HyperDX UI with the initial administrator account created by the chart, then change the default credentials and bind the UI to your identity provider if your access model requires it. From the UI, confirm it can reach the ClickHouse and MongoDB endpoints it was configured with. The check is a successful login by a named account, a changed initial password and a connection status for each backing store that matches the phase one output.
Check ingest and run a test query end to end
Send a known set of test logs and traces through a collector using a sample client, then confirm the records appear in the ClickHouse tables the exporter writes to by querying them directly, and again through the HyperDX UI. Compare the record count, timestamps and service names with what the test client sent. The check is a documented pass with the test identifiers, row counts and the query used, which becomes the baseline for later upgrades and sizing reviews. Ask your team to confirm the expected table names with the chart documentation for the installed version.

Replicas, Keeper and backups to decide before go-live

ClickStack stores its log and trace data in ClickHouse, and the recovery story depends on decisions made before the first production ingest. Replication, Keeper quorum, backup destinations, restore testing and control plane etcd backups each protect against a different failure, so none of them substitutes for the others.

Replication for node loss

ReplicatedMergeTree tables keep copies of each part on more than one replica, and ClickHouse Keeper coordinates them. This protects against losing a single ClickHouse node, not against bad data written to every replica. Confirm the replica count per shard, the table engines your ClickStack schemas create, and whether a recovered replica resyncs in your test environment before you rely on it. Keeper should run on at least three nodes, on dedicated hosts in production, with about 4 GB of RAM each.

Replication does not stop human error

A replicated table faithfully copies a bad DROP, a faulty migration or a corrupted part to every replica. Replication therefore cannot be the only protection for trace and log history that your investigations depend on. Decide who may run destructive statements against the observability cluster, and plan a separate backup copy that sits outside the replication path. Confirm which databases and tables hold ClickStack data so that the backup scope matches them.

BACKUP and RESTORE to a chosen target

ClickHouse BACKUP and RESTORE write tables, partitions or whole databases to local disk, S3-compatible storage or Azure Blob Storage. Choose the target by how far it sits from the cluster, since a backup on the same hosts shares their failure domain. Confirm the object store's own durability, retention and access controls, and whether its credentials are stored in a secrets system rather than in configuration files.

Full and incremental schedules

ClickHouse supports both full and incremental backups, so you can keep a periodic full copy and capture changes between them. Decide the full backup cadence, how many incremental chains you keep, and how old backups expire. Estimate storage growth from your ingest rate in your own environment and ask whether incremental chains depend on a base backup that must be kept until the chain is retired. Record the schedule in the runbook used during an incident.

Scheduled restore tests

The ClickHouse backup documentation warns that a backup that has never been test-restored will probably not restore properly when needed. Schedule restores into a spare cluster, not the production cluster, and check row counts, the schemas ClickStack expects and sample queries against known data. Confirm who owns the test, how often it runs, what time-to-restore your team can accept, and what happens when a restore fails.

etcd backups for the control plane

The Kubernetes control plane stores cluster state in etcd, and losing that state can prevent recovery of the workloads running ClickStack even when ClickHouse data is intact. Kubernetes documentation requires a backup plan for etcd and uses etcdctl snapshot save for backups and etcdutl for restores. Confirm who takes snapshots, where they are stored, how often they are restored in a test, and whether your managed Kubernetes provider already handles this for you.

Where MongoDB state fits and why it needs its own backup plan

ClickStack keeps two kinds of data apart. Telemetry such as logs, traces and metrics lands in ClickHouse, while application state such as user accounts, saved searches and dashboard configuration lives in MongoDB. The two stores fail differently, grow differently and need different recovery steps. A ClickHouse backup restores event data but not the dashboards that explain it, and a MongoDB backup does nothing for the telemetry. Plan a separate storage, backup, restore test and upgrade process for MongoDB before the first install, and assign an owner to each.

The chart can deploy MongoDB inside the cluster, or the HyperDX components can point at a MongoDB service your organization already operates. An in-cluster deployment is quicker to stand up and keeps the stack self-contained, but your team then owns its persistent volumes, snapshots, patching and failover. An existing managed or centrally run MongoDB service inherits the backup, monitoring and access controls your database team already applies, at the cost of a network dependency and a change request for each database user or upgrade. Decide this with the team that will carry the pager, and check that the chosen option can be restored into a separate environment.

Before you install, confirm with the chart documentation and the ClickStack release notes which MongoDB versions the chart supports, and whether it will accept the upgrade path your database team uses. Confirm how the chart creates and stores MongoDB credentials, whether they are plain Kubernetes Secrets, how they are referenced from values, and whether you can supply them from an external secret store. Finally, ask how a restore is tested: restore the MongoDB backup into a spare namespace, start the HyperDX UI against it, and check that users, sources and dashboards come back intact. Repeat that test on a schedule, not only once.

Securing the HyperDX UI, ClickHouse access and telemetry ingest

ClickStack exposes three access surfaces: the HyperDX UI, the ClickHouse database behind it, and the OpenTelemetry Collector endpoints that receive telemetry. Each needs its own authentication, TLS and secret handling, and the cards below set out what to decide and confirm for each before the stack carries production data.

Signing in to HyperDX

Decide how engineers and analysts authenticate to the HyperDX UI before anyone else is given a URL. Confirm in the chart's current documentation whether HyperDX supports SSO through your identity provider or only local accounts, and if local accounts are the only option, plan how they are created, rotated and removed when people leave. Check who can see logs and traces across all services, since the UI is the broadest read path into your telemetry.

Restricting the default ClickHouse user

ClickHouse's default user starts with all permissions, so the first hardening step is to restrict or disable it and create named accounts instead. Use the SQL-driven access workflow to define users, roles, row policies, settings profiles and quotas, giving the UI and the collectors separate identities with only the tables and write rights they need. Confirm that the HyperDX connection and the exporter use these accounts, not the default user.

External authenticators for ClickHouse

ClickHouse can delegate authentication to LDAP directories, Kerberos, SSL X.509 certificates, an HTTP authenticator or JWT tokens. Choose the method that matches your directory and service identity model, and confirm which of these your ClickHouse version and deployment configuration support before committing to it. Test that a removed directory account loses access to ClickHouse, not only to the UI, because the two sign-in paths are separate.

TLS between components and at ingress

Telemetry and credentials cross several hops: browsers to the ingress, the UI to ClickHouse, collectors to ClickHouse, and collectors to any gateway. Decide where TLS terminates at each hop, whether traffic inside the cluster is encrypted, and who issues and renews the certificates. Confirm the certificate chain the ingress presents and which certificates ClickHouse and the collectors trust, so that a renewal does not silently break ingest.

Kubernetes Secrets and etcd

Kubernetes stores Secrets unencrypted in etcd by default, and the ClickStack chart's database passwords and API keys would sit there unless you act. Configure encryption at rest for etcd or move the values to an external store through the Secrets Store CSI Driver. Restrict list and watch access to the namespace with least-privilege RBAC, since anyone who can read Secrets can read every ClickHouse credential in it.

Collector authentication on ingest

Collector receivers accept telemetry from applications and agents, and an open endpoint lets any workload write into your log store. Use authenticator extensions such as OIDC on receivers to check incoming connections, and the oauth2client extension to attach credentials to requests the exporter sends to ClickHouse. Confirm which authenticator components are in your Collector distribution, since component stability varies and is documented per component.

Questions teams ask before self-hosting ClickStack

What does self-hosting ClickStack cost in effort, and which costs recur?

Most of the cost is people time and infrastructure, and only part of it is one-time. The one-time work covers architecture, the Helm-based install, sizing, identity and UI hardening, and validation. The recurring work covers running ClickHouse, MongoDB and Kubernetes, applying upgrades, watching backups and handling on-call. Cost rises with node count, retention period, ingest volume, the number of backup targets and the number of environments you keep. Ask any quote to separate one-time build work from recurring support, and to list the infrastructure you must pay for directly, such as nodes, storage and object storage for backups.

How long do the install and validation take?

The duration depends on the scope and on how ready your environment is, so no fixed timeline applies to every team. The main drivers are whether a Kubernetes cluster with storage, ingress and identity already exists, how much log and trace volume you must sustain, how long you keep data, and whether you need a pre-production environment with representative data and queries. Validation takes longer than the install itself, because it includes load tests, restore tests and access checks. Ask for a written schedule that lists each phase and the inputs you must supply before each one starts.

Can an in-house team deploy ClickStack without a partner?

Yes, a team with Kubernetes, ClickHouse and identity experience can usually run the install itself, and the upstream documentation is the starting point. The harder work is operational: sizing for real ingest, keeping replicas and Keeper healthy, testing restores, and keeping the UI behind your access model. Check whether your team has at least one person who can own each area and whether they have time for upgrades and on-call. A partner is most useful for a fixed-scope build with a written deliverable list, or for a retainer that covers upgrades and testing.

What must our team provide before the work starts?

You provide the target environment and the decisions that only your organization can make. Typically this includes a Kubernetes cluster that meets your distribution and security standards, persistent storage classes, a network path for ingest from your collectors, an identity provider for the HyperDX UI, an object storage target for backups, and a sizing target that states expected events per second, retention per signal and query patterns. Without the sizing target, node counts and storage are guesses. Confirm each item in the written scope so that gaps are visible before the build begins.

How do ClickHouse upgrades and Kubernetes upgrades interact?

They must be planned together because each one can affect the other's compatibility window. The ClickHouse production FAQ recommends stable releases for most production use, a pre-production environment with representative data, and reading the changelog before upgrading, since minor backward-incompatible changes can occur. Kubernetes publishes its own skew policy: kube-apiserver is upgraded one minor version at a time, before the other control plane components and kubelets, and kubelets may be up to three minor versions older. Ask your provider how they stage the two upgrades, which Kubernetes versions they test against, and how they roll back a failed step.

Does running ClickStack on our cluster protect us from a compliance audit?

No. A self-hosted platform can support a compliance assessment, but it does not pass one by itself. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those. For ClickStack, the evidence usually includes access control records for the HyperDX UI, encryption and secret handling for Kubernetes, backup policy and tested restores, and change records for upgrades. Ask what evidence will be delivered and which controls each item supports.

Why does access to the HyperDX UI and secrets need planning up front?

Telemetry often contains credentials, personal data and internal service details, so access must be designed before go-live. Kubernetes stores Secrets unencrypted in etcd by default, so you should configure encryption at rest, restrict list and watch access with least-privilege RBAC, or use an external store through the Secrets Store CSI Driver. For ClickHouse, the access control documentation recommends the SQL-driven workflow and restricting the default user, which starts with all permissions. Confirm how users sign in to the UI, which roles can read which data, and where the database credentials are kept.

Limits to confirm with the vendor and in your environment

A ClickStack install touches several moving parts: the chart, the ClickHouse release it pins, the Kubernetes version under it, the OpenTelemetry components it ships, and the sizing that sets your ingest ceiling. Each one has its own support window and its own limits, so confirm them in writing during scoping rather than assuming they line up.

ClickStack and chart versions

Confirm which ClickStack release and Helm chart version are current, and which ones the vendor still supports. The operator-based two-phase install and the chart values change between releases, so a chart you copy from an older guide may expect a different install order or different keys. Ask for the exact chart version you will deploy and the upgrade notes between it and the next release. Record the version in your change log so later upgrades start from a known baseline.

ClickHouse release line in the chart

Find out which ClickHouse server version the chart pins and whether that line is listed as supported in the ClickHouse security policy. The policy lists 26.9, 26.8, 26.7 and 26.3 as receiving security updates and treats all other versions as unsupported. If the chart pins a line outside that list, plan a version override or an upgrade before go-live. Ask who patches the pinned image and on what schedule.

Kubernetes version and distribution

Check that the chart supports your Kubernetes minor version and your distribution, including the storage class, ingress controller and any managed service that sits in front of it. Kubernetes maintains only its three most recent minor releases, so a cluster on an older minor will need an upgrade first. Ask whether the chart has been tested on your distribution and which Pod Security Standards level the workloads require.

OpenTelemetry components in the chart

Compare the OpenTelemetry Collector version and distribution bundled with the chart against the collector standard you already run. Mixed collector versions can change receiver behavior, exporter schemas and authentication extensions. Confirm whether the bundled collector is contrib or a custom build, whether it uses your agent or gateway role, and how you will point existing services at it. Ask for the list of receivers and exporters the chart enables by default.

Ingest and retention limits of sizing

Ask the vendor for the sustained ingest rate, query concurrency and retention window that the sizing guidance assumes for logs and traces, and what hardware produced those figures. Then test your own event size, attribute cardinality and peak bursts against them. Retention is usually the largest storage driver, so confirm how TTL is set, whether it can change without a rebuild, and how much disk headroom merges and replication need.

Image mirroring for restricted networks

If your cluster has no route to public container registries, list every image the chart pulls, including ClickHouse, MongoDB, HyperDX, the collector and any init or job containers. Ask whether the chart lets you override each repository and tag, and whether image digests are published so you can verify what you mirror. Mirror and scan the images before the first install, and plan how they will be refreshed when security fixes ship.

How DeployOpen scopes, delivers and supports ClickStack

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, including self-hosted ClickStack on Kubernetes. Engagements start with a written scope that sets out the cluster, the ClickHouse and MongoDB sizing basis, the UI access model and the handover, so the work can be checked against it.

How we scope

Written scope names the Kubernetes distribution, cluster topology and namespaces that ClickStack will run in.
Scope records log and trace volumes, retention targets and the ClickHouse sizing assumptions used to plan storage.
Scope states who owns the MongoDB state, backups, identity provider and ingress decisions before the build starts.
Engagements are fixed-scope, so deliverables and timelines are agreed in writing before work begins.

How we build

Installation follows the operator-based two-phase Helm chart process, with each phase verified before the next starts.
ClickHouse replication, Keeper placement and backup settings are configured and tested as part of the build.
The HyperDX UI is secured with TLS, authentication and network restrictions defined in the scope.
Secrets are handled through the Kubernetes secret store approach agreed with your security team, not stored in values files.

How we hand over

Handover includes runbooks for install, upgrade, scaling, restore and access changes for your own team.
A restore test on a spare cluster is documented with its steps, timings and evidence for your records.
Architecture and configuration decisions are recorded so your engineers can explain why each setting exists.
Compliance-relevant evidence is prepared for your assessors, though audit reports remain the auditor's responsibility.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after handover.
Upgrade planning covers ClickHouse, Kubernetes and OpenTelemetry Collector version changes against the written scope.
Retainer scope, response expectations and service levels are confirmed in writing, so ask for them in your quote.
Security testing of the deployed stack can be added to the retainer or scoped as a separate engagement.