Does a Braintrust data plane satisfy your residency or isolation rule?
A security review, a data residency rule or a network isolation requirement has surfaced around LLM traces and evaluation data, and you need to know whether Braintrust's self-hosting answers it. Braintrust's Enterprise self-hosting keeps traces and datasets in your cloud, but the web UI, authentication and metadata remain in Braintrust's control plane, so teams with stricter rules must test that split against their own policy. This is a fit question, not a verdict on Braintrust. Start by writing down, for each data category, where it must sit and who may access it, then ask Braintrust to confirm in writing which components run in your environment and which call out to its services.
If the control plane split does not fit, the alternatives are fully self-hosted options that run entirely in your own infrastructure. Langfuse and Opik can both be deployed in-house. Langfuse's core features are MIT licensed, with some enterprise features such as audit logs and data masking requiring a license key, so check that the features your policy depends on are covered. Opik is Apache-2.0 licensed, and its README states the full platform, including the web application, can be self-hosted without a commercial license. Both need you to operate the storage and database layers, which shifts the residency question from Braintrust's architecture to your own operating model.
Before deciding, map the specific rule to the specific component. Confirm with Braintrust whether metadata such as project names, prompt text, user identifiers or access logs leave your environment, where authentication is processed, and which regions and subprocessors are involved. Then ask the same questions of any open-source option you shortlist, including where its database, cache and object storage run and who holds administrative access to them. A written answer from each vendor and a short test in a non-production environment will settle the question faster than the architecture diagram.
What a fully in-house constraint actually requires
A fully in-house deployment moves more than the storage layer inside your boundary. The cards below list what the reader must confirm about data location, the control plane, outbound traffic, operations, and audit before a security review can sign off on either option.
Where trace and dataset data lives
Traces, prompts, outputs, datasets and scores are the most sensitive content in an LLM stack, since they often contain customer records. Under a fully in-house model, every such record should land in a database and object store you provision in your own account or data center. Confirm the storage region, encryption keys, and backup location for each component, including ClickHouse and blob storage.
Where the UI and authentication run
A data plane deployment keeps the web interface, sign-in and user metadata in the vendor's control plane, so the login path itself sits outside your network. A fully in-house deployment runs the UI and identity flow inside your environment. Ask which login methods each candidate supports, and whether SSO, SCIM and role management need an enterprise license key.
What metadata leaves the boundary
Even when trace bodies stay local, project names, user identifiers, usage counts and billing records can still reach an outside service. List every field your reviewers consider personal or confidential, then test the deployed system with network capture to confirm what is actually sent. Treat any telemetry or update check as metadata that must be documented.
How egress is controlled
A self-hosted service that calls out to a control plane needs an explicit firewall rule, and an in-house deployment should not need one for normal operation. Define a default-deny egress policy for the application namespace, allow only the model providers your gateway uses, and verify that tracing still works with outbound access blocked. Record the policy as part of the architecture evidence.
Who operates the database and storage
Self-hosting moves operating responsibility to your team, including patching, backups, restores, upgrades and capacity for PostgreSQL, ClickHouse, Redis or Valkey, and object storage. Name the on-call owner for each layer and write down who approves database migrations during upgrades. Ask any vendor whether paid support is an option, and what it covers compared with the free open-source version.
How access is audited
Reviewers will ask who viewed or exported production traces, who changed prompts or evaluation rules, and whether those events are retained. Confirm which audit log features each candidate provides in its free edition, and which require an enterprise license. Where a feature is missing, plan to collect logs at the Kubernetes and database layers and send them to your existing SIEM.
Where the control plane sits and why reviewers ask about it
The split that matters in a Braintrust Enterprise self-hosted deployment is between the data plane and the control plane. The data plane is where trace payloads, datasets and scores are written and stored, which in this model is your cloud. The control plane handles the web interface, authentication and account metadata, and in Braintrust's documented self-hosted design these stay with Braintrust. Security reviewers need to see both halves, because a deployment can keep every prompt and response inside your network while still depending on an external service for who can log in, what projects exist and which roles apply.
Reviewers ask about the control plane even when trace payloads stay local because identities, project metadata and access decisions still cross the boundary. User email addresses, SSO assertions, project and experiment names, role assignments and the browser sessions that reach the interface can reveal customers, product plans or internal structure. Project names alone can be sensitive in regulated settings. A fully self-hosted option such as Langfuse self-hosting or Opik places the web application, authentication and metadata in the same network boundary as the data, so one architecture review covers the whole system and no external egress needs a separate justification.
Before comparing platforms, request Braintrust's written list of what the control plane stores, transmits and logs, including identity data, metadata fields, telemetry, support access, retention periods and the regions involved. Confirm in writing whether prompt or response content can ever appear in metadata, error reports or support bundles. Then map each item against your data classification policy, classifying user emails, project names and dataset names as the policy requires. Any item that your policy does not allow outside your boundary is a decision point: negotiate a change, accept it with a documented exception, or plan a move to a deployment where that data never leaves your infrastructure.
Which open-source platform fits a fully in-house requirement
This table compares Langfuse self-hosted, Opik self-hosted and Opik on Comet.com against a fully in-house requirement, where the control plane, traces and metadata all stay in your environment. Read the rows top to bottom: the first two options can meet the rule, while the hosted Opik service does not. The main takeaway is that the in-house options shift the operating work of ClickHouse, PostgreSQL, Redis or Valkey and object storage onto your team.
| Requirement | Langfuse self-hosted | Opik self-hosted |
|---|---|---|
| License | MIT for core code; ee directories under a separate enterprise license | Apache-2.0 for the full platform, including server, UI, tracing and evaluations |
| Where data and metadata live | Your PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage | Your ClickHouse, MySQL and Redis deployment, with your own storage |
| Core features without a key | All core features and APIs available without limits in the MIT version | Full platform, including tracing, evaluations and prompt management, self-hostable without a commercial license |
| Enterprise key features | RBAC roles, audit logs, SCIM, server-side data masking, data retention policies | Confirm with Comet which enterprise controls, if any, apply to your deployment |
| Operating data stores | ClickHouse, PostgreSQL, Redis or Valkey and object storage run by your team | ClickHouse, MySQL and Redis run by your team |
| Main operating trade-off | ClickHouse sizing, upgrades and disk headroom during major version migrations | Multi-store operations and Kubernetes Helm upkeep for production |
| Meets a fully in-house rule | Yes, when all components run in your own cloud account or data center | Yes, when self-hosted in your own infrastructure |
| Opik on Comet.com | Not applicable | No, the hosted service keeps traces and metadata outside your environment |
How to choose by security review, residency or air-gap situation
A security review that questions the control plane usually asks where the web UI, authentication and metadata run, who can administer them, and whether the vendor can reach your data. With Braintrust's Enterprise self-hosting, the web UI, authentication and metadata stay in Braintrust's control plane, so that conversation will center on the vendor boundary. A fully self-hosted deployment moves all of those components into your infrastructure, which is the natural starting point when the reviewers want every administrative path under your own control. Langfuse is the usual first candidate here because its self-hosted deployment runs the same codebase as Langfuse Cloud.
- Control plane location: Confirm that the login, project and metadata services run in your cluster, and document any outbound calls the deployment still makes.
- Enterprise features: Check which controls need an enterprise key in Langfuse, such as project-level RBAC, audit logs, server-side data masking and SCIM, and decide whether the review accepts their absence.
- Identity integration: Confirm whether SSO and SCIM are required, and test provisioning and deprovisioning against your identity provider.
- Licensing review: Ask legal to read the MIT and enterprise license boundaries before the review, since Langfuse's enterprise directories fall under a separate license.
- Evidence trail: Plan how audit logs and access records are collected and retained, and test that they are complete.
Before the review, confirm which features your policy requires and test each one in a staging environment.
A data residency rule usually names the countries or regions where trace data, prompts, datasets and evaluation results may be stored and processed. Hosted offerings can satisfy that rule only when the provider documents the regions it offers and where account and billing data go. LangSmith Cloud, for example, runs in GCP US (Iowa), GCP EU (Netherlands), GCP APAC (Sydney) and AWS US (Ohio), and keeps data in the region where the account signed up while billing information stays in the US. An in-house Opik or Langfuse deployment lets you choose the region directly, which is the natural starting point when the rule names specific locations and you want the storage, database and backup tiers pinned there.
- Region pinning: Choose a cloud region or data center for ClickHouse, PostgreSQL, Redis or Valkey, and object storage, and confirm each service stays inside it.
- Backups and replicas: Confirm where backups and any replicas are stored, since a residency rule often covers copies as well as primary data.
- Support access: Ask who can reach the system from outside the region, including vendor support for enterprise add-ons.
- Hosted service exclusions: Confirm whether policy excludes hosted services such as Comet.com for Opik, which means in-house installation is the only option.
- Export paths: Test whether exports such as LangSmith bulk export or Braintrust exports land in a bucket that the rule covers.
Before go-live, run a test trace through each region and confirm the stored data matches the rule.
An air-gapped or isolated network has no route to the public internet, so every container image, package, model and license check must be staged inside the boundary. Braintrust's hosted control plane cannot be reached from that network, so the question is whether the self-hosted open-source option you choose installs and runs without outbound calls. Do not assume offline operation from a feature list. Confirm the dependency chain for each candidate, starting with Langfuse, which needs PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage, and Opik, which uses ClickHouse, MySQL and Redis. Both are natural starting points, but the install process and any telemetry or licensing checks must be verified for your network.
- Image and chart mirroring: Mirror all container images and Helm charts into an internal registry, and confirm the exact versions your upgrade plan requires.
- Outbound calls: Ask each vendor to list any network calls made at install time and at runtime, including license validation.
- Storage backends: Confirm that the S3-compatible store and the database services run without internet access.
- Enterprise key handling: If a Langfuse enterprise key is needed, confirm how it is validated on an isolated network.
- Update path: Plan how patches and major upgrades are brought across the boundary, including database migrations.
Before go-live, run the installation and a full trace and evaluation cycle with the outbound route blocked to confirm it works.
How to deploy Langfuse or Opik entirely inside your network
This sequence takes a team from an approved target environment to a Langfuse or Opik deployment where traces, datasets, metadata and access controls all stay inside the organization's own infrastructure.
Confirm the target environment and data classification
Security, compliance and platform leads agree which cloud account, data center or air-gapped network will host the platform, which data classes the traces may contain (prompts, completions, user identifiers, retrieved documents), and which network zones can reach it. Platform engineering records these decisions in writing so that every later step can be checked against them. The step is done when the owner signs off the environment, the data classification and the list of applications that will send telemetry. Confirm with your security team whether prompts and outputs count as regulated personal data in your jurisdiction before any trace is sent.
Choose Kubernetes with Helm for production
Platform engineering selects the production path. Langfuse documents Kubernetes with Helm or Terraform-based deployments on AWS, Azure and GCP for production, and Opik publishes Helm charts for production as well. Docker Compose on a single VM is documented for testing only and has no high availability, so it should stay in a non-production environment. The step is done when the cluster, namespace, ingress design and chart version are chosen and recorded, and when the team confirms that the cluster sits inside the private network with no public ingress unless your policy allows it.
Provision the data stores inside the private network
The platform team provisions the supporting services in the same private network. Langfuse needs PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage, while Opik uses ClickHouse, MySQL and Redis. Langfuse's scaling guide lists minimum resources of 2 CPU and 4 GiB for web, worker and PostgreSQL, 1 CPU and 1.5 GiB for Redis or Valkey, and 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments. Treat these as floors and size from your own trace volume, retention period and concurrent users. The step is done when every service answers only on private addresses and backups are configured for PostgreSQL and object storage.
Restrict egress and configure SSO
The network team applies egress rules so the platform pods can reach only the data stores, the internal identity provider and approved internal registries. Any outbound route to a vendor endpoint, including telemetry or license checks, is reviewed and blocked unless a documented need exists. If your policy requires single sign-on, configure it using the authentication and SSO options described in the Langfuse documentation. Note that some access controls, such as project-level RBAC roles, data retention policies, audit logs and SCIM, need a Langfuse Enterprise license key. The step is done when a test connection from the platform to an external address fails and an SSO login succeeds against your internal identity provider.
Route SDK and OpenTelemetry traffic internally
Application teams point their SDKs or OpenTelemetry exporters at the internal endpoint instead of any hosted service. Langfuse and Opik both accept OpenTelemetry, and Opik offers native integrations and a @track decorator for custom code. Each application's configuration sets the host, keys and environment name, and secrets are stored in your existing secrets manager rather than in source code. The step is done when a test request from a workload inside the network appears in the platform UI with the expected project and environment labels.
Verify that no trace data leaves the network
Security engineers test the boundary with evidence rather than assumption. They run a sample workload that generates traces, capture egress logs from the firewall, proxy and DNS resolver, and confirm that the only destinations are internal services. They also review the platform's logs and environment variables for any external callback or analytics setting and document each one that is disabled. The step is done when the test report lists the observed destinations, the controls that enforce them and the owner who accepts the result. Ask your auditor which evidence format they expect, because DeployOpen prepares controls and evidence but independent assessors issue any opinion.
Plan upgrades with the documented migration guides
The platform owner schedules a repeatable upgrade process. Langfuse states that updates within a major version are non-disruptive and apply database migrations automatically at startup, while major upgrades follow dedicated guides. The v3 to v4 guide requires ClickHouse 25.12 or later (26.4 recommended), PostgreSQL 15 or later and Redis 7.0 or later, and advises about 3x ClickHouse disk headroom when backfilling historic data. Opik publishes its releases on GitHub, so the team should track them the same way. The step is done when a staging environment has been upgraded from a backup, performance and data checks have passed, and a rollback procedure is written down.
What security reviewers and regulators ask to see
Security reviews of an LLM observability platform tend to follow the same evidence trail, whether the vendor's control plane or your own stack holds the data. The cards below cover what reviewers request, where the artifacts come from, and where a Braintrust Enterprise self-hosted data plane leaves gaps that a fully self-hosted deployment must close. DeployOpen prepares these artifacts, but independent auditors and certification bodies issue any attestation.
Architecture and trust boundaries
Reviewers want a diagram showing every component, where each one runs, and where trust boundaries sit. For a Braintrust data plane, the split between your cloud, which holds traces and datasets, and Braintrust's control plane, which handles the web UI, authentication and metadata, must be drawn explicitly. For a fully self-hosted Langfuse or Opik deployment, the diagram covers the application containers, ClickHouse, PostgreSQL, Redis or Valkey or MySQL, and object storage inside one boundary. Ask Braintrust to confirm in writing which metadata fields leave your environment.
Data stores and their locations
A reviewer will ask which stores hold prompts, model inputs and outputs, scores, user identifiers and backups, and in which region each sits. Self-hosted Langfuse depends on PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage, so each one needs a named location, encryption setting and retention rule. Opik uses ClickHouse, MySQL and Redis. In a Braintrust data plane, confirm the region of the traces and datasets bucket and whether any copies exist in Braintrust's infrastructure.
Network egress rules and blocked calls
Reviewers increasingly want evidence that the platform cannot reach external endpoints it should not reach, not just a policy that says so. Collect the egress firewall or security group rules, the DNS policy, and test output showing that connections to unapproved hosts fail. A data plane that calls a vendor control plane over HTTPS needs an allow-list entry for that endpoint and a documented justification. A fully self-hosted deployment should show that model provider calls go only to approved gateways, with blocked calls logged.
Access control and SSO configuration
Show who can sign in, how roles map to projects, and how accounts are removed when people leave. Langfuse keeps single sign-on configuration in its documented authentication options, but project-level RBAC, audit logs and SCIM provisioning require an Enterprise license key. Export the identity provider settings, the role matrix and a sample of deprovisioning events. For a Braintrust data plane, ask which authentication flows run in Braintrust's control plane and which identity provider records they depend on.
Audit log samples and feature licensing
Audit logs are often the first thing a reviewer asks to sample, and they are where open-core licensing matters. In self-hosted Langfuse, audit logs sit behind an Enterprise license key, so a fully open-source deployment may not have them at all unless you buy that key. Opik's README states that the full platform can be self-hosted without a commercial license, so confirm which audit and access records it writes before you promise them. Keep dated samples of administrative actions, data exports and key rotations.
Backup, restore and upgrade records
Reviewers want proof that a restore has actually worked, not a backup schedule alone. Keep dated restore test results covering PostgreSQL, ClickHouse and object storage, with recovery time and data completeness recorded. Add the upgrade and patch log, including the Langfuse v3 to v4 migration, which requires ClickHouse 25.12 or later, PostgreSQL 15 or later and Redis 7.0 or later, and advises about 3x ClickHouse disk headroom for backfills. For a Braintrust data plane, ask who performs and records the upgrades of the parts you run.
What does moving to a fully in-house LLM observability stack cost and take?
What does a fully self-hosted Langfuse or Opik stack cost in licenses?
The open-source cores carry no license fee: Langfuse's repository is MIT licensed outside its ee directories, and Opik is Apache-2.0 with the full platform self-hostable without a commercial license. The license cost to check is an optional Enterprise key. For Langfuse, project-level RBAC, audit logs, data retention policies, server-side data masking and SCIM provisioning need a key, and its self-hosted pricing page describes that plan as custom-priced. Ask the vendor for a written quote covering exactly which of those features your security review requires, since that is the only recurring license line in a Langfuse deployment.
Which infrastructure components does a self-hosted LLM observability stack need?
A self-hosted Langfuse deployment runs Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. Opik needs ClickHouse, MySQL and Redis. Langfuse's scaling guide lists minimum resources of 2 CPU and 4 GiB for web, worker and PostgreSQL, 1 CPU and 1.5 GiB for Redis or Valkey, and 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments. Those are floors, not sizing. Your trace volume, retention period and number of evaluations drive the final footprint, and the infrastructure bill recurs every month in your own cloud account or data center.
Who has to operate ClickHouse, PostgreSQL and storage after go-live?
Your platform or data team owns that work after go-live: backups, patching, capacity planning, monitoring and upgrades of each stateful service. Braintrust's Enterprise self-hosting moves the control plane to Braintrust, so those operations sit on a different party, which is the main ownership difference to weigh. Upgrades need care: Langfuse applies database migrations automatically at startup within a major version, but major upgrades such as v3 to v4 require ClickHouse 25.12 or later, PostgreSQL 15 or later and Redis 7.0 or later, and advise about 3x ClickHouse disk headroom when backfilling history. Plan a named owner and an on-call route before you commit.
How long does an in-house LLM observability rollout take?
No researched duration exists for this work, so treat any figure from a vendor as a claim to verify against your own scope. Duration depends on the deployment target (Docker Compose on a single VM is for testing with no high availability, while production uses Kubernetes with Helm or Terraform), your identity and network controls, the number of applications to instrument, the volume of historic data to migrate and how many evaluators and dashboards you rebuild. Ask for a phased plan that lists those items separately, with the acceptance tests for each phase.
Can our own team run this stack without outside help?
Many teams can, provided they already run Kubernetes, managed PostgreSQL, a ClickHouse cluster and object storage, and have engineers who can read upgrade guides carefully. The gaps usually appear in backup and restore testing, capacity planning for trace volume, SSO and key management, and the first major upgrade. Assess those skills honestly against your roster before deciding. An outside partner can plan, implement, harden, back up and document the stack in your environment under a fixed-scope agreement, and retainers cover ongoing upgrades and support if you want that continuity.
How do we compare an in-house stack with a Braintrust Enterprise quote?
Put both options on the same list of components. For the in-house path, list the open-source license (none for the core), any Langfuse Enterprise key, infrastructure for each stateful service, the people who operate them, and implementation and support fees. For Braintrust, ask for the Enterprise contract terms in writing, including what the control plane keeps, what the data plane holds in your cloud, the support level and how overages apply. Compare recurring costs over the same period, and note that the Braintrust Pro figure is public but Enterprise pricing is set by contract, so request it in writing rather than estimating it.
How does a fixed-scope DeployOpen engagement get agreed?
Scope, deliverables and timelines are agreed in writing before work starts. DeployOpen plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. The written scope lists the components, the environments, the integrations and the handover items, and the quote follows from that scope. DeployOpen does not publish a price list. Ongoing support, upgrades and continuous testing are available as retainers, and DeployOpen prepares systems and evidence for compliance assessments without issuing audit reports or certifications.
How DeployOpen deploys Langfuse or Opik under your constraint
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a Braintrust replacement, that means running Langfuse or Opik entirely in your infrastructure so the control plane, traces and datasets stay under your governance, with the security evidence your reviewers need.
How we scope
- Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- The written scope names which data stays in your cloud, which identity provider and network paths apply, and which teams use the platform.
- We confirm with you whether Langfuse Enterprise license key features or Opik's Apache-2.0 feature set match your required controls.
How we build
- We deploy the chosen platform on Kubernetes with Helm or Terraform in your environment, or Docker Compose for a non-production test.
- We size and configure PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible storage, and document each component's ownership and backups.
- We run security testing of the deployed platform and its APIs, and prepare configuration and access evidence for your compliance assessment.
How we hand over
- You receive architecture, runbooks, backup and restore procedures, and an upgrade plan that follows the vendor's major-version migration guides.
- We document SDK and OpenTelemetry instrumentation changes so application teams can switch exporters with a clear reference.
- Handover includes a walkthrough with your operators so your team can run upgrades, scaling and incident response without us.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped separately from the initial build.
- Retainers can cover version upgrades, ClickHouse capacity reviews and periodic security testing as your trace volume grows.
- DeployOpen does not issue audit reports or certifications; we prepare the systems and evidence that independent auditors assess.
