Migrate from Braintrust to Opik

Migrate from Braintrust to Opik: what the move involves

Migrating from Braintrust to Opik means moving your experiments, datasets, traces and LLM-as-a-judge metrics out of a managed control plane and into a platform you run yourself. Teams usually start this work at a contract renewal, when a residency or control requirement cannot be met by the current deployment, or when the team wants to stop depending on a single vendor's data model. Platform or ML engineering normally owns the work, and security or compliance signs off on where traces and evaluation data will sit once the move is done. Opik's Apache-2.0 repository includes the full platform, including the server backend, web application, tracing, evaluations and prompt management, and the README states it can be self-hosted without a commercial license. The effort, however, depends on three things you can count before you start: how many applications and agents are instrumented, how much dataset and trace volume you hold, and how complex your judge prompts and scoring functions are. Ask Braintrust for an inventory of projects, datasets and scorers, and inventory your own instrumentation points, so the plan reflects real counts instead of estimates.

Operating Opik is the cost line most often underestimated. The Opik server needs ClickHouse, MySQL and Redis, so someone on your team must own backups, upgrades, capacity planning and recovery for all three, along with the Docker Compose or Kubernetes Helm deployment the README describes. Confirm who will be on call for that stack, what your backup and restore procedure is, and whether your hosting environment already has ClickHouse or MySQL experience. Judge calls also keep consuming model API spend on your own keys, which is a recurring cost that moves from the vendor's invoice to your own provider accounts. Before deleting anything in Braintrust, run both systems in parallel on the same traces and datasets, and compare the scores your LLM-as-a-judge metrics produce in each. Differences in judge prompts, model versions or scoring thresholds will show up here, so record a baseline and agree with the evaluation owners what counts as a match. Keep the Braintrust export as your archive until the parallel results agree and your retention policy is satisfied.

Which Braintrust capabilities map cleanly to Opik

This table maps each Braintrust capability on the migration path to its closest Opik equivalent and marks it as clean, needs rework or gap. Use it to scope the migration: rows marked clean can usually move with configuration, rows marked needs rework require code or data changes, and gaps need a decision before cutover. Confirm every row against the current Opik documentation and your own Braintrust project before you commit.

Braintrust capability	Opik equivalent	Status and what to confirm
Tracing of LLM calls and multi-step agents	Opik tracing with native framework integrations and a track decorator for custom code	Clean for supported frameworks. Confirm which of your frameworks have native integrations and whether custom spans need rework.
Datasets	Opik datasets, recreated from exported records	Needs rework. Records must be re-imported and field names mapped. Confirm how Opik expects inputs and expected outputs to be structured.
Experiments and baselines	Opik experiments for tracked evaluation runs	Needs rework. Past Braintrust experiments are not a native import target, so archive results and decide which baselines to recreate. Confirm the import options in the Opik docs.
Scorers written in code	Custom metrics using Opik's evaluation metric interface	Needs rework. Port scoring logic into Opik's metric signature and test it against stored outputs. Confirm the current base class and parameter names.
Braintrust-specific scorer syntax and templates	Opik metric definitions and built-in LLM-as-a-judge metrics for hallucination, moderation and RAG	Gap for proprietary syntax. Built-in judge metrics cover common cases, but any Braintrust-only templates must be rewritten. Confirm metric names and judge model settings.
Online scoring of production traces	Opik online evaluation rules	Needs rework. Rules must be recreated and their sampling and filter behavior verified against Braintrust's configuration. Confirm rule syntax and supported metrics.
Prompt management	Opik prompt management with versioning	Needs rework. Move prompt text and version history by hand or through the API. Confirm whether version labels carry over.
Playground	Opik prompt playground	Clean for ad hoc testing. Confirm model provider keys are configured on your self-hosted instance.
Hosted control plane and user access	Self-hosted Opik on your Kubernetes or Docker Compose stack, or the hosted Comet.com service	Gap if you need Braintrust's managed control plane. Self-hosting moves operation of ClickHouse, MySQL, Redis, authentication and backups to your team.

What data moves, and which integrations to rebuild

A Braintrust to Opik move has two halves: the historical records you carry over, and the live instrumentation and checks you rebuild on the Opik side. The cards below separate what can be exported from Braintrust, what must be recreated by hand, and what needs to be confirmed in your own environment before cutover.

Logs and traces from the UI

Braintrust lets you export logs, datasets and experiments from the UI as JSON or CSV. This suits small projects and spot checks, but it is a manual step for each project and does not give you a complete, repeatable archive. Before relying on it, confirm which fields, attachments and scores come through in your export and whether the file structure matches what your analysis scripts expect. Treat the UI export as a sample, not a migration path.

Bulk export through the BTQL API

For full history, the BTQL API returns logs, datasets and experiments as JSON or Parquet. Requests can time out at 30 seconds, so large projects need paged queries that walk through time windows or record ranges. Plan the paging logic, write each page to durable storage with a checksum, and confirm the total row count per project against the Braintrust UI before you delete anything.

Recreating datasets in Opik

Confirm with Opik's current documentation how datasets are created and populated through its SDK and API, since no Opik import tool for Braintrust datasets is confirmed here. Map each Braintrust dataset's input, expected output and metadata fields to Opik dataset items, and keep Braintrust record IDs in metadata so you can trace a migrated item back to its source. Validate a sample of items field by field.

LLM-as-a-judge metrics and scorers

Opik ships LLM-as-a-judge metrics for hallucination, moderation and RAG, but your Braintrust scorers are your own prompts and code. Export each scorer definition, rebuild it as an Opik metric, and pin the judge model and prompt version. Rerun a fixed set of past experiments on both systems. Differences in scores usually come from judge model or prompt drift, so record the judge configuration with every score.

SDK instrumentation swaps

Replacing Braintrust's tracing wrappers and experiment calls with Opik's SDK changes how spans, inputs and outputs are captured. Inventory every call site, swap the wrapper for Opik's native integration for your framework or the @track decorator for custom code, and compare span trees for the same request on both systems. Remove the Braintrust SDK only after both traces match on your test cases.

OpenTelemetry exporters and the Collector

If applications already emit OpenTelemetry, route traces through an OpenTelemetry Collector to Opik instead of adding a second SDK. The Collector can authenticate incoming connections with authenticator extensions such as OIDC, and attach credentials to outgoing exports with the oauth2client extension. Review the Collector configuration best practices, pin the release you test, and confirm which Opik endpoints and headers your version accepts.

CI gates and parallel runs

Evaluation gates in CI often call Braintrust directly, so they must be repointed before Braintrust access ends. Run both systems in parallel on the same test set until pass or fail decisions agree across a defined window. Store API keys for both systems in your secrets manager, and confirm that each gate fails closed if Opik is unreachable.

How the migration runs in five steps

The sequence moves Braintrust projects, datasets and scorers into Opik one service at a time, proves that scores agree in a parallel run, and only then retires the Braintrust side. Each step ends with a written output that the next step depends on.

Inventory projects, datasets and scorers
Your engineering lead and an evaluation owner list every Braintrust project, dataset, experiment, scorer and prompt, then map each one to the application that produces its traces. The output is a migration register that names the owning team, the instrumented services and the scorers each service depends on. Confirm with your own records which projects still receive live traffic, because inactive projects often need only archival export.
Stand up Opik in your infrastructure
Your platform team deploys Opik in your own environment, using the Docker Compose path for a local evaluation or the Helm charts for a production cluster, as described in the Opik README. The output is a running instance with its ClickHouse, MySQL and Redis dependencies, backups configured and access restricted to your identity controls. Confirm the sizing and operating model with your own engineers, since those components are operated by your team in this model.
Pilot one service end to end
Choose one low-risk application and recreate its datasets in Opik, then re-instrument it with Opik's SDK or with OpenTelemetry, depending on what the service already uses. Rebuild one LLM-as-a-judge metric, run the same test cases through both systems, and record where the outputs and scores differ. The output is a pilot report listing each dataset, metric and tracing gap with an owner and a resolution, signed off before any other service moves.
Run both systems in parallel
For each migrated service, send traces to Braintrust and Opik at the same time and score the same traces with equivalent metrics in both. Compare aggregate scores, per-example results and failure categories over the period your evaluation owner agrees on. The output is a comparison log per service that shows where results match and where they diverge, with each divergence traced to a metric definition, a judge model setting or a data mapping issue. Do not move to cutover until the owner signs off on the comparison.
Cut over SDK and exporter configuration
Change application configuration so traces go only to Opik, updating the SDK initialization or the OpenTelemetry exporter endpoint and credentials for each service. If you use an OpenTelemetry Collector between services and storage, route its exporter to Opik using the authentication options in the Collector configuration documentation. The output is a change record per service with the commit, the deployed version and a confirmation that new traces appear in Opik and stop appearing in Braintrust.
Verify exports before decommissioning
Before you disable Braintrust, export all remaining projects, logs, datasets and experiments in the formats Braintrust supports, then check record counts, sample records and score values against the exported files. Store the exports in your own retained storage with an access policy that matches your data classification. The output is a verified export archive with a checksum manifest and a sign-off from the data owner and the security lead.
Decommission Braintrust and close the register
Once the archive is verified, remove Braintrust API keys from applications and secret stores, revoke service accounts, and confirm with your contract owner the end date and any data deletion request that your agreement requires. Update the migration register so every project shows its new home or its archival location. The output is a closed register and a written record of key revocation, contract status and deletion confirmation from the vendor.

What drives cost and duration, and how to reduce it

This table lists the components that set the cost and effort of moving Braintrust work to self-hosted Opik, with the one-time and recurring parts marked. Read it as a checklist of what to size and what to request in a quote, since none of the rows carry a fixed price or duration.

Driver	Why it changes cost or effort	How to keep it down
Number of instrumented services	Each service using the Braintrust SDK or an OpenTelemetry exporter must be switched to Opik tracing, and each one needs testing before traces are trusted.	Start with the services that produce the most evaluated traffic, and confirm whether Opik's OpenTelemetry support covers your existing exporters before changing code.
Dataset size and export paging	Braintrust exports are paged, so larger logs and datasets take more runs to move and more checks to confirm record counts.	Export and import in batches, record row counts per dataset, and remove logs you no longer need before export.
Custom scorers and judge prompts to port	Each Braintrust scorer must be rewritten as an Opik LLM-as-a-judge metric or custom metric, then re-run against the same examples to confirm it scores the same way.	Inventory scorers first, retire unused ones, and reuse Opik's built-in judge metrics for hallucination, moderation and RAG where they match.
Baseline experiments to reproduce	Teams often need old scores to stay comparable, so baseline runs may need to be re-executed on Opik with the same models and prompts.	Decide which baselines must carry over and archive the rest as exported files.
CI gate count	Each pipeline that blocks a release on an evaluation score must point to Opik, get new credentials and pass a parallel run.	Migrate gates in priority order and keep Braintrust gates active until Opik results match on the same test set.
Self-hosting operations on ClickHouse, MySQL and Redis	Opik needs these three stores running, backed up and upgraded. Operating them is recurring staff time and infrastructure cost, and the README states the platform handles 40M+ traces per day.	Size storage and retention against your real trace volume, and confirm backup and upgrade steps for each store in the Opik documentation.
Helm on Kubernetes for production	Production installs use Helm charts, so the cluster, ingress, TLS, secrets and upgrade process must exist or be built.	Reuse an existing cluster and secrets process, and check the Helm chart version before planning upgrades.
Hosted Opik on Comet.com versus self-hosting	Hosted use shifts infrastructure work to Comet but keeps a recurring service fee, while self-hosting moves that work to your team or implementation partner.	Ask Comet for the hosted fee structure and compare it with the staff hours and infrastructure you would carry.

What usually goes wrong, and how to prevent it

Most failed Braintrust to Opik moves fail quietly. Scores drift, exports come up short, and traces disappear without an error. The cards below cover the failure points that matter most during a parallel run and the check that catches each one before the Braintrust project is retired.

Judge drift after a model change

An LLM-as-a-judge metric is only as stable as the judge model and its prompt. If the judge model, its version or the metric wording changes during the migration, scores move even when the application did not change. Pin the judge model and prompt in version control, then rerun a fixed set of past traces through both systems. Record the judge configuration with every experiment so a score change can be traced to a configuration change.

Incomplete exports

Bulk exports often carry rows and fields but can miss attachments, feedback history, comments or edit history, depending on the export path you use. Before you delete anything in Braintrust, compare row counts per dataset and experiment, and spot-check attachments and score history against the Opik copy. Confirm with Braintrust and Comet exactly which objects each export includes, and keep the raw export files as an archive.

Mismatched scoring results

The same answer can score differently in two systems when the scorer code, the default parameters, the rounding or the handling of missing values differs. A score that looks like a regression may be an implementation difference. Run the same dataset and the same scorer through both platforms, then compare per-row results, not only averages. Document every intentional difference so reviewers can accept it.

Unpinned SDK versions

If the Opik SDK, the Braintrust SDK or an instrumentation package floats to a new version during the migration, span names, attribute fields and default behavior can change without notice. Pin exact versions in your lockfiles and in CI, and record the Opik server version alongside the SDK version. Opik's own releases page lists the versions to track, so review it before each upgrade and rerun your evaluation suite after any change.

Trace loss during exporter changes

Switching the exporter from one backend to another can drop spans during the cutover, especially when batching, retries or sampling settings differ between the old and new configuration. Traces that never reached either system leave gaps that are hard to notice later. Run both exporters in parallel for a fixed window, count spans per trace in each backend, and set alerts on dropped or refused data at the OpenTelemetry Collector before you remove the old path.

How the move differs by team size and starting point

A single product team with a modest set of datasets, a handful of scorers and one or two applications usually runs the smallest migration. One engineer can often own the cutover, and the main work is re-creating the datasets and LLM-as-a-judge metrics in Opik and pointing the application's tracing at the new endpoint. The sequence is short: export from Braintrust, stand up Opik locally with Docker Compose, import the datasets, rebuild the judges, then run both systems on the same inputs until scores agree.

Ownership: one engineer or a small pair usually owns the export, the Opik instance and the sign-off on results.
Sequence: export first, then local Opik, then dataset import, then judge rebuild, then parallel runs.
Instrumentation: swap the Braintrust SDK calls for Opik's @track decorator or native framework integrations in the application code.
Tooling: Docker Compose through the opik.sh script is enough for evaluation work before any production sizing.
Cost drivers: mostly one-time engineering hours, plus a small recurring host to keep Opik running.

Before deleting anything in Braintrust, confirm with the team which experiments must remain comparable over time, since those are the ones worth carrying across.

Questions buyers ask before moving from Braintrust to Opik

What drives the effort and cost of moving from Braintrust to Opik?

The main drivers are the number of applications instrumented, the number of datasets and scorers to recreate, and how many LLM-as-a-judge metrics must match Braintrust's results. Opik is Apache-2.0 licensed and its README states the full platform can be self-hosted without a commercial license, so there is no license fee to Comet for self-hosting. The recurring costs sit in your own infrastructure: compute, ClickHouse, MySQL, Redis, and the people who operate them. Ask any services partner for a written scope that lists each application, dataset and metric in the move, since that list sets the work.

How long does a move from Braintrust to Opik take?

No supplied figure gives a standard duration, so treat any number as something to confirm against your own inventory. The timeline depends on how many projects, experiments and datasets you run, how many applications need new instrumentation, and whether your judges and scoring functions need rewriting. A practical approach is to plan in phases: stand up Opik, instrument one application, rebuild one evaluation suite, then repeat. Ask for milestones tied to parity checks rather than a single cutover date.

What does our team need to provide before the work starts?

You need an inventory of Braintrust projects, experiments, datasets and scorers, plus named owners for each application that sends traces. Your team should also supply the infrastructure target, whether Docker Compose for evaluation or Kubernetes with Helm charts for production, and the model providers and API keys the LLM-as-a-judge metrics will call. Access to the Braintrust export is required before any deletion decision. Confirm these items in writing as part of scope.

Can our own engineers do this migration in-house?

Yes, if your team already runs Kubernetes, ClickHouse and a Helm-based deployment, and has engineers who know your evaluation code. Opik's Docker Compose path through its opik.sh script suits a small first trial, while production needs Helm charts and ongoing operation of ClickHouse, MySQL and Redis. Teams that lack this operations experience often hand the deployment and hardening to an implementation partner, then keep operating it internally. DeployOpen plans, implements, hardens, backs up and documents self-hosted platforms in your own infrastructure under a fixed-scope written agreement.

How do we confirm Opik returns the same evaluation results as Braintrust?

Run both systems in parallel on the same dataset versions and compare scores per example, not only averages. Use identical judge models, prompts and temperature settings where possible, and record any differences that come from the judge model rather than the platform. Keep the Braintrust baseline for each experiment until the agreed comparisons pass over several runs. Document the acceptance criteria before you start, so the team agrees on what counts as a match.

What happens to our Braintrust data when we exit?

Export logs, datasets and experiments from Braintrust before you end the contract, and verify the exported files are complete and readable before you delete anything. Confirm the export formats and the permissions your account needs with Braintrust directly, since those terms are set by the vendor and your agreement. Once your Opik copy is verified and your retention obligations are met, you can decide how to retire the Braintrust project. Record the export checksums and the date you verified them for your audit trail.

Do we need to re-instrument our applications, or can OpenTelemetry help?

You will change instrumentation in each application, but the scope depends on how you send traces today. Opik supports OpenTelemetry along with native integrations for LangChain, LangGraph, LlamaIndex, CrewAI and 60+ other frameworks, plus a @track decorator for custom code. Applications already exporting OpenTelemetry may need only an exporter change, while custom Braintrust SDK calls need rewriting. Test each application against a staging Opik instance before switching production traffic.

How DeployOpen scopes and runs this migration

DeployOpen plans, implements, hardens, backs up and documents self-hosted Opik inside your own infrastructure, and most engagements of this kind run under a fixed-scope agreement with scope, deliverables and timelines agreed in writing before work starts. For a Braintrust migration, the written scope should name the target cluster or Docker host, the Opik release to deploy, the ClickHouse, MySQL and Redis components and who operates each, the export and import jobs for experiments and datasets, the SDK changes per application, and the parallel-run period with its pass criteria. Opik self-hosts with Docker Compose through its opik.sh script for local use or on Kubernetes with Helm charts for production, so the scope should state which of those paths applies to your environment.

Ask for the scope to separate one-time work from recurring work. One-time items include the deployment, the hardening of ingress, authentication and secrets, the migration tooling, the rebuild of LLM-as-a-judge metrics, and the runbooks. Recurring items include upgrades, backups and restore tests, and monitoring of the ClickHouse and MySQL tiers. Ongoing support, upgrades and continuous testing are available as a retainer, so decide early whether your team will own day-to-day operations or buy them as a service. The quote follows from the written scope, and DeployOpen does not publish a price list, so the scope document is the thing to compare across bids.

DeployOpen does not hold partner, reseller or certified status with Comet or Braintrust, and it does not issue audit reports or certifications. Where your migration supports a compliance review, DeployOpen prepares the systems, controls and evidence, and an independent auditor or certification body makes the assessment. Confirm with your own security and compliance leads which evidence the review needs, such as access logs, backup restore records and the retention settings you configure in Opik, before the scope is signed.

Migrate from Braintrust to Opik