Open-source alternative to the Braintrust AI proxy: the short answer
If your applications call Braintrust's hosted proxy for one API across providers and response caching, LiteLLM is the self-hosted option with the broadest proxy feature set, with its core under the MIT license. Portkey's AI Gateway is also MIT licensed and can run in your own infrastructure, but several features you may rely on, including semantic caching, advanced logging and tracing, access control and PII redaction, sit outside the open-source gateway as enterprise features. Either choice moves the proxy into your environment, but they differ in what you get without a commercial contract. The real work is not the license comparison. You will be operating a gateway runtime, a PostgreSQL database that LiteLLM uses for virtual keys and spend tracking, a Redis instance once more than one proxy instance runs, and the master key and provider credentials that sit behind them. You will also change the base URL and API key in every application that calls the proxy today, and that change needs a rollout plan, a rollback path and a way to confirm that requests, costs and cache behavior match what you had before. Before you compare features, confirm how Braintrust's proxy handles routing, caching scope, key handling and logging against Braintrust's own documentation, because those internals were not part of the research behind this page. Write down the caching behavior your applications depend on, such as whether identical requests are served from cache and for how long, and test the same behavior against the gateway you choose before you change any production base URL.
Which proxy capabilities map to which open-source options
This table maps the capabilities teams usually rely on in Braintrust's hosted proxy to LiteLLM and Portkey's open-source gateway. Read each row for the LiteLLM feature, the Portkey status and the coverage note, and confirm any item marked for checking in the project's current docs before you commit to a design.
| Capability | LiteLLM (self-hosted) | Portkey AI Gateway | Coverage note |
|---|---|---|---|
| One OpenAI-compatible endpoint across providers | Supports more than 100 providers through the OpenAI request format, including OpenAI, Anthropic, Gemini, Bedrock and Azure | Open-source gateway runs via npx, Docker, Node.js, Cloudflare Workers or Kubernetes | Confirm each provider and model your applications call is supported in the current release |
| Virtual keys | Per-key, per-user and per-team keys, requires PostgreSQL | Access control is listed as an enterprise feature outside the open-source gateway | Plan a PostgreSQL database before you issue keys to other teams |
| Budgets and rate limits | Per-key, per-user and per-team budgets, TPM and RPM limits, temporary budget increases | Not described as part of the open-source core; confirm in the repository | Budget enforcement is a key difference to verify in a test environment |
| Fallbacks and routing | Retries, cooldowns after repeated failures, priority ordering for failover and session affinity | Gateway routing and fallback features to confirm in the repository docs | LiteLLM documents these routing modes in detail; confirm behavior under your load |
| Spend and request logging | Logs to Langfuse, Datadog, OpenTelemetry and Prometheus; spend tracking built in | Advanced logging and tracing are listed as enterprise features | Route LiteLLM logs into your existing observability stack or into Langfuse or Opik |
| Response caching | Confirm caching options and backend requirements in the LiteLLM docs | Semantic caching is listed as an enterprise feature | Do not assume a caching feature matches Braintrust's behavior; test cache hits and misses |
| SSO and access control | SSO with Google, Microsoft Entra ID, Okta and generic OIDC; free for up to 5 users from v1.76.0 | Access control listed as an enterprise feature | Beyond five SSO users, LiteLLM requires an Enterprise license; confirm current terms |
| PII redaction and guardrails | Team-level guardrails and logging are in LiteLLM Enterprise | PII redaction is listed as an enterprise feature | Confirm whether any redaction is needed in the open-source tier or only in the enterprise tier |
| Braintrust-specific proxy features | Confirm any Braintrust-specific headers or logging behavior with the Braintrust docs | Confirm any Braintrust-specific headers or logging behavior with the Braintrust docs | Check every request header, base URL and key format your code relies on before the switch |
LiteLLM, Portkey's gateway and the enterprise tiers compared
Replacing Braintrust's proxy means choosing a gateway you operate yourself. The options below differ in what you run, which features sit behind an enterprise license, and how much operational work they add. Use them to decide which gateway fits your provider count, identity requirements and team size.
LiteLLM open-source proxy
The LiteLLM proxy is MIT licensed everywhere except its enterprise directory, and it gives applications one OpenAI-compatible endpoint across more than 100 providers. Virtual keys, budgets, rate limits and fallbacks need PostgreSQL, and the production guide names PostgreSQL as the primary supported database. Once more than one proxy instance runs, Redis 7.0 or newer is required for shared rate limits, router state and cache. Check who will patch, back up and monitor both databases before you commit.
LiteLLM Enterprise
LiteLLM Enterprise adds SSO and SCIM beyond the free SSO limit, JWT authentication with identity provider tokens, audit logs, organization and team RBAC, and IP allowlisting. It also integrates with secret managers such as HashiCorp Vault and cloud key services. Since v1.76.0, SSO is free for up to 5 users, so teams past that count should price the license before rollout. Ask the vendor for the license terms in writing, including what happens to existing keys if the license lapses.
Portkey AI Gateway
Portkey's AI Gateway is MIT licensed and can run with npx, Docker, Node.js, Cloudflare Workers or Kubernetes, so it can sit inside your own network. Prompt management, semantic caching, advanced logging and tracing, access control and PII redaction are enterprise features outside the open-source gateway. Confirm which of your Braintrust proxy features, especially response caching, are in the open core before you plan the switch. Teams that want the enterprise features without running the control plane should review Portkey's hybrid model separately.
Thin in-house gateway
If you call only a few providers, a small internal service that forwards requests with one key format may be enough. It keeps the dependency surface small and gives your security team a codebase they can review. The cost is that you build and maintain fallbacks, retries, spend tracking and logging yourself, and every provider API change becomes your work. Choose this route only if the feature list you need is short and written down, and if your team will own the code after the first release.
Choosing a deployment shape by team size and risk
A single team with a few providers usually needs one LiteLLM proxy instance in front of the applications that currently call Braintrust's proxy. The gateway is MIT licensed, so the main choices are where it runs and which database holds keys and spend records. LiteLLM's virtual keys need PostgreSQL, so that database is the one piece of state you cannot skip, even for a small deployment. A Docker Compose setup is a reasonable way to trial the gateway before production, but treat it as a test environment rather than the production target.
- Operations: one proxy process, one PostgreSQL database, backups for the database and a secure location for
LITELLM_MASTER_KEY, which must start withsk-. - Redis: not required with a single proxy instance, but LiteLLM's production guide requires Redis 7.0 or newer once more than one instance runs, so decide now whether you expect to scale out.
- Provider keys: move the upstream provider credentials into your own secret store rather than leaving them in application environment files.
- Cutover decisions: which applications switch first, whether you keep Braintrust's caching behavior or accept a different cache, and who owns on-call for the proxy.
- Licensing: the features most small teams need, such as virtual keys and budgets, are in the open-source code, so confirm this against the current LICENSE file before you assume it.
Before cutover, confirm the base URL and key changes in a staging environment and check which provider and model names your applications use.
A platform serving several teams needs more than one proxy instance, because a single process becomes a shared failure point and a shared bottleneck. LiteLLM's guidance for Kubernetes is to run one Uvicorn worker per pod and scale horizontally, and to share state through Redis so rate limits, router state and cache stay consistent across pods. Per-team budgets and virtual keys then become the main governance tool, since each team's spend and request limits can be set separately from the applications that use them.
- Operations: a Kubernetes deployment with horizontal scaling, PostgreSQL as the primary database, and a shared Redis service running version 7.0 or newer.
- Resource sizing: LiteLLM suggests 1 vCPU and 4 GiB memory per worker as a starting point, so model load per pod before choosing replica counts.
- Spend tracking: LiteLLM batches spend updates to the database every 60 seconds at scale, so confirm that this delay is acceptable for your budget alerts.
- Team model: decide which teams get keys, who may raise temporary budgets, and which models each key can reach.
- Routing and failover: choose a routing strategy from the options LiteLLM documents, and test how cooldowns and retries behave under provider errors.
Before cutover, run a load test that matches your peak traffic, and confirm your team agrees on who approves budget increases.
In a regulated environment the deployment question shifts from capacity to control. Your security team will likely require single sign-on through an existing identity provider, audit logs of administrative actions, secrets held in an approved secret manager, and network restrictions on who can reach the admin interface. Several of these controls sit in LiteLLM's Enterprise tier, so the license decision becomes central. SSO is free only for up to 5 users from v1.76.0 onward, and SCIM, JWT authentication, audit logs, organization and team RBAC, IP allowlisting and secret manager integrations are listed under Enterprise.
- Identity: the admin UI supports Google, Microsoft Entra ID, Okta and generic OIDC, but confirm whether your user count requires an Enterprise license.
- Audit and access: plan how audit logs will be exported, retained and reviewed, and which roles can change keys and budgets.
- Secrets and network: use a supported secret manager for the master key and provider credentials, and define IP allowlisting for the proxy and admin paths.
- Evidence collection: DeployOpen can prepare systems, controls and evidence for compliance assessments, but independent auditors issue any audit report or attestation.
- Licensing decision: compare the cost of an Enterprise license with the effort of building equivalent controls yourself, and get the scope and terms in writing.
Before cutover, confirm with your auditor which controls must be evidenced and whether the open-source features alone satisfy them.
Running the gateway beside Braintrust during the switch
A gateway cutover rarely happens in one step. Braintrust evaluations, tracing and the existing proxy path usually keep running while applications move their model calls, so each coexistence point needs a named owner before the first base URL changes.
Keep Braintrust evaluations running
Until applications are switched, Braintrust remains the system of record for experiments, datasets and scores. Evaluation jobs and CI gates that read from Braintrust should stay pointed at it, so results before and after the cutover can be compared on the same datasets. The coordination risk is a silent change in model routing that shifts scores without anyone noticing. The owner is the evaluation lead, who should record which gateway routes each experiment used.
Choose the backend for gateway logs
Decide where gateway request logs go before enabling them. LiteLLM can send logs to Langfuse, OpenTelemetry, Prometheus and Datadog, and Langfuse or Opik are the self-hosted tracing options covered in this group. Running both backends for a period can help validate parity, but it doubles storage and access reviews. The owner is the platform engineering lead, who should confirm retention, PII handling and where traces with prompts are stored.
Keep a rollback path open
Keep the Braintrust SDK configuration and the original provider keys reachable until the gateway has carried production traffic for an agreed period. Rollback should be a configuration change, such as restoring a base URL in a deployment variable, rather than a code release. The risk is that rotated or deleted provider keys leave no fallback. The owner is the application team lead, who signs off on when the old path is retired.
Protect the master key
The LiteLLM master key, which must start with sk-, grants administrative control over the gateway, so it belongs in a secret manager rather than in configuration files. During coexistence, both the gateway and the applications may reference provider secrets in two places, which increases the chance of drift. The owner is the security team, which should set rotation and access rules before virtual keys are issued to teams.
Connect admin UI to existing SSO
The LiteLLM admin UI supports SSO with Google, Microsoft Entra ID, Okta and generic OIDC providers. Confirm which identity provider holds the group membership that should map to administrator and team roles, and test with a small user set first. Free SSO covers a limited number of users, so the count of administrators needs checking against the current license terms. The owner is the identity team, which should review the redirect configuration and the account lifecycle process.
Agree on who runs each system
During the switch, two sets of credentials, dashboards and alerts exist side by side, and it must be clear who responds to an outage on each. Assign on-call ownership for the gateway, the chosen tracing backend and the remaining Braintrust integration before cutover. The risk is alerts that go to a team that no longer operates the component. The owner is the operations manager, who should document this in the runbook.
How to switch base URLs and keys without an outage
This sequence moves applications from the Braintrust proxy to a self-hosted LiteLLM gateway one service at a time, so each caller is verified before its old key is retired. The result is a documented inventory, a running gateway with scoped virtual keys, and a rollback path.
Inventory every caller of the Braintrust proxy
Engineering leads and platform owners list each application, service, scheduled job and notebook that sends requests to the Braintrust proxy. For each one, record the base URL, the model names it requests, the key it uses, the provider credentials behind it, and whether it depends on cached responses. Done means every caller has a row in a shared register and each owner has confirmed the entry. Ask Braintrust to confirm in writing how its proxy handles provider credentials and cached responses, so you know what must be recreated on your side.
Deploy PostgreSQL and a shared Redis
The platform team deploys PostgreSQL as the primary database for LiteLLM, because virtual keys, budgets and spend tracking require it. If more than one proxy instance will run, they also deploy Redis 7.0 or newer for shared rate limits, router state and cache. Done means both services accept connections from the gateway network and backups are configured for the database. Confirm both are inside your own infrastructure and that access is limited to the gateway.
Pin a supported LiteLLM minor version
The platform team selects one LiteLLM version and pins its image tag or package version in the deployment manifest. LiteLLM supports the four most recent stable minor versions for open source and Enterprise deployments, so choose one of those and record the decision. The LiteLLM releases page lists v1.105.1, published October 14, 2026, which the team should check against its supported range before pinning. Done means the version is written in the change record and the deployment uses that exact tag rather than a floating latest.
Store the master key in a secret manager
The security team generates LITELLM_MASTER_KEY, which must start with sk-, and stores it in your secret manager rather than in configuration files or pipeline variables. Application teams never receive this key; it is used only for administration. Done means the gateway reads the key from the secret store at startup, the key is absent from the repository and container images, and access to it is logged and limited to named administrators.
Create scoped virtual keys per team
Each team gets its own virtual key with a budget, TPM and RPM limits, and a list of permitted models. Budgets can be set per key, per user or per team, and temporary increases can be granted when a launch needs them. Done means every caller from the inventory maps to a key that can reach only its approved models, and a test request from each team returns a successful response within its limits. Keep a record of who approved each budget.
Map model aliases and test each route
Applications should keep requesting the model names they already use, so the platform team defines aliases in the LiteLLM configuration that point each name to the provider model it should reach. Test each alias against a staging key, including a deliberate failure to confirm that fallbacks and retries behave as intended. Done means every model name in the inventory returns the expected provider output in staging, and the alias table is version-controlled.
Cut over one service at a time with a canary
Change the base URL and key in one low-risk service first, send a small share of its traffic through the gateway, and compare error rates, latency and returned content with the Braintrust path. Move the remaining services in batches once the canary holds. Monitor spend and error metrics through the gateway's logging integrations, such as Prometheus or OpenTelemetry, and set alerts before each batch. Done means each service runs fully on the new base URL and key, with its previous path still available for rollback.
Retire old keys after a rollback window
Keep the Braintrust keys and base URLs in place, and documented, until every service has run on the gateway through a full business cycle with no unresolved incidents. Then revoke the old keys, remove the Braintrust base URLs from configuration, and confirm with Braintrust how your account and data will be handled under your contract. Done means the old keys return authentication errors, the inventory shows only gateway endpoints, and the cutover record is signed off by the service owners.
What a self-run gateway costs: the components and who pays for them
The main cost of a self-run LiteLLM gateway is people's time, not licenses. The core software is MIT licensed, so the recurring bill is for the engineers who deploy it, keep it on a supported version, and answer when it fails. One-time work covers the design of the proxy topology, mapping existing Braintrust keys and base URLs to virtual keys, migrating provider credentials, and load and failover testing. Recurring work covers upgrades within the supported minor versions, on-call coverage for a service that every LLM call now passes through, key rotation, and a regular review of spend against the budgets you set. Ask your platform team to estimate each of these as a separate line, with the hours assigned to a named role, so that the comparison with Braintrust is not hidden inside a single headcount figure.
The infrastructure lines are less obvious. LiteLLM's production guidance names PostgreSQL as the primary database, requires Redis 7.0 or newer once more than one proxy instance runs, and suggests 1 vCPU and 4 GiB memory per worker, with one Uvicorn worker per pod on Kubernetes. Your cost therefore depends on replica count, provider call volume and how much spend history you keep. Budget for managed or self-operated PostgreSQL and Redis, their backups and restore tests, the logging backend that receives gateway telemetry (Langfuse, Datadog, OpenTelemetry or Prometheus, as configured), and a secret manager for LITELLM_MASTER_KEY, which the guidance says must start with sk-. Each of these adds compute, storage and operations work that recurs for as long as the gateway runs.
Questions buyers ask before replacing the Braintrust proxy
How much effort does switching from the Braintrust proxy to LiteLLM take?
The effort depends mainly on how many applications send traffic through the proxy, how many model providers they call, and how many API keys and teams need their own budgets. Each application needs its base URL and key changed and tested, each provider account needs to be connected to the gateway, and each team needs virtual keys with budgets and model restrictions designed before cutover. The work also includes running the gateway's own infrastructure: PostgreSQL for virtual keys, Redis once more than one proxy instance runs, and Kubernetes or another container platform. The duration depends on the agreed written scope, and no figure applies until that scope is defined.
Can our own team replace the Braintrust proxy without outside help?
A team that already runs PostgreSQL, Redis and Kubernetes can often deploy LiteLLM itself, because the core components are familiar. The harder parts are usually design decisions rather than installation: how virtual keys map to applications and teams, what budgets and TPM or RPM limits each key carries, which models each key may call, and how fallbacks and routing order should behave during a provider outage. Teams without those skills in-house, or without time to test failover, often bring in an implementation partner for a fixed scope. DeployOpen plans, implements, hardens and documents self-hosted platforms in your own infrastructure under a written scope.
Does LiteLLM cache responses the same way the Braintrust proxy does?
Do not assume it does until you have checked. Braintrust's proxy offers response caching, and LiteLLM documents caching features in its proxy and router documentation, but the exact behavior of cache keys, scope, time-to-live and what counts as a cache hit for your request patterns must be confirmed in the LiteLLM documentation for the version you plan to run. Test it with representative traffic: repeated identical prompts, prompts that differ only in whitespace or metadata, and streaming requests. Confirm that cached responses respect per-team boundaries before you route production traffic through the cache.
When do we need the LiteLLM enterprise license?
The open-source gateway covers the core proxy, including virtual keys, budgets, rate limits, routing, fallbacks and logging. The admin UI supports SSO with Google, Microsoft Entra ID, Okta and generic OIDC providers, and from v1.76.0 SSO is free for up to 5 users, with an Enterprise license needed beyond that. Enterprise adds SCIM, JWT authentication with identity provider tokens, audit logs, organization and team RBAC, IP allowlisting, secret manager integrations and team-level guardrails and logging. Ask LiteLLM for the current terms, since licensing applies to files under the enterprise directory, which carry their own license.
What do we need to provide for the migration?
The work needs named owners, and the most common delays come from waiting for them. Assign one owner for the gateway platform, one for virtual keys and budgets, and one for each application team that changes its base URL. You will need access to your identity provider so SSO and group mapping can be configured, administrative access to each model provider account so keys can be created and rotated, and a secret manager for the master key and provider credentials. Test traffic from real applications, in a non-production environment, is needed to verify fallbacks, caching and logging before cutover.
What changes for applications when we switch base URLs and keys?
Applications keep the OpenAI request format, so the main change is the base URL and the API key they send. Braintrust-issued keys should be replaced with virtual keys that LiteLLM issues, each scoped to the models and budget that application needs. Provider credentials move into the gateway configuration and out of application code. Plan a staged cutover: switch one low-risk application, compare logs and cost records against the previous path, then move the rest. Keep the old path available until error rates and spend match expectations.
Does the gateway need to run in our own network?
Yes, LiteLLM is a self-hosted server, so you choose where it runs, whether in your cloud accounts, your data centers or an air-gapped network that can reach the model providers you have contracted. Confirm the network paths your providers need, the egress rules for each region, and how logs leave the gateway, because logging to Langfuse, Datadog, OpenTelemetry or Prometheus creates additional data flows. Record these decisions for your security review, since they determine where prompts and responses are processed and stored.
How DeployOpen scopes and delivers a gateway replacement
DeployOpen plans, implements, integrates, hardens, backs up, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For an AI proxy replacement, that means a LiteLLM gateway running in your environment, wired to your providers, identity system and logging, with the cutover planned and written down before any application changes.
How we scope
- Inventory every application, script and team that currently calls the Braintrust proxy and the provider keys behind it.
- Agree in writing which providers, virtual key structure, budgets, rate limits and fallback routes the gateway must support.
- Confirm the hosting target, PostgreSQL and Redis arrangements, and whether SSO needs the free tier or an Enterprise license.
How we build
- Deploy the gateway with PostgreSQL as the primary database and Redis for shared rate limits and router state across instances.
- Keep the master key in a secret manager and configure routing, cooldowns and fallbacks to match the agreed scope.
- Test budgets, model restrictions and failover against representative traffic before any application is pointed at the new endpoint.
How we hand over
- Document the architecture, base URL and key issuance process, runbooks for scaling and upgrades, and the configuration as code.
- Run a parallel period in which Braintrust and the new gateway both serve test traffic, with a written cutover and rollback plan.
- Walk your operations team through monitoring, spend reports and the steps for rotating keys and removing old credentials.
How we support
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
- Ongoing support, version upgrades and continuous testing are available as a retainer.
- Ask any provider for a quote built from the written scope, covering one-time build work and recurring support separately.
