Migrate from Portkey to LiteLLM

What a Portkey to LiteLLM migration involves

The move from Portkey to a self-hosted LiteLLM proxy is a gateway, key, routing and logging migration, and it runs in parallel with application cutover. Most teams arrive here because a renewal is coming, a data residency rule means gateway traffic or logs cannot leave the network, or the control plane sits outside their boundary and they want to own that layer. Plan it as four workstreams that move together: stand up LiteLLM with its PostgreSQL database, recreate virtual keys and budgets, translate routing and fallback configs, and replace hosted logs with a store you run. Applications that already call an OpenAI-compatible endpoint mostly change a base URL and a key, which keeps the cutover small, but the logging and budget behavior will differ and must be checked.

Platform engineering usually owns the work, with security reviewing key handling and log contents, and application owners validating model behavior, latency and error handling after each change. Confirm first which Portkey features your teams actually depend on, such as virtual keys, guardrails, prompt management or semantic caching, because some of these sit outside the MIT-licensed Portkey gateway and need a replacement plan before cutover. The main risks are keys and budgets that do not carry over cleanly, fallback order that behaves differently under real errors, and a logging gap during the switch. No effort figures are supplied for this migration, so ask your implementation partner for a written estimate built from your own inventory of routes, keys and applications.

Operating the gateway is the part teams most often underestimate. LiteLLM recommends Redis 7.0 or newer once more than one proxy instance runs, because rate limits, router state and cache are shared across instances, and it recommends a secret manager for LITELLM_MASTER_KEY. Check which controls need a LiteLLM Enterprise license, particularly SSO beyond the free user limit and audit logs, and confirm them against the current LiteLLM enterprise documentation before you commit to a design. Run the old and new gateways side by side for a fixed window, compare spend and error rates, and keep a rollback path to Portkey until the new logging pipeline has proven it can answer your audit questions.

Which Portkey capabilities map to LiteLLM and which need rework

This table maps each Portkey gateway capability to its closest LiteLLM equivalent and labels it as clean, needs rework or gap. Read the label as a starting point for your own config review, because exact config format equivalence must be confirmed in your environment before cutover.

Portkey capability	LiteLLM equivalent	Status
OpenAI-compatible endpoint	LiteLLM proxy exposes one OpenAI-compatible endpoint, so applications that already send OpenAI-format requests can usually change only the base URL and key	Clean
Provider routing across models	LiteLLM calls more than 100 providers using the OpenAI request format; model names and provider mappings must be rewritten in the LiteLLM model list	Needs rework
Virtual keys	LiteLLM virtual keys backed by PostgreSQL; existing Portkey keys cannot be imported, so new keys must be issued and distributed to applications	Needs rework
Budgets and rate limits	Per-key, per-user and per-team budgets plus TPM and RPM limits; budget values and reset periods must be re-entered and checked against Portkey settings	Needs rework
Fallbacks, retries and cooldowns	LiteLLM router supports retries, cooldowns after repeated failures and priority ordering for failover; confirm how each Portkey fallback chain translates	Needs rework
Load balancing	Simple-shuffle (recommended), usage-based, latency-based, least-busy and cost-based routing; choose a strategy and test weights against real traffic	Clean
Spend tracking	LiteLLM tracks spend per key, user and team and writes it to PostgreSQL; spend history in Portkey is not carried over and should be exported before cutover	Needs rework
Logging and tracing	No hosted log store; LiteLLM sends logs to Langfuse, OpenTelemetry, Datadog or Prometheus, so you must run and size the destination yourself	Gap
SSO for the admin UI	Google, Microsoft Entra ID, Okta and generic OIDC are supported; free up to 5 users from v1.76.0, and LiteLLM Enterprise is needed beyond that	Needs rework
Audit logs and team RBAC	Audit logs, organization and team RBAC, and SCIM sit in LiteLLM Enterprise, so budget for a license if you need them	Gap
Secret handling	Keep LITELLM_MASTER_KEY, which must start with sk-, in a secret manager; Enterprise adds integrations with AWS, Azure, Google, HashiCorp Vault and CyberArk	Needs rework
Prompt management and semantic caching	Portkey lists both as enterprise features outside its open-source gateway; LiteLLM offers caching on Redis, Valkey or Qdrant, while prompt management is handled elsewhere (for example, Langfuse), and both need separate design	Gap

Which data, keys and integrations move and which must be rebuilt

Most of the migration work sits in the inventory, not the gateway deployment. Each item below either transfers in some form or has to be recreated on the LiteLLM side, and the split determines how much time the cutover needs.

Virtual keys and their owners

List every Portkey virtual key, the application or team that uses it, and the provider it points to. Record which key belongs to which owner so that reissued LiteLLM keys go to the right people. LiteLLM keys must be reissued rather than copied, because the key values are generated by the LiteLLM proxy and stored in its PostgreSQL database. Plan a window where each application switches to its new key and the old one is retired.

Budgets and rate limits

Write down each budget and any request or token limit attached to a Portkey key, including whether it applies per key, per user or per team. Map these to LiteLLM per-key, per-user and per-team budgets, TPM and RPM limits. Budgets rarely carry over automatically, so confirm the current spend baseline with finance before setting new limits, or the first days after cutover may show spend that does not match prior reporting.

Routing configs and fallbacks

Export each Portkey config that defines load balancing, retries or fallback targets, and translate it into LiteLLM router settings such as simple-shuffle, priority ordering for failover and cooldowns. Routing semantics differ between the two products, so test each fallback chain with a forced provider error in a non-production environment. Confirm which configs are still used, since unused configs are the easiest to drop.

Provider credentials and secrets

Provider API keys stored in Portkey must be re-entered into the LiteLLM configuration, which should draw them from a secret manager rather than a plain config file. Rotate each provider key during the move so that the old copy in Portkey stops being a live credential. Keep the LITELLM_MASTER_KEY, which must start with sk-, in the same secret manager.

Application base URLs

Inventory every service, script, CI job and SDK call that sends traffic to Portkey, including its base URL and headers. Applications that already call an OpenAI-compatible endpoint mainly need the base URL and key changed, but any Portkey-specific headers must be removed or mapped. Cut over one application at a time and keep the Portkey route available until each one reports successful calls.

Hosted logs and alerting rebuilt

Portkey's hosted log history does not move on its own. Its log export is Enterprise-only and capped per job, so confirm the export path and the cap in Portkey's export documentation before planning. Then rebuild the destinations: LiteLLM can send logs to Langfuse, OpenTelemetry, Prometheus or Datadog, and each alert that depended on Portkey dashboards needs a new rule. Decide early which history is kept as archived files and which is no longer needed.

Migration sequence from inventory to decommission

This sequence moves gateway traffic from Portkey to a self-hosted LiteLLM proxy without a single risky switch. Each step produces a written artifact that the next step depends on, so the team can see what has moved, what has been verified, and how to reverse a cutover.

Inventory keys, configs and callers
Export a list of every Portkey virtual key, config object, prompt reference and budget, then match each one to the application, team and environment that uses it. Ask the Portkey account team or check your own admin console for the export options available on your plan. Include scheduled jobs, notebooks and third-party tools that call the gateway, since they are often missed. The output is a key map that records each virtual key, its provider credentials, its budget and its owning team. The step is done when every caller has an owner and every key has a planned LiteLLM equivalent.
Write the routing and budget specification
Translate each Portkey config into a LiteLLM model list, router strategy, retry and cooldown setting, and fallback order. Confirm which routing strategies and fallbacks LiteLLM supports for each model group, and note any Portkey behavior that has no direct match. Define the LiteLLM virtual keys, per-team budgets, rate limits and model restrictions that replace the Portkey keys in the key map. The output is a routing spec and a budget spec, both reviewed by the platform owner and one security reviewer. The step closes when each Portkey config has a written counterpart or a documented gap.
Stand up LiteLLM and its dependencies
Deploy the LiteLLM proxy in your own infrastructure with PostgreSQL for virtual keys and spend tracking, and Redis 7.0 or newer once more than one proxy instance runs. Store the master key in a secret manager, not in configuration files, and confirm the key starts with sk- as the production guide requires. Connect the logging destination you chose, such as Langfuse or OpenTelemetry. The output is a staging gateway with a health check, a test request through each provider, and a documented restore procedure for the database. The step is done when a test call succeeds through every routing group in the spec.
Pilot on one non-critical application
Choose one internal application with low business impact and a single provider, and point its OpenAI-compatible base URL at the LiteLLM endpoint with a new virtual key. Run it for a fixed review window agreed in advance, and compare response quality, error rates, latency and per-team spend against the Portkey logs for the same workload. Record any differences in request shape, streaming behavior or error codes, because applications often depend on them. The output is a pilot sign-off form naming the owner, the checks passed and the open issues. The pilot is complete only when the owner signs off in writing.
Run Portkey and LiteLLM in parallel
Route a defined share of traffic for each remaining application through LiteLLM while Portkey keeps serving the rest, and log both paths to the same observability backend so results are comparable. Reconcile spend from LiteLLM budgets against provider invoices and Portkey logs at each review point. Watch for missing fields in traces, cache behavior that changed, and rate limit differences between the two gateways. The output is a parallel-run report that shows matching request counts, cost, error categories and any unexplained gaps. The step closes when the reconciliation shows no unexplained difference that affects billing or access.
Cut over each application with a rollback runbook
Move applications one at a time, each with its own change ticket, owner and rollback step. The rollback runbook for each application names the configuration value to restore, the Portkey virtual key to re-enable, and the person who approves the reversal. Keep Portkey keys active but unused during the cutover window so reversal does not require reissuing credentials. The output is a cutover log that records the date, the change, the verification result and any rollback. Each application is done when its owner confirms that normal traffic flows through LiteLLM and the logs show the expected attribution.
Verify logs and budgets before decommissioning
Confirm that every application's requests appear in the new logging backend with the expected team, key and model tags, and that LiteLLM budgets stop traffic at the configured limits in a controlled test. Export the Portkey history you need for audit or retention before any deletion, using the export method your plan supports. Ask your compliance lead which records must be kept and for how long. The output is a verification record signed by the platform owner and the security reviewer. This step is done when no application depends on Portkey and the retention copy is stored.
Decommission Portkey and close the change
Revoke Portkey virtual keys, remove stored Portkey credentials from application secrets and CI variables, and disable the Portkey workspace or project once the verification record is signed. Confirm provider API keys have been rotated if they were held inside Portkey, and remove any Portkey endpoints from firewall or egress allowlists. Update the architecture diagram and runbooks to show LiteLLM as the only gateway. The output is a decommission checklist with each item marked complete and a final review date set for the LiteLLM upgrade plan. The migration closes when the checklist is signed and no Portkey traffic appears in network logs.

How LiteLLM's production stack changes the operating model

Moving from Portkey's managed or hybrid gateway to LiteLLM changes who operates the gateway's state. LiteLLM's production guide names PostgreSQL as the primary supported database, and virtual keys need PostgreSQL to store keys, budgets and spend. Before cutover, decide where that database runs, who owns its backups and restore tests, and how it is sized. Your team will own the schema, the connection pool and the failure mode when the database is unavailable, because Portkey's control plane no longer carries that work for you.

Once more than one proxy instance runs, LiteLLM requires Redis 7.0 or newer for shared rate limits, router state and cache. Treat Redis as a production dependency, not an optional cache. Confirm the Redis topology (managed or self-run, persistence, failover) with your platform team, and test what happens to rate limits and cooldowns when Redis is lost. LiteLLM also recommends one Uvicorn worker per pod on Kubernetes and scaling horizontally, so capacity planning becomes a question of pod count and autoscaling rules rather than a plan tier.

Two settings need explicit ownership. LITELLM_MASTER_KEY must start with sk- and should live in a secret manager, not in a Helm values file or a container image. At scale, LiteLLM batches spend updates to the database every 60 seconds, so budget figures in the admin view can lag real traffic by up to that window. Your team should agree on who rotates the master key, who approves budget changes, and how spend reporting is reconciled against provider invoices. Upgrades follow the same discipline: LiteLLM maintains support for the four most recent stable minor versions, so schedule upgrades against the LiteLLM releases page and test them in staging first.

What drives the cost and duration of the move

This table lists the factors that change how much a move from Portkey to a self-hosted LiteLLM proxy costs and how long it takes, with the reason each matters and how to keep it down. Read it before requesting a quote, since most of the variation comes from the scope rows rather than the software itself. Most of the recurring cost sits in infrastructure and operations, not licensing.

Driver	Why it changes cost or effort	How to keep it down
Number of applications and virtual keys	Each calling service needs its key moved, its budget mapped and its base URL changed, so inventory size sets the cutover work.	Export current keys and group them by owner. Move low-risk services first and reissue keys in batches.
Providers and model routes	Each provider needs credentials, model name mapping and tested fallbacks, and the router config grows with every route.	Limit the first release to providers already in production. Retire unused routes before mapping.
Routing and fallback rules	Portkey configs with nested strategies, conditions or weighted targets need manual translation into LiteLLM router settings and retested.	Document each active config. Remove dead rules and test failover behavior with simulated provider errors.
Log history to retain	Keeping past gateway logs means exporting them before Portkey access ends and storing them somewhere searchable, which adds storage and migration work.	Decide the retention period with your compliance owner. Export only the fields and window you must keep.
Log backend build-out (Langfuse or OpenTelemetry)	Langfuse needs PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage, while OpenTelemetry needs a collector and a trace backend. Either is a platform to size, secure and back up.	Reuse existing observability or object storage where your standards allow. Confirm the Langfuse component minimums in its scaling guide before sizing.
SSO and audit requirements	SSO beyond the free user limit and audit logs sit behind LiteLLM Enterprise, so an enterprise license becomes a recurring line item and a dependency.	Count the admin and developer users who need SSO early. Confirm current tier limits with LiteLLM before design.
Production gateway infrastructure	PostgreSQL is the primary supported database, Redis 7.0 or newer is required once several proxy instances run, and each worker needs sizing and secret handling.	Start from LiteLLM's production guide. Confirm instance counts, high availability needs and secret manager use early.
Environments to validate	Each of development, staging and production needs its own configs, keys, provider credentials and test runs, so validation repeats per environment.	Promote the same versioned configuration through environments instead of editing each one by hand.

What usually goes wrong and how to prevent it

A Portkey to LiteLLM move rarely fails in one dramatic place. It fails in small gaps between the two systems: keys, budgets, fallbacks, credentials, logs and state. The cards below name the risks we see most often and the check that prevents each one.

Keys copied with the wrong scope

A LiteLLM virtual key is only as narrow as the model list, budget and team attached to it. Copying a Portkey virtual key's intent without re-deriving its limits can give a service access to models or spend it never had. Before cutover, export the list of Portkey keys and their allowed providers, then create each LiteLLM key with an explicit model restriction and budget. Test each key with a request that should be denied.

Budgets that diverge in parallel runs

While traffic runs through both gateways, spend is counted in two places, so per-team budgets can be exhausted in one system and still open in the other. Budgets also reset on different schedules if the two systems were configured separately. Decide which gateway is the budget authority for each team during the parallel run, freeze the other side, and reconcile spend totals from both sources at the end of each day.

Fallback chains that behave differently

Portkey fallback and retry settings do not map one to one onto LiteLLM's router. Retry counts, cooldown after repeated failures, and priority ordering can change which provider answers when a primary fails, and therefore cost, latency and output style. Write the expected provider order for each route, then trigger a forced failure on the primary in staging and confirm the same provider answers in both gateways.

Missing provider credentials

Portkey often holds provider keys in its own configuration, so the new gateway may start without the credentials for a secondary provider. The failure appears only when a fallback fires, often at night or during an outage, which is the moment the fallback is needed. Inventory every provider and model referenced in Portkey configs, confirm each has a working credential in LiteLLM, and send a test request to each secondary route before cutover.

Logs lost at cutover

Portkey's hosted logs disappear from the working view once traffic moves, and logs from the last days before cutover may be hard to find later. LiteLLM sends logs only to the destinations you configure, such as Langfuse, OpenTelemetry or Prometheus. Confirm that a request through LiteLLM appears in the trace store, and export the Portkey history you need for audit before the subscription changes. Keep the export location and retention in the cutover record.

Missing Redis or PostgreSQL on a second replica

A single LiteLLM proxy can start with local state, but a second replica without shared storage gives each instance its own rate limits, router state and cache. Virtual keys also need PostgreSQL. LiteLLM's production guide names PostgreSQL as the primary database and requires Redis 7.0 or newer once more than one instance runs. Check both services are reachable from every pod before scaling out, and test a rate limit that is shared across two replicas.

Secrets left in config files

Migrating from a hosted service often means copying provider keys and the LiteLLM master key into a YAML file or environment file checked into Git. That file then becomes a copy of every credential the gateway uses. Keep the master key and provider credentials in a secret manager, reference them from config, and scan the repository for key patterns before the first commit. Rotate any key that was ever pasted into a shared document during the move.

How the migration differs by team size and starting point

A single team using Portkey Cloud usually has a small number of virtual keys, a handful of routing configs and logs that the team reads mostly in the Portkey dashboard. The migration can be sequenced in a short run: stand up one LiteLLM proxy with PostgreSQL, recreate the providers and fallbacks, and move one non-critical application first. Because the hosted logs are the main history, decide early whether you need an export before the switch, and whether Langfuse or OpenTelemetry will hold new traces.

Key handling: Issue new LiteLLM virtual keys per application rather than copying Portkey keys, and roll them out one application at a time.
Routing: Translate each Portkey config into LiteLLM router settings and test fallbacks with a deliberate provider failure in staging.
Logs: Choose one logging destination on day one, such as a self-hosted Langfuse instance, so new traces do not depend on Portkey.
Single-instance operation: A single proxy may run without Redis, but plan for Redis 7.0 or newer before adding a second instance.
Review: Confirm with the provider contracts and your own data policy where prompts and responses are sent after the switch.

Before cutover, confirm the export options available on your current Portkey plan with Portkey directly.

A platform team running Portkey's hybrid deployment already keeps the gateway data plane in its own VPC, so the main change is moving the control plane. Configuration, virtual keys and logs that lived in Portkey's control plane must be recreated in LiteLLM's database and log stores inside the VPC. Sequencing matters more here because several application teams depend on a shared gateway. Build the new proxy beside the existing data plane, migrate one tenant at a time, and keep the Portkey route available until each team has verified its traffic.

Key handling: Map team and user budgets to LiteLLM teams and users, and reissue keys through a controlled process so teams do not share a single master key.
Multi-instance state: Plan for Redis for shared rate limits and router state once several proxy instances run, and keep LITELLM_MASTER_KEY in a secret manager.
Logs and configuration: Record where Portkey's control plane stored configuration and logs, then decide which of these move to PostgreSQL, Langfuse and object storage.
Enterprise controls: SSO beyond the free limit, audit logs and team RBAC need a LiteLLM Enterprise license, so confirm which controls your review requires.
Egress: Remove the outbound HTTPS path to the Portkey control plane only after the new gateway handles every route.

Check which team owns each virtual key and budget before the first tenant moves.

Questions buyers ask before moving from Portkey to LiteLLM

How much effort and time does moving from Portkey to LiteLLM take?

The effort depends mostly on how many applications call Portkey, how many routing configs and virtual keys you have, and how much logging history you must keep. No fixed duration is set here, so ask for one in the written scope after an inventory. The main drivers are:

The number of applications and SDK integrations that point at the Portkey endpoint
The count of virtual keys, budgets and fallback chains that must be recreated as LiteLLM keys and routing settings
Whether the LiteLLM production stack (PostgreSQL, and Redis 7.0 or newer once more than one proxy instance runs) is already in place or must be built
The observability destination you choose, such as Langfuse, which carries its own deployment work

Can our internal team migrate from Portkey to LiteLLM without a partner?

Yes, many teams with platform engineering experience can do it internally, provided they have capacity for the gateway, database and logging work. The skills that matter are Kubernetes or container operations, PostgreSQL administration, secret handling and familiarity with OpenAI-compatible request formats. Teams that lack these skills, or that need a fixed deadline, often bring in outside help for a written, fixed-scope engagement. Confirm internal skills and staffing before you commit to a cutover date.

What does the migration need from our teams in access, owners and change windows?

The move needs named owners for each application team, access to the Portkey configuration and virtual key inventory, and permission to deploy into your cloud account or cluster. You also need a change window for each application cutover, because applications must switch their base URL and API key. Ask your team to confirm who approves production changes, who can read secrets, and which applications are critical enough to need a staged rollout.

Does SSO for the LiteLLM admin UI require LiteLLM Enterprise?

Not always. LiteLLM's admin UI supports SSO with Google, Microsoft Entra ID (Azure AD), Okta and generic OIDC providers, and from v1.76.0 SSO is free for up to 5 users. An Enterprise license is needed beyond that limit. SCIM, JWT authentication with identity provider tokens, audit logs and organization and team RBAC are Enterprise features. Count your admin users and check the current LiteLLM Enterprise documentation before you assume the free tier covers you.

How do we reissue virtual keys to applications without downtime?

The usual approach is to create the new LiteLLM keys first, with budgets and model restrictions matching the Portkey keys they replace, then roll them into each application one at a time while the old path stays available. LiteLLM virtual keys need a PostgreSQL database, so the database must be running and backed up before keys are issued. Ask the implementation team to document the rollback step for each application, and confirm that your secret manager can hold the new keys, since LITELLM_MASTER_KEY must start with sk-.

What happens to our Portkey log history after we switch gateways?

Portkey history stays in Portkey unless you export it, and the export is a separate step. Portkey's logs export is an Enterprise feature, so confirm whether your plan includes it before you plan around it. New requests will be logged in the destination you configure, such as Langfuse or OpenTelemetry, starting at cutover, so there will be a gap between the two stores. Decide early whether you need a long-term archive, and ask your team to confirm retention rules for both systems.

Where do our prompts, traffic and logs go once LiteLLM runs in our infrastructure?

When LiteLLM and its logging backend run in your own environment, prompts, responses, keys and spend records stay inside your network, except for calls you make to the model providers themselves. Unlike a hybrid setup, there is no required outbound connection to a vendor control plane. Confirm the exact egress rules with your security team, including any provider endpoints, telemetry settings and the location of Langfuse's PostgreSQL, ClickHouse and blob storage.

How DeployOpen scopes and runs the migration

DeployOpen plans, implements, integrates, hardens, backs up and documents self-hosted open-source platforms inside your own infrastructure, which for this migration means your cloud accounts, data centers or air-gapped network. The written scope comes first. Before any work starts, the scope document should name the Portkey features in use (virtual keys, budgets, routing configs, fallbacks, hosted logs, any prompt management or semantic caching), the applications that call the gateway, the LiteLLM deployment target, the PostgreSQL and Redis footprint, and the log destination you choose. It should also list the deliverables, such as the mapped configuration, the cutover runbook, the backup and restore procedure and the documentation, along with the timelines both sides agree to.

Most engagements are fixed-scope, so the decisions that change cost are made early and recorded. Ask for a quote that separates one-time work (configuration mapping, key reissue plan, observability wiring, hardening and testing) from recurring work (upgrades, on-call support and continuous testing). Confirm whether enterprise controls such as SSO beyond the free limit, audit logs or team RBAC are in the scope, because those depend on LiteLLM Enterprise licensing that DeployOpen does not resell. Quotes follow from the written scope, and DeployOpen does not publish a price list, so the scope document is the basis for comparing offers. Ask any vendor, including DeployOpen, to show exactly which items are included.

Once cutover is complete, the operating work continues. LiteLLM's guidance on upgrades, supported minor versions and Redis once several proxy instances run all affect the ongoing workload. Ongoing support, upgrades and continuous testing are available as retainers that run after the migration closes, and they can be scoped separately from the migration itself. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. If your review requires an independent auditor, confirm early with that auditor which evidence they expect from the gateway and logging stack.