Move from Portkey Cloud to the open-source Portkey Gateway

Running only Portkey's MIT-licensed gateway in your own infrastructure keeps the routing format and core reliability features, but prompt management, semantic caching, advanced logging and access control need replacements. This page sets out what moves, in what order, and what drives the work.

On this page

Migrate from Portkey Cloud to the open-source gateway: what the move involves

The open-source Portkey Gateway keeps the routing layer but not the hosted control plane, so moving from Portkey Cloud or hybrid means taking over everything Portkey's console and enterprise tier used to do for you. Teams usually start this work after a renewal, a data residency review, or a security concern about configuration and logs leaving the network through the outbound HTTPS path to control.portkey.ai in a hybrid setup. The move is typically led by a platform or ML platform team, with security and compliance signing off on where keys, prompts and logs will live afterward. The routing side is the easier part. Fallbacks, retries, load balancing and simple caching are part of the MIT-licensed gateway, which runs with npx, Docker, Node.js, Cloudflare Workers or Kubernetes, so your existing config objects can usually be carried over with testing. The real effort sits elsewhere: prompt management, semantic caching, advanced logging and tracing, access control and PII redaction are listed as enterprise features outside the open-source repository. Confirm in the repository which guardrail checks it ships, and plan replacements for each enterprise feature your teams rely on.

Expect the work to concentrate on three areas: logging, key management and the features that moved behind the enterprise line. Hosted logs need a new destination, such as your own observability stack, and you should decide what fields you retain, how long they live and who can read them. Virtual keys, budgets and per-team access will also need a new home, since they no longer sit in Portkey's console. Ask Portkey for the current terms of your contract, including any export rights for historical logs, before you decommission anything. For cost, the open-source gateway removes the platform fee but adds compute, the storage and database behind logging and keys, and the staff time to patch, monitor and upgrade the gateway. Those costs recur, while the migration design, testing and cutover are one-time. Fixed-scope help with the design, integration and documentation is one option to weigh against internal capacity, and any quote should list the components, environments and acceptance tests in writing before work starts.

Which Portkey features carry over to the open-source gateway

This table maps each Portkey feature to the open-source Portkey Gateway or a replacement tool, labeled clean, needs rework or gap. Portkey's repository lists prompt management, semantic caching, advanced logging and tracing, access control and PII redaction as enterprise features outside the open-source gateway. Confirm each row against the gateway version you deploy, because feature sets change between releases.

Portkey featureStatus in open-source gatewayReplacement or next step
Routing formatCleanPoint existing client base URLs at your gateway and confirm the config schema for your version.
Fallbacks, retries and load balancingCleanTest fallback order, retry limits and weights against each provider in staging before cutover.
Simple cachingCleanConfirm the cache key scope and storage backend, and check that cached responses respect tenant boundaries.
GuardrailsNeeds reworkHosted guardrail checks depend on plan tier. Run input and output checks beside the gateway and confirm which hooks your version exposes.
Virtual keys and budgetsNeeds reworkHosted keys live in Portkey's control plane. Recreate keys and budgets in a self-hosted key store, such as LiteLLM virtual keys on PostgreSQL.
Prompt managementGapEnterprise feature outside the open-source gateway. Move prompt versions and labels to a self-hosted tool such as Langfuse.
Semantic cachingGapEnterprise feature outside the open-source gateway. Evaluate a self-hosted semantic cache and tune its similarity threshold on your own prompts.
Advanced logging and tracingGapEnterprise feature outside the open-source gateway. Export traces with OpenTelemetry to Langfuse or your existing observability stack.
Access control and PII redactionGapEnterprise features outside the open-source gateway. Add SSO at the front door and a self-hosted PII scanner before requests reach providers.

What stays in the gateway and what moves to other tools

The open-source Portkey Gateway can run with npx, Docker, Node.js, Cloudflare Workers or Kubernetes, so the deployment target changes while the request format your applications send can stay the same. Your applications keep calling the gateway endpoint with the routing configuration they already use, and the work moves to where the gateway runs: a container image on your own cluster, a Node process on a VM, or a worker in an environment you control. The first decision is the runtime. Choose the target that matches your network controls, scaling model and patching process, then confirm with your engineers that the gateway in that target handles the fallbacks, retries, load balancing and guardrails your routing config depends on. Confirm the exact feature list against the gateway's own repository, since the enterprise boundary is documented there and may change.

The gap sits in what the open-source gateway does not include. Portkey's repository lists prompt management, semantic caching, advanced logging and tracing, access control and PII redaction as enterprise features outside the open-source gateway, so the hosted dashboard that held your request logs, prompt versions and access rules does not come with it. Teams usually pair the gateway with a self-hosted trace store, such as Langfuse or OpenTelemetry-based tooling, so that request metadata, latency, token counts and errors still have a home inside your network. Plan that store before cutover, because it adds its own database, storage and upgrade work. Also decide where prompt templates will live once the hosted prompt registry is no longer available, and who owns access to them.

Which data and integrations to move, and which need rebuilding

A move from Portkey Cloud touches five kinds of material: routing configuration, provider credentials, virtual key and budget records, hosted logs, and the applications that call the gateway. Each one has a different export path and a different owner, so inventory them separately before any traffic changes.

Routing configs and fallback rules

Portkey configs define the routing, retry, fallback and load balancing behavior that applications rely on. Copy each config into the gateway's own configuration format and test it against the same provider list. The open-source gateway keeps Portkey's routing format, but confirm each config key your team uses is supported in the MIT-licensed version before you retire the hosted config. Check guardrail references too, since guardrails sit outside the open-source gateway.

Provider credentials in your secret manager

Provider API keys for OpenAI, Anthropic, Bedrock, Azure and other providers should move into a secret manager you control, then be injected into the gateway at runtime. Rotate every key that was stored in Portkey's console during the move, since the old copies sit outside your environment. Confirm with Portkey how credentials were stored on your plan and whether any were held as virtual keys that must be reissued.

Virtual keys and budget records

If your team issued Portkey virtual keys or set budgets and rate limits, those records do not carry over automatically to the open-source gateway. Write down each key's owner, allowed models, spend limit and rate limit, then decide which controls the open-source gateway covers and which need a replacement. Plan the reissue of application keys as a scheduled change, because applications will fail once an old key is revoked.

Hosted logs and their export path

Request logs, traces and cost data in Portkey Cloud are the hardest material to move, because they live in Portkey's store. Ask Portkey which log export and API options apply to your plan, and read the export guide before you plan the cutover, since export availability depends on plan and feature set. Decide where logs will go after the switch, such as OpenTelemetry, Langfuse or your existing log platform, and confirm the retention you need in that destination.

Application calls to the gateway endpoint

Applications call the gateway through a base URL, headers and sometimes a config identifier. Inventory every service, script and notebook that sends traffic through Portkey, including scheduled jobs that teams often forget. Change the endpoint in a staged rollout, keep the request format identical where possible, and monitor error rates per service so a missed caller is caught quickly.

Enterprise-only features that need rebuilding

Prompt management, semantic caching, access control and PII redaction are listed as enterprise features outside the open-source gateway. Any prompt templates, cache settings or access rules you depended on must be rebuilt in another component or in your own policy layer. List these dependencies early, because they shape the scope and the owners of the new design.

Migration sequence from inventory to decommission

This sequence moves gateway traffic from Portkey Cloud to the open-source Portkey Gateway in steps that each produce evidence, so you can confirm behavior before any application depends on the new path and retire the hosted control plane only when nothing still needs it.

  1. Inventory configs, routes, keys, prompts and logs

    Platform engineering and the application owners list every Portkey config, route, virtual key, provider credential, prompt and guardrail in use, and record which applications call each one. Export the data you must keep, including logs, using the export options your plan allows. Confirm with Portkey which items are enterprise features that the open-source gateway does not include, such as prompt management, semantic caching, advanced logging and PII redaction, because each one needs a replacement decision. The step closes with a signed inventory that marks every item as keep, replace or retire.

  2. Run a pilot route on the gateway in your VPC

    Deploy the MIT-licensed Portkey Gateway in your own VPC or Kubernetes cluster, using one of the supported runtimes such as Docker or Kubernetes. Recreate one low-risk route with its fallbacks, retries and load balancing, using provider keys stored in your secret manager. Send only test traffic from a non-production application. Confirm that the gateway makes no calls to Portkey's control plane, and check the network egress rules to prove it. The step is done when the pilot route returns correct responses and the egress check passes.

  3. Compare outputs in a parallel run

    Split a defined share of production traffic between Portkey Cloud and the self-hosted gateway for the same route. Compare status codes, latency, token counts, fallback behavior and response content on matched requests, and record any differences. Review differences with the application owners, because a change in model output can come from a changed provider key, a model alias or a different default setting, not from the gateway itself. The step closes with a written comparison report that the application owners approve, or a list of fixes that must be retested before the next gate.

  4. Cut over each application

    Move applications one at a time by changing their base URL and gateway key to the self-hosted endpoint, and keep the previous Portkey path ready for rollback. Start with the applications whose traffic and data are least sensitive. Watch error rates, latency and spend after each change before moving the next application. For each application, confirm that its logs and traces reach the logging destination you chose, since hosted logs will no longer populate. The step is done when every application in the inventory points to the self-hosted gateway and the rollback path has been tested.

  5. Decommission the hosted control plane

    After every application has run on the self-hosted gateway through your agreed retention period, export any remaining records you need for audit or legal reasons. Confirm with Portkey how long your account data is kept after cancellation and request written confirmation of deletion. Revoke Portkey API keys and provider credentials that were stored there, rotate any secrets the hosted service could have seen, and remove the old routes. The step closes with a decommission record that lists each item removed, the date, the approver and the evidence of deletion.

How the move differs by starting point

Teams on Portkey Cloud send traffic to a hosted gateway, so the first change is moving the data plane into infrastructure you control. Application traffic, provider keys and request logs all currently live in Portkey's environment, which means the move has to rebuild the parts Portkey operated for you. Start by deciding where the gateway will run (npx, Docker, Node.js, Cloudflare Workers or Kubernetes, as listed in Portkey's repository) and where logs will go. Then point applications at the new endpoint in stages, keeping the old route available until traffic is confirmed.

  • Gateway hosting: You take over availability, scaling and patching of the gateway process itself.
  • Provider credentials: Provider keys move from Portkey's vault into your own secret store, and you own their rotation.
  • Logging: Hosted logs are lost unless you export history first and choose a replacement destination such as Langfuse or OpenTelemetry.
  • Cutover: Run both paths in parallel for a period, comparing responses and error rates before retiring the Portkey route.
  • Enterprise features: Hosted features that depend on Portkey's platform need an open-source replacement or a documented decision to drop them.

Before committing, confirm which log history you need to keep and whether your Portkey plan permits the export you require.

What drives the cost and duration of the migration

This table lists the factors that raise or lower the effort and running cost of moving from Portkey Cloud to a self-run Portkey Gateway. Read each row as a lever: the driver, why it changes the work, and how to keep it contained. The main takeaway is that the gateway itself is the cheap part, while the platform team's time and the surrounding logging and identity components usually set the total.

DriverWhy it changes cost or effortHow to keep it down
Number of routes and providersEach route, fallback chain and provider credential must be rewritten, tested and kept in version control, so effort grows with the config surface.Inventory every config before starting, group near-identical routes into shared templates, and retire unused routes first.
Number of applications calling the gatewayEvery application that points at the Portkey endpoint needs a base URL change, a key rotation and a regression check during cutover.Move applications in waves, starting with the lowest-risk ones, and keep the gateway URL in one shared configuration value.
Volume of stored prompts and logsLogs and prompt history stay in your own store once Portkey Cloud is removed, so storage, retention rules and export work grow with history kept.Decide the retention window early, export only the history you need for audits, and use Portkey's export before cancelling.
Depth of virtual key and budget usePer-key, per-team and per-user budgets, rate limits and model restrictions must be recreated and tested, and the key store adds a database to run.Start with team-level budgets and add finer controls only where finance or security require them.
Required SSO and audit trailsSingle sign-on and audit logging for the gateway's admin surface may sit behind a licensed enterprise component in the open-source options, adding a recurring license cost.Confirm early which controls are mandatory, check the current license terms with the vendor, and avoid licensed features that no requirement names.
Prompt management and semantic cachingThese are enterprise features outside the open-source gateway, so each one needs a replacement component, its own tests and its own operating work.Measure how much of each feature is in use before choosing a replacement, and drop features that no application depends on.
Hosting target (Kubernetes, Docker or serverless)Kubernetes adds cluster and networking work if your platform team does not already run it, while Docker or a single host is faster to set up but needs its own availability plan.Reuse the platform your team already operates, and size high availability to the business impact of a gateway outage.
Platform team time and operating modelMost of the effort is engineering and operations time for config migration, testing, upgrades and on-call, which is paid staff time rather than a vendor fee.Ask any implementation partner for a fixed scope with named deliverables, and agree in writing who owns upgrades after handover.

What usually goes wrong and how to prevent it

Moving from Portkey Cloud to the open-source gateway fails less often on routing syntax than on what sits around it: history, behavior under failure, prompts, keys, rollback and feature assumptions. Each card names one risk and the check that prevents it before traffic moves.

Losing history before exporting logs

Hosted gateway logs are the record of what your applications sent and received, and they may be the only copy you have. Bulk log export from Portkey is an Enterprise-only capability, so confirm your plan includes it before you plan the cutover. Export and verify a full copy of logs before you cancel the hosted account, and confirm that retention and audit needs are covered by your own store.

Silent differences in fallbacks and retries

A fallback chain that worked in Portkey can behave differently in the open-source gateway, changing which provider answers, how many attempts run, and how long a request waits. Those changes appear as latency spikes or duplicate spend, not as errors. Replay a sample of production requests against the self-hosted gateway, and compare the chosen provider, attempt count and timeouts with the hosted logs.

Prompts left behind in the hosted console

Teams often keep prompt templates in the Portkey console and reference them by ID from application code. Prompt management is an enterprise feature outside the open-source gateway, so those IDs stop resolving after the switch. Inventory every prompt ID referenced in code and config, then move each template into a file or prompt store you control before the first application is repointed.

Provider keys stored in plain config

Migrating a gateway config often means copying provider API keys into a YAML file or environment settings on a shared host. Anyone with read access to that file or the container spec can then call the providers directly, outside your budgets and logs. Keep provider keys in a secret manager or the Kubernetes secret store, restrict read access to the gateway service account, and rotate the keys that were used in the hosted console.

Cutover without a rollback path

Switching every application to the self-hosted gateway in one change leaves no quick way back if latency, errors or cost move unexpectedly. Keep the hosted Portkey route available behind a configuration switch for the first production window, and move traffic by service or by percentage. Define in advance the error rate and latency thresholds that trigger a return to the hosted route.

Assuming enterprise features exist in the open-source gateway

The open-source gateway covers routing and resilience, but Portkey's repository lists prompt management, semantic caching, advanced logging and tracing, access control and PII redaction as enterprise features. Teams that assume these come along often discover the gap after cutover. Map each feature your teams use against the open-source gateway and the replacement you will run, and confirm the gaps in writing before the migration plan is approved.

Questions teams ask before moving off Portkey Cloud

How much effort and cost does moving off Portkey Cloud involve?

The largest cost is the work of moving, not the software, because the Portkey AI Gateway is MIT licensed and carries no license fee to run. Expect one-time costs for planning, translating routing configs and virtual keys into your own setup, testing fallbacks and cutting over applications. Recurring costs include compute for gateway replicas in your VPC or cluster, any shared store you add for state or caching, the logging and tracing backend you choose, and staff time for upgrades and on-call. Costs rise with the number of applications, providers and enterprise features you must replace. In any quote, ask for a written list of components, who pays for each, and how long Portkey Cloud stays active during cutover.

How long does a move to the self-hosted Portkey Gateway take?

No duration is fixed by the open-source project, so the timeline depends on your scope and should be stated in a written plan. The main drivers are how many applications call Portkey, how many routing configs, fallbacks and virtual keys need mapping, which enterprise features you must replace, where logs must land, and how much testing your change and security processes require. Teams that already run containers and have a logging backend usually move faster than teams building both at once. A staged cutover runs the self-hosted gateway beside Portkey, sends a small share of traffic through it, compares results, then moves the rest. Keep Portkey Cloud active until replacement logging and access controls are verified.

What skills and ownership does the team need to run the gateway?

The team needs someone who can deploy a containerized service with Docker or on Kubernetes, manage provider API keys in a secret manager, configure network egress and ingress, and connect metrics, traces and logs to your existing stack. Application engineers must update base URLs, headers and model names, then test fallback behavior by forcing failures on purpose. Security must review where keys, prompts and logs are stored and who can read them. Assign a named owner for upgrades, since you now carry that work, and another for the enterprise features you replace, such as prompt versions and caching, which now live in your systems. Without clear owners, the gateway tends to drift from the configuration that Portkey used to hold.

Can our own team migrate off Portkey Cloud without outside help?

A small gateway with few routes is often within reach of an experienced platform team, but the answer depends on your skills and how much of the replacement is custom. DeployOpen plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts, and ongoing support, upgrades and continuous testing are available as retainers. DeployOpen does not publish a price list, so a quote follows from the written scope. Ask any provider to split the work into gateway setup, feature replacement, logging, key migration, testing and documentation, so you can see which parts your team keeps.

Which Portkey features are lost when we run only the open-source gateway?

The open-source gateway keeps the core routing layer, so most of the work is replacing the enterprise features that sit outside it. Portkey's repository lists prompt management, semantic caching, advanced logging and tracing, access control and PII redaction as enterprise features. The core routing features in this migration path are fallbacks, retries, load balancing and simple caching, which you should confirm against the Portkey AI Gateway repository for your version. Prompt management moves to a versioned store you control, and semantic caching needs its own store and threshold tuning. Logging moves to your tracing stack, while access control and PII redaction need your own identity and filtering controls. Check guardrails feature by feature, because Portkey's guardrail tiers depend on plan.

Does the self-hosted gateway still report to Portkey?

It depends on the deployment model you leave behind. In Portkey's hybrid model, the gateway runs in your VPC while the control plane stays in Portkey's VPC, and the gateway calls out over HTTPS to control.portkey.ai, as described in the hybrid deployment documentation. Running only the open-source gateway means you define configuration, keys and logs yourself, so routing no longer depends on a Portkey control plane. Confirm this in your environment rather than relying on the description alone. Block egress to control.portkey.ai in a test environment, replay your full set of request types, and check that nothing fails or sends data out. Also review any SDK, header or client setting that still points at Portkey.

Does moving the gateway in-house change our security and compliance review?

Yes. The gateway holds provider API keys, sees prompts and responses, and writes logs, so your review must cover where each of these lives, who can read them and how keys rotate. Store provider keys in a secret manager, restrict network paths to model providers, and protect logs at rest and in transit. Your current Portkey vendor assessment covered a third party that handled routing data, so the new review shifts that scope to your own controls and operating procedures. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those assessments, so ask them early which evidence they will expect.

Where outside help changes the outcome

The gateway itself is usually the easy part. The harder work is knowing what your current Portkey setup actually does: which virtual keys and configs route which traffic, which guardrails and prompt templates are deployed through the gateway, and which applications depend on Portkey-hosted logs. DeployOpen can run a written inventory and gap assessment that maps each of these to a component of the open-source Portkey Gateway, marks the enterprise features with no direct replacement (prompt management, semantic caching, advanced logging and tracing, access control and PII redaction, according to the gateway repository), and records which replacement you choose for each. That gap list is the input to every later decision, so ask for it as a named deliverable.

Deployment is the next scoped item. DeployOpen can deploy the gateway in your own cloud account, data center or air-gapped network using the runtime you pick (npx, Docker, Node.js, Cloudflare Workers or Kubernetes), wire it to your identity provider and secret store, and harden the configuration. Because the open-source gateway does not call Portkey's control plane, the hardening work is about your own perimeter, egress rules, key handling and patching, and that ownership should be written into the scope. Replacement logging and key controls are separate deliverables, and they should be scoped against the gap list rather than assumed.

Parallel run support and decommission evidence close the engagement. During a parallel run, DeployOpen can help compare routing outcomes, fallback behavior and log completeness between Portkey and the self-hosted gateway, and document the cutover criteria you agree on. Decommission evidence records which keys were revoked, which configs were removed, and where historical logs were exported, so your auditors can see the change. These engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts, and an accurate quote follows from that written scope. Ask each proposal to price the inventory, deployment, logging, parallel run and decommission items separately so you can see what is one-time and what needs an ongoing retainer.

How DeployOpen scopes, runs and supports the migration

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a move from Portkey Cloud to the open-source Portkey Gateway, we start with a written scope that maps your current routing configs, fallbacks, guardrails and logging to what the MIT-licensed gateway covers and what needs a replacement.

How we scope

  • Inventory every Portkey config, virtual key, fallback chain and guardrail that applications depend on today.
  • Mark which features stay in the open-source gateway and which enterprise features need a named replacement.
  • Agree deliverables, timelines and exclusions in writing before any cutover work starts, since engagements are fixed-scope.

How we build

  • Deploy the Portkey AI Gateway in your environment using the runtime your platform team already operates, such as Docker or Kubernetes.
  • Rebuild retries, fallbacks, load balancing and simple caching as gateway configuration and test each route against real provider errors.
  • Test guardrail checks and logging destinations, and confirm no request or log leaves your network unless you approved that path.

How we hand over

  • Document the gateway configuration, deployment manifests, secrets handling and the list of features that moved or were replaced.
  • Run a cutover plan with rollback steps, so applications switch endpoints in stages and each team knows what changed.
  • Deliver runbooks and evidence for your internal reviews. DeployOpen prepares this material but does not issue audit reports or certifications.

How we support

  • Support is available as a retainer for patching, gateway upgrades and re-running tests after provider or config changes.
  • Continuous security testing of the gateway and its routes can be added to the same retainer if your risk review requires it.
  • Ask for a quote based on the written scope, including the effort for each replacement feature and who operates it after handover.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.