Portkey or LiteLLM: which fits your gateway decision
Most teams reach this decision because a Portkey renewal arrives, a data residency review asks where prompts and keys live, or an internal rule requires gateway keys, budgets and logs to stay inside the network. Portkey fits teams that want a vendor-run control plane and its hosted logging, with the gateway deployed either fully in the cloud or in the customer's VPC under the hybrid model. LiteLLM fits teams that want the gateway, virtual keys, budgets and logs running entirely in their own infrastructure, backed by their own PostgreSQL and, once more than one proxy instance runs, Redis. The starting point is the data flow, not the feature list: ask Portkey exactly which endpoints the gateway calls during normal operation, and map each one against your egress policy.
In hybrid deployment, Portkey keeps the control plane in its own VPC while the gateway in your VPC calls out over HTTPS to control.portkey.ai, as its hybrid deployment documentation describes. That split may be acceptable for a regulated workload, or it may rule Portkey out for disconnected networks. A fully self-hosted LiteLLM proxy has no such dependency, but you take on the work of operating it. Confirm with your security team whether a control plane outside the network is acceptable before comparing feature lists, because that answer removes options early.
Three questions decide the rest. First, do you need SSO beyond five users? LiteLLM's free SSO covers up to 5 users from v1.76.0, and SAML, SCIM, audit logs and team RBAC sit behind an Enterprise license, so request that price in writing. Second, which logging destination will your auditors accept? LiteLLM can send logs to Langfuse, Datadog, OpenTelemetry and Prometheus, while Portkey's advanced logging is hosted. Third, who will operate the gateway: someone must own database backups, upgrades and provider credentials. If no internal team can carry that, a fixed-scope implementation with a support retainer is a realistic option to price against the Portkey renewal.
Which gateway fits which scenario
This table matches common operating conditions to the gateway that usually fits better, with the reason and what you must confirm in your own environment. Read each row as a starting point, then test it against your network rules, team skills and identity setup. The main takeaway is that LiteLLM fits most self-run scenarios, while Portkey's hybrid model suits teams that accept an outbound control plane and want managed tooling.
| Scenario | Better fit | Reason and what to confirm |
|---|---|---|
| Air-gapped or strict egress network | LiteLLM | Portkey's hybrid model calls out over HTTPS to control.portkey.ai, so confirm whether your policy allows that path. LiteLLM runs as a self-hosted server, but confirm each provider and logging endpoint it needs is reachable from your network. |
| Strict egress, outbound HTTPS allowed to one vendor | Either, depending on policy | Hybrid Portkey keeps the gateway in your VPC, but configuration and control traffic leave it. Confirm with security whether control-plane traffic and metadata are acceptable before choosing. |
| No dedicated platform engineers | Portkey hosted or hybrid | A managed control plane reduces the in-house operating work. Self-run LiteLLM needs PostgreSQL, Redis for multiple instances and upgrades, so confirm who on your team owns those tasks. |
| Small team with a single application | LiteLLM (illustrative) | A single proxy with virtual keys can cover one application. Confirm your team can operate one database and one proxy deployment before committing. |
| Multi-team budgets and model access | LiteLLM | Per-key, per-user and per-team budgets, TPM and RPM limits and model restrictions are documented LiteLLM virtual key features. Confirm which of these your teams need, since team-level guardrails and logging are Enterprise features. |
| SSO for more than five admin users | Portkey or LiteLLM Enterprise | From v1.76.0, LiteLLM SSO is free for up to 5 users, and Enterprise is needed beyond that. Ask LiteLLM for current Enterprise terms and ask Portkey which SSO features sit on your plan. |
| Existing Kubernetes operations | LiteLLM | LiteLLM recommends one Uvicorn worker per pod and horizontal scaling, with the master key in a secret manager. Confirm your cluster team can run PostgreSQL and Redis alongside the gateway. |
| Log destination inside your own stack | LiteLLM | LiteLLM can send logs to Langfuse, Datadog, OpenTelemetry and Prometheus. Confirm which destination your audit team accepts, since Portkey's advanced logging and tracing are enterprise features in its open-source repository. |
Where traffic and logs go in each model
The question is less which gateway is faster than which feature list and more where prompts, responses, keys and logs travel. Portkey's hybrid model and a self-run LiteLLM proxy put the boundary in different places, and each card below is a check your security and compliance reviewers will ask about.
Hybrid control plane traffic
In Portkey's hybrid deployment, the gateway (the data plane) runs in your VPC while configuration and the control plane stay in Portkey's VPC. The gateway calls out over HTTPS to control.portkey.ai, so outbound connectivity to that host is a design decision you must document. Confirm with Portkey exactly which metadata, configuration and log fields reach that endpoint, and whether any prompt or response content is included.
Self-run proxy log destinations
A self-hosted LiteLLM proxy has no vendor control plane in the request path. Logs go only to the backends you configure, such as Langfuse, Datadog, OpenTelemetry or Prometheus, and each of those becomes a data store you must classify, secure and retain. Map every callback in your configuration to a named system with an owner before go-live.
Identity and admin access
Portkey's hosted and hybrid administration runs in Portkey's environment, so your administrator identities and their actions sit partly outside your tenant boundary. With LiteLLM, the admin UI and SSO run wherever you deploy them. Note that SSO with Google, Microsoft Entra ID, Okta or generic OIDC is free for up to 5 users, and an Enterprise license is needed beyond that. Budget for it if many users need the admin UI.
Data residency and provider routing
Residency depends on two paths: where the gateway runs and where each model provider processes the request. In either model, the gateway forwards prompts to OpenAI, Anthropic, Bedrock, Azure or other providers you configure, so the gateway location alone does not settle residency. Ask each provider for its processing region and retention terms, and record them alongside your gateway design.
Compliance review scope
Hybrid and self-run designs pass different reviews. A hybrid model adds a third-party processor and its sub-processors to vendor risk management. A self-run proxy shifts patching, access reviews, backup and log retention to your team and expands the scope of your own control evidence. DeployOpen prepares systems, controls and evidence for assessments, but it does not issue audit reports, attestations or certifications; independent auditors do.
Questions to put to Portkey
Before a renewal or a design sign-off, get written answers on four points: the exact data sent to the control plane, whether any content is retained there and for how long, where the control plane is hosted, and how the hybrid gateway behaves if that connection is unavailable. Ask for the answers in writing and attach them to your data flow diagram.
Hosting options by environment
In the hosted model, Portkey runs the gateway and the control plane in its own cloud. Your applications send LLM requests to Portkey, which forwards them to the model providers you configure. Portkey operates the gateway, its availability and its upgrades, and you rely on its contractual and technical controls for where requests, logs and configuration are processed. Your team manages configs, virtual keys and workspace settings through Portkey's interface and API rather than on infrastructure you control.
- Outbound connectivity: Application traffic goes to Portkey's endpoints, and Portkey then calls each model provider. Your network only needs outbound access to Portkey and to the providers' APIs.
- What you operate: Very little on the gateway itself. You manage configs, keys and provider credentials inside Portkey's console.
- What you control: Plan tier, feature access, log retention settings and the contract terms that govern data handling.
- What leaves your network: Prompts, responses and logs pass through Portkey's infrastructure. Confirm the processing locations and sub-processors in writing.
- Compliance reviews: Expect your assessor to ask for Portkey's third-party reports and data processing terms, which Portkey supplies, not DeployOpen.
Before you choose this model, confirm with Portkey where logs are stored and which enterprise features your plan includes.
The hybrid deployment places the data plane, the gateway, inside your VPC, while the control plane stays in Portkey's VPC. Requests and responses flow through the gateway you run, but the gateway calls out over HTTPS to control.portkey.ai for configuration and control functions. The split is documented in Portkey's hybrid deployment docs, and those docs are the place to confirm exactly which data crosses the boundary.
- Outbound connectivity: The gateway needs HTTPS egress to control.portkey.ai and to each model provider. Firewall rules and proxy settings must allow both.
- What you operate: The gateway runtime and its host or cluster, including patching, scaling and availability.
- What you control: Where the gateway runs, network placement and which provider endpoints it can reach.
- What stays with Portkey: The control plane, which holds configuration and management functions. Ask Portkey which configuration and telemetry items reach it.
- Compliance reviews: The assessor will need a data flow diagram that shows the control plane call and the logs that remain in Portkey.
Do not assume that running the gateway in your VPC removes all external dependencies. Verify the control plane traffic with your security team.
A self-hosted LiteLLM deployment runs the proxy in your own cluster, with PostgreSQL for virtual keys and spend data, and Redis for shared state when more than one proxy instance runs. LiteLLM's production guidance recommends one Uvicorn worker per pod and horizontal scaling, with the master key kept in a secret manager. Once deployed, no part of the gateway needs to contact a vendor-hosted control plane, though the model providers you call still receive requests.
- Outbound connectivity: Only the model provider endpoints you configure, plus any observability or secret manager endpoints you choose. You can restrict egress to an allowlist.
- What you operate: The proxy deployments, PostgreSQL, Redis, upgrades, backups, monitoring and the logging destination.
- What you control: Every config, key, budget, log sink and network path. The LITELLM_MASTER_KEY, which must start with sk-, belongs in your secret manager.
- Who is paid for what: Your cloud or infrastructure costs, your platform team's time, and any LiteLLM Enterprise license for SSO beyond the free user limit.
- Compliance reviews: You produce the evidence yourself, such as configuration baselines and access reviews. Independent auditors issue any attestation.
Confirm the current LiteLLM release and support policy against the releases page before you pin a version.
In a disconnected network, the gateway cannot reach Portkey's control plane or public model APIs. The practical pattern is to run LiteLLM or the open-source Portkey gateway inside the isolated enclave, in front of locally hosted models such as those served by vLLM or Ollama. Container images, Python packages and any licenses must be transferred through an approved process, and updates follow the same path.
- Outbound connectivity: None to the internet. Internal traffic runs only between the gateway, the database, Redis and local model servers.
- What you operate: Everything, including image mirroring, dependency intake, certificate management and offline upgrades.
- What you control: All keys, budgets, logs and model weights, which never leave the enclave.
- Hosted services to replace: Any SSO, observability or secret manager integration that normally calls a vendor endpoint needs an internal equivalent or a documented exception.
- Compliance reviews: Your evidence must show the transfer process for updates and the absence of unexpected egress.
Request from DeployOpen or your team a written scope that lists every endpoint the gateway will need, including any provider endpoints you intend to keep.
What the MIT core and enterprise directory mean for you
LiteLLM's licensing splits the codebase in two. Everything outside the enterprise/ directory is under the MIT license, while content inside that directory carries its own license in enterprise/LICENSE. In practice, the proxy, virtual keys, budgets, routing and logging integrations you deploy from the main repository fall under MIT, but any feature you pull from the enterprise directory follows different terms. The most consequential example for this comparison is SSO. From v1.76.0, the admin UI SSO with Google, Microsoft Entra ID, Okta and generic OIDC is free for up to 5 users, and an Enterprise license is needed beyond that. If your identity model involves more than five administrators or users, the license cost becomes a line item in your LiteLLM budget, not a free feature.
Portkey's AI Gateway is itself MIT licensed, so the routing core can be self-hosted under open terms. The features Portkey's repository lists outside that core, including prompt management, semantic caching, advanced logging and tracing, access control and PII redaction, are governed by Portkey's plan terms and, for hybrid deployments, by the control plane that stays in Portkey's VPC. Before you commit, confirm with Portkey exactly which of the features you need are open source and which are plan-bound, and get that answer in writing. Do not infer it from the gateway repository alone.
Ask counsel to review both license files, LiteLLM's LICENSE and the Portkey gateway license, together with the enterprise terms you would sign. Ask specifically whether a self-hosted deployment of enterprise-directory code is permitted for your entity and its affiliates, how seat or user counts are measured for the SSO threshold, and whether license terms change between versions. Your procurement team should also ask for the support and upgrade commitments attached to each paid tier, since LiteLLM's enterprise documentation describes version support for both open source and Enterprise deployments and those terms determine how long you can stay on a given release.
What running LiteLLM costs compared with a Portkey plan
A Portkey plan bundles the gateway, logs and enterprise features into one invoice, while a self-hosted LiteLLM setup spreads cost across compute, a database, a cache, an observability backend, an optional enterprise license and staff time. The cards below separate those components so you can ask each vendor for the right numbers before you compare totals.
Portkey plan and enterprise fees
With Portkey, you pay a recurring subscription or enterprise contract that covers hosted logs, prompt management, semantic caching and access control. Hybrid deployment adds contract terms for the control plane. Ask Portkey which features sit in your tier, how log volume is metered, and whether hybrid deployment is priced separately. Compare that recurring line with the components below, not with the open-source gateway alone.
LiteLLM compute and gateway pods
The LiteLLM proxy is free under the MIT license for the code outside its enterprise directory, so your main compute cost is the infrastructure that runs it. LiteLLM's production guide suggests 1 vCPU and 4 GiB of memory per worker as a starting point to validate under your own load. Costs rise with request volume, streaming and the number of replicas. Recommend one Uvicorn worker per pod and scale horizontally, which makes replica count your main lever.
PostgreSQL and Redis dependencies
Virtual keys, budgets and spend tracking need PostgreSQL, which LiteLLM names as its primary supported database. Once more than one proxy instance runs, Redis 7.0 or newer holds shared rate limits, router state and cache. Both are recurring costs whether you use a managed service or run them yourself. Confirm the database size, backup schedule and high-availability design with your platform team, because these services often carry more operating weight than the gateway itself.
Observability backend and its stores
LiteLLM logs to Langfuse, Datadog, OpenTelemetry and Prometheus, so the log store is a separate cost in a self-hosted setup. Self-hosted Langfuse depends on PostgreSQL, ClickHouse, Redis or Valkey, and S3 or blob storage, and its production path runs on Kubernetes with Helm. Log volume, retention and prompt payload size drive storage growth. Hosted logs in a Portkey plan hide this line, so ask what retention you need before comparing totals.
LiteLLM Enterprise license
SSO is free for up to 5 users from v1.76.0, and an Enterprise license is needed beyond that. Audit logs, SCIM, JWT authentication, team RBAC and secret manager integrations also sit in the enterprise tier. This is the one recurring line that depends on headcount and audit requirements, so decide early whether you need it. Ask LiteLLM for a quote based on your user count, the features you need and your support terms.
Operating labor and upgrades
Running a production gateway means patching, upgrading across minor versions, rotating the master key from a secret manager, tuning database writes and monitoring spend updates. These tasks recur and are usually the largest hidden cost. LiteLLM maintains support for the four most recent stable minor versions, so plan an upgrade cadence. A fixed-scope implementation or a support retainer can shift part of this work, and you should request its scope in writing.
How a move from Portkey to LiteLLM runs
The sequence takes gateway traffic from Portkey's hosted or hybrid setup to a LiteLLM proxy you run, with keys, budgets, routing, logs and cutover each checked before production traffic moves.
Inventory applications and Portkey configs
The platform team lists every application that calls the Portkey gateway, the Portkey configs and virtual keys each one uses, the providers and models behind them, and any prompt management or semantic caching that depends on Portkey features. Application owners confirm which calls are production, which are batch, and which endpoints they hard-code. The step is done when each application maps to one owner and one set of providers, and the team has a written list of Portkey-only features that LiteLLM will not replace directly.
Map keys, budgets and routing
Each Portkey virtual key and its budget, rate limit and model restriction is translated into a LiteLLM virtual key tied to a user or team, with PostgreSQL holding the key state. Portkey fallback and load balancing settings are mapped to LiteLLM router settings such as priority ordering for failover, retries and cooldowns, chosen after confirming which routing strategy suits each route. The output is a mapping table that a reviewer can check line by line against the original Portkey configs.
Stand up PostgreSQL and Redis
Platform engineers deploy PostgreSQL as the database for virtual keys and spend records, and Redis 7.0 or newer once more than one proxy instance runs, since shared rate limits, router state and cache depend on it. The LiteLLM production guide sets the baseline for compute per worker and for keeping the master key in a secret manager. The step is complete when the proxy runs in staging with multiple instances, rate limits behave the same across them, and the backup and restore of the database has been tested.
Choose the logging destination
Because Portkey's advanced logging and tracing are enterprise features, the team decides where request logs go once Portkey's hosted logs stop being the record. Options include Langfuse, Datadog, OpenTelemetry and Prometheus, which LiteLLM supports, and the decision should name who can read prompts and responses in each store. Security and privacy staff sign off on retention and on any prompt content stored. The step closes when a test request appears in the chosen destination with the expected metadata and nothing sensitive is written where it should not be.
Export existing Portkey history
Historical logs are pulled from Portkey before the account is reduced, using the export method available on the current plan, and stored in the archive that compliance has approved. Teams confirm the export format and any row limits with Portkey before scheduling it, because limits and plan access can change. The step is done when a sample of exported records has been opened, the record count has been reconciled with the Portkey dashboard for the same period, and the archive location and access list are documented.
Run parallel traffic and verify
Selected non-critical applications send a share of requests through LiteLLM while the remaining traffic stays on Portkey, so responses, latency, token counts, spend and error rates can be compared side by side. Reviewers check that fallbacks trigger as designed by forcing a provider failure in staging. The step closes when the comparison shows that cost tracking, budgets and model restrictions behave as the mapping table states, and every difference has an owner and a fix.
Reissue keys and cut over applications
Applications that already call an OpenAI-compatible endpoint change their base URL and receive new LiteLLM keys, moving one application group at a time under a written change window. Key reissue, secret rotation and the revocation date for Portkey keys are defined in writing before the first cutover, along with who approves each move. The step is complete when every group is on LiteLLM, Portkey keys are revoked on schedule, and the change log matches the mapping table.
Define rollback and close the migration
A written rollback plan states the conditions that trigger a return to Portkey, who decides, how the Portkey endpoint and keys are kept available during the rollback window, and how logs are reconciled if traffic moves back. Operations owners then sign off on monitoring, on-call runbooks for the proxy and database, and the upgrade process for LiteLLM releases. The step is done when the rollback test has been run in staging, the runbooks are stored with the platform documentation, and the Portkey subscription can be ended with the data archive confirmed.
Questions buyers ask before choosing a gateway
What does running LiteLLM actually cost in people and infrastructure?
The software is MIT licensed outside its enterprise directory, so the main costs are the infrastructure it runs on and the engineering time to operate it. Infrastructure includes compute for the proxy, a PostgreSQL database for virtual keys and spend, and Redis 7.0 or newer once more than one proxy instance runs. LiteLLM's production guide suggests 1 vCPU and 4 GiB memory per worker as a starting point, so the compute bill scales with replicas and traffic. People costs come from upgrades, database backups, key lifecycle, monitoring and on-call coverage for the gateway, which sits on every LLM call path. Add the observability backend you choose for logs, such as Langfuse or Opik, since it has its own database and storage needs. Ask any provider for a written breakdown of these components against your expected request volume.
How long does a gateway migration take, and what drives that time?
The duration depends mainly on how many applications call Portkey today, how many virtual keys, budgets and routing configs must be mapped, and how much of your logging and dashboards depend on hosted Portkey data. Teams that already send requests to an OpenAI-compatible endpoint can cut over application by application, while teams with custom routing rules, fallbacks and guardrails need more testing per rule. Other drivers include whether you must export historical logs before the hosted account closes, how many environments need parallel running, and the change approval process in your organization. Your provider should size the work from a written inventory of applications, keys, routing rules and log consumers, and fix the scope and timeline in writing before work starts.
What does the work need from our platform team?
Your platform team needs to own the runtime, the secrets and the network path. Expect to provide a Kubernetes cluster or VM environment, a PostgreSQL instance and, for multiple replicas, Redis. LiteLLM recommends running one Uvicorn worker per pod and scaling horizontally, and keeping LITELLM_MASTER_KEY, which must start with sk-, in a secret manager. Your team also decides how provider API keys are stored, how TLS terminates, how autoscaling and health checks are configured, and where logs are shipped. Security and identity teams should confirm which SSO and access model applies to the admin UI. Ask whether the platform team needs to staff a named owner for the gateway, and what handover documentation covers.
Can our own team run a LiteLLM gateway without outside help?
Many teams with container and database experience can run a gateway in-house, especially when the scope is a few applications and standard providers. The harder parts are production configuration, which includes database connection handling, the 60-second batching of spend updates at scale, rate limit behavior across replicas, router cooldowns and fallback tuning, and upgrades. Recent LiteLLM releases, such as v1.105.1 published October 14, 2026, change behavior over time, and the project supports the four most recent stable minor versions, so upgrades become recurring work. Decide whether you want a one-time build, a documented runbook for your own staff, or an ongoing support arrangement. DeployOpen plans, implements, hardens, backs up, upgrades and documents self-hosted platforms in your environment, and most engagements are fixed-scope with a written scope before work starts.
Do we need LiteLLM Enterprise for SSO and audit logs?
It depends on how many people use the admin UI. According to LiteLLM's documentation, SSO with Google, Microsoft Entra ID, Okta and generic OIDC is free for up to 5 users from v1.76.0, and an Enterprise license is needed beyond that. Audit logs, SCIM provisioning, JWT authentication with identity provider tokens, organization and team RBAC, IP allowlisting and secret manager integrations are listed under LiteLLM Enterprise. Confirm the current license terms and the user count in writing with LiteLLM before you commit, because the free tier and feature boundaries can change between releases. If audit trails are a control requirement, ask whether an alternative such as shipping proxy logs to your own SIEM meets the same evidence needs.
Does any traffic or log data leave our network with self-hosted LiteLLM?
A self-hosted LiteLLM proxy only sends traffic where you configure it to go. Requests go from your applications to the proxy, then to the LLM providers you have configured, such as OpenAI, Anthropic, Gemini, Bedrock or Azure. Logs go to the destinations you set, for example Langfuse, Datadog, OpenTelemetry or Prometheus, and virtual keys, budgets and spend records stay in your PostgreSQL database. Hybrid Portkey, by contrast, keeps its control plane at control.portkey.ai, so the gateway calls out over HTTPS. To verify the claim, your security team should review egress rules, confirm which provider endpoints are reachable, and test the logging configuration in a non-production environment. If you need an air-gapped network, provider endpoints must be replaced with local models.
What should we ask a provider before signing a gateway scope?
Ask for the written scope to name the applications, providers, virtual keys, routing rules, guardrails and log destinations included, along with the environments to be built. Ask which components are priced or delivered separately, such as the database, observability backend, enterprise license and any ongoing support. Ask how cutover and rollback will be tested, how keys will be reissued without downtime, and what handover documentation and runbooks you receive. Also ask which controls and evidence are prepared for your compliance reviews, and confirm that DeployOpen prepares systems and evidence but does not issue audit reports or certifications, which come from independent auditors. Quotes follow from the written scope, so a clear scope makes comparisons meaningful.
How DeployOpen helps evaluate, implement or migrate
DeployOpen is a Chainscore Labs company that plans, implements, hardens, backs up and documents self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a decision between Portkey and LiteLLM, the usual first step is a fixed-scope evaluation: we map your current Portkey usage (virtual keys, routing configs, fallbacks, guardrails, log destinations and whether you run hybrid mode) against what a self-hosted LiteLLM proxy would need, then agree the scope, deliverables and timeline in writing before any work starts. The quote follows from that written scope, not the other way round.
If you proceed, implementation covers the parts this page has flagged as work: a production deployment backed by PostgreSQL for virtual keys and spend tracking, Redis for shared rate limits and router state once more than one proxy instance runs, secret handling for the master key, the chosen log destination, and SSO if you need it. Where an enterprise license is required for SSO, audit logs or team RBAC, we confirm that requirement with you and with LiteLLM before the design is fixed. Migration of applications from Portkey's endpoint to the new gateway can be scoped as a separate phase. Ongoing upgrades, monitoring and continuous testing are available as an optional retainer, scoped and priced separately.
DeployOpen prepares systems, documentation and evidence for compliance assessments, but we do not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those functions. If you need evidence for a review of where prompts, completions and logs are stored, we will document the architecture and controls so your assessor can test them, and we will tell you plainly where a control depends on your own environment or on a vendor's license terms.
