Why a hybrid Portkey gateway may not meet your residency or egress rules
A hybrid Portkey deployment still needs outbound HTTPS from the gateway to control.portkey.ai, so a fully self-hosted gateway is the option to compare when that connection is not acceptable. This matters most when a renewal, a data residency review or an egress review asks why a gateway inside your VPC still calls an external control plane. Portkey's hybrid model, described in its hybrid deployment documentation, keeps the data plane in your VPC and the control plane in Portkey's VPC, so configuration and the control plane sit outside your network boundary even though requests stay local.
The open-source candidates differ in what they keep inside your perimeter. The MIT-licensed Portkey AI Gateway can run on your own Docker, Node.js or Kubernetes hosts without a control plane call, although enterprise features such as prompt management, semantic caching, advanced logging and PII redaction sit outside the open-source gateway. LiteLLM runs as a self-hosted proxy with PostgreSQL for keys and budgets, and logs can go to Langfuse or Opik, which can also be self-hosted. Confirm with each vendor, in writing, which features make outbound calls by default and which need a license key, because some enterprise tiers add telemetry or license checks.
Before you shortlist, have your security team list every outbound destination the current gateway uses, including license validation, configuration sync and log upload, and decide whether any of them is allowed in your regulated environment. Then check where prompts, responses and metadata are stored at each step. If the answer is that nothing may leave the network, the choice narrows to fully self-hosted components, and the work shifts to who patches, backs up and monitors the gateway, its database and its log store.
What a fully self-hosted gateway must satisfy
A hybrid gateway still depends on a vendor control plane, so the constraints below describe what a gateway you run entirely yourself has to satisfy. Each one is a question to put to any candidate, whether it is Portkey's open-source gateway, LiteLLM or another option. Answer them before you compare features or prices.
No required outbound control plane calls
Portkey's hybrid deployment places the gateway in your VPC, but the gateway calls out over HTTPS to control.portkey.ai. Your security team should confirm whether the chosen gateway can start, route and serve requests with no external call at all. Test it by blocking egress from the gateway subnet and checking which requests still succeed. This check decides whether the design fits disconnected or tightly restricted networks.
Where logs and configuration live
In a hybrid setup, configuration and logging sit in Portkey's control plane, so you need to know exactly what leaves your network. A self-hosted gateway keeps routing config, virtual keys and request logs in stores you operate. Ask which fields are written, where, and who can read them. Map each store to your data classification policy before go-live.
Keys and budgets kept inside
Virtual keys, per-team budgets and rate limits are only as trustworthy as the store that holds them. LiteLLM virtual keys need a PostgreSQL database, and a multi-instance proxy also needs Redis 7.0 or newer for shared rate limits and router state. Confirm where provider credentials are stored, how the master key is protected, and how budget counters behave if the database or cache is unavailable.
Patching and upgrades owned by you
With a hybrid model, the vendor controls part of the upgrade path. A fully self-hosted gateway moves patching, version upgrades and dependency updates to your team or your implementation partner. LiteLLM supports the four most recent stable minor versions for open source and Enterprise deployments, so plan upgrade windows around that. Write down who approves a release, who tests it and who rolls it back.
Enterprise features and their licenses
Some controls sit behind enterprise licenses even in open-source projects. Portkey's repository lists prompt management, semantic caching, advanced logging and tracing, access control and PII redaction as enterprise features outside the open-source gateway. LiteLLM SSO beyond the free user limit and audit logs also need an Enterprise license. List each control your review requires and confirm whether it is open source or licensed before you commit.
Which Portkey deployment model fits which constraint
This table compares Portkey hybrid, Portkey Cloud, the MIT-licensed Portkey Gateway run by you, and self-hosted LiteLLM across outbound connectivity, log and configuration location, and enterprise features. Read each row against your network rules and audit scope, then check the trade-off column before shortlisting. The main takeaway is that only the self-run options keep the control plane inside your boundary, and every option that does needs you to confirm which features your license covers.
| Constraint | Portkey hybrid | Portkey Cloud | Portkey Gateway (self-run) | Self-hosted LiteLLM |
|---|---|---|---|---|
| Outbound connectivity | Gateway in your VPC calls out over HTTPS to control.portkey.ai | Application traffic routes through Portkey's hosted API | No required call to Portkey; you confirm any telemetry settings for your build | No required call to a vendor control plane; provider calls go out from your network |
| Where logs live | Logs and analytics sit in Portkey's control plane | Logs sit in Portkey's hosted platform | Local gateway console logs; you choose any external log destination | You choose the store, such as Langfuse, OpenTelemetry or Prometheus, and host it |
| Where configuration lives | Configs and keys managed in Portkey's control plane | Configs and keys managed in Portkey's hosted platform | JSON configs attached to client requests, kept in your repositories | Config file plus PostgreSQL for virtual keys and budgets, in your infrastructure |
| Prompt management | Enterprise feature, confirm entitlement with Portkey | Included in the hosted product tier you buy, confirm plan | Outside the open-source gateway; replace with your own store | Not built into the proxy; pair with a prompt tool such as Langfuse |
| Semantic caching | Enterprise feature, confirm entitlement with Portkey | Confirm per plan | Outside the open-source gateway; simple caching only | Exact-match and semantic caches on Redis, Valkey or Qdrant |
| Access control and PII redaction | Enterprise features, confirm entitlement with Portkey | Confirm per plan | Outside the open-source gateway; add your own checks | Virtual keys and team budgets included; SSO and audit logs need LiteLLM Enterprise |
| Who patches what | Portkey patches the control plane; you patch the gateway in your VPC | Portkey operates the whole service | You patch and upgrade the gateway and its hosting | You patch the proxy, PostgreSQL, Redis and the log backend |
| Main trade-off | Keeps the gateway local but still depends on a Portkey control plane | Least operating work, but traffic and logs leave your network | Most control, but enterprise features and support must be built or bought | Broad provider coverage and self-run keys, with operating work and possible enterprise license for SSO |
Where traffic, logs and configuration actually go
In Portkey's hybrid deployment, the data plane (the gateway) runs in your VPC while the control plane stays in Portkey's VPC, and the gateway calls out over HTTPS to control.portkey.ai. Your application traffic to model providers goes through your gateway, but the gateway still depends on that outbound channel to Portkey for its control functions. Before you approve the design, ask Portkey in writing which request metadata, token counts, prompt or completion content, API keys, virtual key references, routing configs and error payloads are sent to control.portkey.ai, which of these are stored there, for how long, and in which region. The answers decide whether the deployment fits a data residency or egress policy, and they belong in your data flow diagram and vendor risk file. The Portkey hybrid deployment docs are the starting point for that conversation.
A self-run gateway changes where those answers come from. If you run the MIT-licensed gateway yourself, using the Docker image, Node.js or Kubernetes options in Portkey's gateway repository, there is no mandatory call to Portkey's control plane, so you can verify egress with network policies and firewall logs. Logs then go to a destination you operate. For example, teams often send traces to Langfuse or to an OpenTelemetry collector inside their own network (illustrative). Confirm this in your own environment with a packet capture or egress test, because the gateway's configuration and any provider telemetry settings must be checked individually.
Do not assume the local Gateway Console gives you a logging solution. It shows local logs for development and testing only. Advanced logging and tracing are listed by Portkey as enterprise features outside the open-source gateway, so production log retention, export and dashboards must be designed separately. Plan a log pipeline with retention, access control and deletion rules that match your data classification, and confirm which log fields carry prompts or personal data before they leave the gateway.
How to deploy a fully self-hosted gateway under the constraint
This sequence moves LLM gateway traffic into your own infrastructure, with no outbound call to a vendor control plane, and ends with applications calling a gateway whose keys, budgets and logs you hold.
Choose the gateway and confirm its release status
The platform team and the security lead agree which gateway fits the routing and access model. The options are the MIT-licensed Portkey AI Gateway, deployed with Docker or on Kubernetes, or LiteLLM, which adds virtual keys and budgets backed by PostgreSQL. Portkey's enterprise deployment guide lists features such as secure key management, caching and PII redaction, so check which ones you need and whether they sit in the open-source gateway or an enterprise tier. Before pinning any version, confirm the current release and whether the branch you plan to use is stable, since Portkey's 2.0 work is a pre-release. The step is done when the chosen gateway, its version and its license are recorded in your architecture decision log.
Provision PostgreSQL and Redis inside your network
The infrastructure team provisions the state stores in the same network as the gateway, with no route to a vendor endpoint. For LiteLLM, PostgreSQL is the primary supported database and holds virtual keys and spend data. Once more than one proxy instance runs, Redis 7.0 or newer holds shared rate limits, router state and cache. Restrict database access to the gateway subnet and enable encryption at rest and in transit under your standard policy. The step is done when the gateway can connect to both stores from a test pod or container and a failover test shows the shared state is visible to a second instance.
Store the master key in a secret manager
The security team creates the admin credential, which for LiteLLM is the LITELLM_MASTER_KEY value and must start with sk-. The key goes into your existing secret manager, such as Vault or a cloud provider's secret service, and is injected at runtime rather than written into a Helm values file or environment file in source control. Limit read access to the deployment service account and rotate the key on your normal schedule. The step is done when the secret resolves in the running pod and a search of your repositories and CI logs finds no copy of the value.
Deploy one worker per pod and scale horizontally
The platform team deploys the gateway as a Kubernetes Deployment or Docker service with one Uvicorn worker per pod, following LiteLLM's production guidance, and scales by adding replicas. Each worker is sized against the vendor's documented baseline of 1 vCPU and 4 GiB memory, then adjusted from load tests on your own traffic. Place the replicas behind an internal load balancer that is not exposed to the internet. The step is done when a load test shows requests spread across replicas, rate limits hold across pods, and a pod restart does not drop the service.
Route logs to Langfuse or OpenTelemetry
The observability owner chooses where traces and spend records go. Options include a self-hosted Langfuse instance, which needs its own PostgreSQL, ClickHouse, Redis or Valkey and object storage, or an OpenTelemetry collector feeding your existing backend. Hosted logging from Portkey is not used, so no prompt or response leaves your network for a vendor console. Confirm the redaction rules for prompts and responses before enabling full logging, because the gateway writes whatever it is configured to send. The step is done when a test request appears in the chosen backend with the expected model, key and cost fields, and the team confirms no log is sent to an external endpoint.
Configure routing, fallbacks and access controls
The platform team translates existing routing rules into the gateway's configuration. For LiteLLM this covers the router strategy, retries, cooldowns and fallbacks, along with per-key, per-user and per-team budgets and model restrictions. For Portkey's gateway, rules are JSON configs attached to the client. Check each control against the edition you deployed, because SSO, SCIM, audit logs and some team controls may sit behind an enterprise license. The step is done when a test key is blocked from a restricted model, a forced provider error triggers the expected fallback, and the configuration is stored in version control.
Cut over applications to the new endpoint
Application owners change the base URL and API key in one service at a time, starting with a low-risk workload. Applications that already call an OpenAI-compatible endpoint need only the URL and key change. Keep the Portkey route active during the window so you can roll back by reverting configuration. Watch error rates, latency and spend during the first full business cycle. The step is done when every application points at the self-hosted gateway, the Portkey API key has been revoked, and the team has signed off the cutover checklist.
Who patches and upgrades what in each model
Each component of a gateway deployment has an owner, and that owner decides who patches it, who upgrades it and who carries the risk when it breaks. The split below shows where Portkey hybrid deployment hands work to the vendor and where a fully self-hosted gateway leaves every component with your team.
Gateway binary or image
In Portkey's hybrid model, your VPC runs the data plane, so your team deploys, pins and upgrades the gateway container while Portkey operates the control plane it calls. In a fully self-hosted setup, you own the gateway image, its base OS layers and its dependency updates end to end. Check who issues security fixes for the gateway version you run, and whether your change window can absorb a new minor release. The gateway version you pin is your responsibility in both models.
Database and Redis
A self-hosted LiteLLM proxy needs PostgreSQL for virtual keys and budgets, and Redis 7.0 or newer once more than one proxy instance runs, for shared rate limits, router state and cache. Under hybrid Portkey, the stores behind its control plane are Portkey's to run, while the gateway itself holds only what your deployment needs. Under self-hosting, backup, failover, major-version upgrades and capacity for both stores fall to your database and platform team. Ask who restores data after an outage and how long it takes.
Provider credentials
Provider API keys for OpenAI, Anthropic, Bedrock, Azure and others are the most sensitive secrets in the stack. In hybrid mode you still store them in Portkey's Model Catalog or in your own VPC, so confirm where each key is held and who can read it. In a self-hosted gateway, your secret manager holds them and your team rotates them, and LiteLLM recommends keeping the master key in a secret manager. Document rotation ownership for every provider account.
Enterprise license decisions
Some controls sit behind paid licenses in both ecosystems. LiteLLM's SSO is free for up to 5 users from v1.76.0, and SCIM, audit logs, IP allowlisting and secret manager integrations require LiteLLM Enterprise. Portkey's enterprise features, including prompt management, semantic caching and PII redaction, sit outside its open-source gateway. Decide which controls you need before choosing the model, and ask any vendor in writing which license covers each one and for how long.
Logs backend
Hybrid Portkey sends gateway calls to its control plane over HTTPS, so logs and traces leave your VPC unless you configure otherwise, and retention follows Portkey's terms. A self-hosted gateway writes logs to a destination you choose, such as Langfuse, Prometheus or OpenTelemetry, and you then operate its storage, retention and access controls. Confirm which fields reach each system, including prompts and responses, and who can query them.
Control plane and upgrade cadence
Under hybrid Portkey, Portkey runs the control plane and its upgrades, while your team still plans gateway upgrades on the data plane. Self-hosting moves the control plane, configuration and routing rules onto your team. LiteLLM supports the four most recent stable minor versions for open source and Enterprise, and its releases page listed v1.105.1 on October 14, 2026, so a self-hosted gateway needs a standing schedule to stay inside that window. Ask Portkey for its own support and patch commitments in writing.
Questions buyers ask before choosing a self-hosted gateway
Does Portkey hybrid deployment send prompts or only metadata to the control plane?
The supplied documentation describes the gateway in your VPC calling out over HTTPS to control.portkey.ai, but it does not settle in our sources exactly which request fields leave your network in every configuration. Confirm this in writing with Portkey before approval, and ask specifically whether prompt bodies, completions, headers, API keys, user identifiers and log payloads are sent, and which of them can be disabled. Ask for the data flow in their hybrid deployment documentation and for a network diagram you can test against your own egress rules.
What does a self-hosted gateway need from my team to run?
A production gateway is a set of components your team operates, not a single binary. For LiteLLM, the supplied production guidance names PostgreSQL as the primary database, Redis 7.0 or newer once more than one proxy instance runs, and a secret manager for the master key. Add your container platform, a log and trace backend, TLS termination, monitoring and an on-call path. Your team needs people who can run Kubernetes or Docker, manage databases, rotate keys and respond to provider outages.
Does SSO require an enterprise license on LiteLLM?
Not for small teams. From v1.76.0, the LiteLLM admin UI supports SSO with Google, Microsoft Entra ID, Okta and generic OIDC providers for up to 5 users at no charge, and an Enterprise license is needed beyond that. Enterprise also adds SCIM, JWT authentication, audit logs, organization and team RBAC and IP allowlisting. Confirm the current terms in the LiteLLM license and enterprise documentation, because the user limit and feature split can change between releases.
What drives the effort and cost of a self-hosted gateway up or down?
The main drivers are the number of gateway instances and whether you need high availability, the log and trace backend you choose, retention and storage volume for prompts and responses, the number of providers and models, and how often you upgrade. Costs fall into one-time work (design, migration, hardening, documentation) and recurring work (compute, databases, storage, licenses if needed, patching and on-call). Ask any quote to split these components so you can see what scales with traffic and what scales with headcount.
How long does a migration from Portkey to a self-hosted gateway take?
The duration depends on scope, not on a fixed figure. The variables that matter are how many applications call the gateway, how many virtual keys, budgets, routing configs and fallbacks must be mapped, whether historical logs must be exported, how many environments need cutover, and how many compliance reviews must accept the new design. A narrow scope with one or two applications is much shorter than a multi-team cutover with log migration. Ask for a timeline in the written scope and for the milestones it depends on.
Can our team run this alone, or do we need a fixed-scope engagement?
Teams with experienced platform, database and security engineers can run a self-hosted gateway in-house, and the open-source projects publish deployment and production guidance. An external engagement makes sense when the deadline is fixed, the compliance review is strict, or the team lacks capacity for design, hardening and cutover at the same time. DeployOpen plans, implements, hardens, backs up, upgrades and documents self-hosted platforms in your own infrastructure, under fixed-scope work agreed in writing, with ongoing support available as a retainer. It does not issue audit reports or certifications.
Which compliance reviews does a self-hosted gateway help with?
A self-hosted gateway keeps prompts, keys, budgets and logs inside your environment, which helps with data residency and third-party data flow questions. It does not make your organization compliant on its own. Reviews such as SOC 2, ISO/IEC 27001 or HITRUST examine your controls, access management, change management, logging and vendor management. DeployOpen prepares systems, controls and evidence for these assessments, but independent auditors and certification bodies issue the reports. Ask each reviewer which controls they will test before you finalize the design.
Which compliance reviews each model passes
DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make those determinations. Before choosing between Portkey's hybrid model and a fully self-hosted gateway, ask your auditor which framework applies and which evidence they expect to sample. Then confirm framework fit with Portkey for the hybrid option and with your certification body for any claim you intend to make about the self-hosted stack.
Most security and compliance reviews of an LLM gateway ask the same questions: a data flow diagram showing every hop a prompt and response takes, a list of subprocessors that touch that data, the egress rules on the gateway's network, how provider and virtual keys are created, rotated and stored, and which access logs show who changed routing or budgets. In the hybrid model, the gateway runs in your VPC, but the gateway calls out over HTTPS to control.portkey.ai, so the data flow diagram must show that control plane and Portkey appears as a subprocessor for configuration and, depending on your settings, logs. Confirm with Portkey exactly which payloads and metadata leave your network and where they are stored.
In a fully self-hosted design, the data flow stays inside your environment and the subprocessor list shrinks to the components you choose, such as the gateway, PostgreSQL, Redis and your log store. Egress rules can then deny all traffic from the gateway except the model providers you have approved. Keys live in a secret manager that you control, and access logs come from your identity provider and the gateway's own records. The trade-off is evidence you must produce yourself: change records, patch history, backup tests and vulnerability scan results all belong to your team. Ask your assessor which of these artifacts they will request and whether a self-hosted component's upstream license or support status needs to be documented, for example the enterprise license needed for SSO on LiteLLM.
Evidence to collect for a residency or egress review
A residency or egress review asks one question in several forms: what leaves your network, where it lands, and who can change it. The six items below map to controls you own in a hybrid Portkey deployment or a fully self-hosted gateway, so you can show the reviewer exactly where each answer comes from.
Network egress rules and firewall exports
Export the outbound rules for the subnet or namespace where the gateway runs, including DNS and proxy settings. In a hybrid model, the rule you need to explain is the HTTPS path to control.portkey.ai. In a fully self-hosted model, the review should show that the gateway can reach only your model endpoints, your database and your telemetry sink. Confirm any default route to the internet and any egress allowlist in your cloud account, and keep dated copies for the review.
Data flow diagrams
Draw each request path from the calling application through the gateway to every provider, cache, log store and billing or identity service. Mark which hop carries prompts, which carries only metadata, and which crosses a region or account boundary. For hybrid Portkey, show that the control plane sits outside your VPC even though the gateway runs inside it. Reviewers often ask where prompt bodies are stored, so label that hop explicitly.
Configuration and key inventory
List every gateway configuration file, provider credential, virtual key and master key, with its owner, scope and storage location. In a self-hosted LiteLLM setup, the master key should sit in a secret manager, and virtual keys live in PostgreSQL. In hybrid Portkey, record which configs and provider keys are held in Portkey's Model Catalog and which remain in your secret store. Include the last rotation date for each credential.
Log retention settings
Record where request and response logs are written, how long each store keeps them, and who can query or delete them. Hybrid and hosted tiers keep traces in the vendor's control plane, so ask the vendor for the retention period in writing and for the deletion procedure. In a self-hosted stack, the retention setting belongs to your database, your object store or your observability backend, and the review should show the configured value and the job that enforces it.
Upgrade and patch records
Keep a dated log of gateway, proxy, database and cache versions, along with the security advisories reviewed and the change tickets that applied each update. A self-hosted team owns this entire record. For LiteLLM, track the releases you run against the versions it supports under its support policy. For hybrid Portkey, ask which components Portkey patches and which fall to your team, and get the answer in the contract or security documentation.
Third-party services that receive data
Build a register of every external service that touches prompts, responses, metadata or credentials, including model providers, observability vendors, guardrail APIs, identity providers and support tooling. For each entry, record the data categories sent, the region of processing, and the contract or data processing terms that cover it. Hybrid Portkey adds its own control plane to this list, so state exactly which fields reach it. Confirm each entry with the vendor rather than relying on the product marketing pages.
How DeployOpen deploys a self-hosted gateway under your constraint
DeployOpen plans, implements, hardens and documents self-hosted gateways inside your own cloud accounts, data centers or air-gapped networks. For a Portkey hybrid deployment versus a fully self-hosted gateway, we start by mapping which outbound connections, log stores and configuration locations your controls allow, then build only the design that fits.
How we scope
- We record which outbound HTTPS destinations your security policy permits, including whether any control plane call is allowed at all.
- We define where request logs, virtual keys, budgets and routing configuration will live, and who holds administrative access to each.
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before any build work starts.
How we build
- We deploy the chosen gateway into your environment, with its database, cache and log destinations placed inside your network boundary.
- We migrate provider credentials into your secret manager and configure SSO and audit logging only where your license and needs require them.
- We test failover, routing, key revocation and log completeness against the scope before anything is cut over to production traffic.
How we hand over
- You receive architecture diagrams, network flow records, runbooks for patching and upgrades, and the configuration files as they were deployed.
- We document which controls and evidence exist for your compliance reviews, while independent auditors remain responsible for any assessment opinion.
- Knowledge transfer sessions walk your platform team through restarts, key rotation, scaling and log retention procedures.
How we support
- Ongoing support, upgrades and continuous testing of the gateway are available as a retainer after handover.
- Upgrades are planned against the release notes of the chosen gateway, with changes tested in a staging copy first.
- Ask any provider for a written scope, deliverables and support terms, and confirm the license terms of every component before signing.
