OpenRouter vs LiteLLM: when a self-hosted gateway fits better than a hosted router
A self-hosted LiteLLM gateway fits better than OpenRouter when your organization wants to hold its own provider contracts and API keys, enforce spend limits per team, and keep request logs inside its own infrastructure. OpenRouter fits better when a small team needs fast access to many models through one account and has little capacity to operate a service. The decision usually turns on three questions: who holds the accounts with the model providers, how much operating capacity you can staff, and whether the platform fee on OpenRouter credits outweighs the cost of running your own gateway. Whether LiteLLM is the right tool is a separate question from whether LiteLLM Enterprise is worth buying, because the open-source core and the enterprise features carry different licenses and different costs.
OpenRouter is the simpler path when prompts can flow through a third party's account and no one on the team wants to run a database or a proxy. You get one billing relationship, one key and quick trials of new models, and you avoid operating PostgreSQL, Redis and the application servers that a production LiteLLM deployment needs. The trade-off is that provider usage, budgets and logs sit in OpenRouter's account, so audits of data flows, per-team chargeback and retention rules depend on what OpenRouter exposes. Before you commit, confirm with OpenRouter which of your data classes it may process and what its account and billing data retention looks like for your region.
LiteLLM fits when you need the gateway to sit in your own cloud account. Its virtual keys support per-key, per-user and per-team budgets, TPM and RPM limits and model restrictions, which lets finance and platform teams set limits without issuing provider keys to every application. Logs can flow to your own Langfuse, OpenTelemetry or Prometheus stack. Plan for a PostgreSQL database for virtual keys and Redis once you run more than one proxy instance. Use the LiteLLM enterprise documentation to confirm which SSO, audit log and RBAC features need a license, since SSO is free only up to a small user limit in current versions.
Which option fits which scenario
This table matches common situations to the option that usually fits better, with the reason stated as a condition. Read each row against your own constraints, because the answer shifts when a requirement such as key custody or residency is fixed by policy rather than preference. The main takeaway is that OpenRouter suits teams that want to outsource provider contracts and operations, while LiteLLM suits teams that need control and can staff the service.
| Scenario | Better fit | Reason |
|---|---|---|
| Small team with no platform staff | OpenRouter | Running LiteLLM means owning PostgreSQL, Redis, upgrades and on-call. If nobody can carry that work, the hosted fee may cost less in effort than self-hosting. |
| Regulated team that must hold provider keys | LiteLLM | Direct provider accounts and keys stay under your control when LiteLLM is the only route to providers. Confirm your provider agreements allow direct billing and key use. |
| Many teams needing per-team budgets | LiteLLM | Per-key, per-user and per-team budgets, TPM and RPM limits are part of LiteLLM's virtual keys, which need PostgreSQL. Confirm the exact limits your rollout needs in its docs. |
| Multi-region data residency | Depends on deployment | LiteLLM runs wherever you deploy it, so routing can stay in region. OpenRouter's in-region options depend on its plan, so confirm current terms before relying on them. |
| Team already running Kubernetes | LiteLLM | LiteLLM is a self-hosted server that scales horizontally, and its production guide recommends one Uvicorn worker per pod. Your existing cluster and secret manager can host it. |
| Need for SSO and audit logs | LiteLLM with Enterprise | SSO is free for up to 5 users from v1.76.0, and audit logs, SCIM and team RBAC need an Enterprise license. Confirm license terms for your user count before choosing. |
| Need to avoid operating a database and cache | OpenRouter | LiteLLM at more than one proxy instance requires Redis 7.0 or newer and PostgreSQL, so those become services you must back up and monitor. |
Who holds the keys, the data and the provider contracts
With OpenRouter, you buy credits from OpenRouter, which then pays the upstream providers and charges a platform fee. Your commercial relationship, usage terms and billing records sit with OpenRouter, and your traffic is routed through its service. With a self-hosted LiteLLM gateway, your organization can open direct accounts with each provider it uses, so the provider contracts, data processing terms and invoices are in your name, and LiteLLM holds only the keys you give it.
- Direct accounts: Each provider key is issued to your organization, so you negotiate terms, spend limits and retention settings directly.
- Virtual keys: Applications receive LiteLLM virtual keys with budgets and model restrictions, so provider keys never need to be shared with application teams.
- Key storage: The
LITELLM_MASTER_KEYand provider credentials belong in a secret manager, not in configuration files or environment variables committed to source control. - Fee exposure: OpenRouter's platform fee is a recurring charge on top of model usage, while a self-hosted gateway shifts cost to infrastructure and staff.
- Offboarding: Keys and contracts you own can be rotated or moved without asking a router vendor to change anything.
Confirm with each provider which account type, data terms and key rotation options apply to your contract, and ask OpenRouter for its current terms on how it handles and retains request data.
Prompts, completions, token counts and request metadata pass through the gateway, so the location of its logs is a data handling decision. With OpenRouter, request data is processed on its hosted service and in the upstream provider regions it selects, and you depend on its published retention and region controls. With LiteLLM, the proxy writes spend and key data to PostgreSQL in your environment and can send request logs to tools you operate, such as Langfuse, OpenTelemetry or Prometheus.
- Database location: PostgreSQL holds virtual keys, budgets and spend records, and it should sit in the same region and account as the gateway.
- Log destinations: LiteLLM can forward logs to Langfuse, Datadog, OpenTelemetry and Prometheus, so decide which of those receive prompt content and which receive only metadata.
- Retention: You set how long request and spend records persist, and you apply deletion to backups as well as live tables.
- Redaction: Prompt and output redaction are not covered by the open-source proxy alone, so plan guardrails or masking before logs leave the gateway.
- Cache and state: Redis holds shared rate limit and router state once more than one proxy instance runs, so it needs the same controls as the database.
Check which fields your logging configuration sends to each destination, and confirm the storage region of every system that receives them.
Identity decides who can open the admin UI, who can create keys and who can see spend. OpenRouter accounts are managed in its own service, which means your staff sign in to a vendor tenant unless you manage that relationship separately. LiteLLM's admin UI supports single sign-on with Google, Microsoft Entra ID (Azure AD), Okta and generic OIDC providers, so access follows the directory you already control.
- Supported providers: Google, Microsoft Entra ID, Okta and generic OIDC providers are documented for the admin UI.
- Free tier limit: From v1.76.0, SSO is free for up to 5 users, and an Enterprise license is needed beyond that.
- Enterprise identity: LiteLLM Enterprise adds SCIM provisioning and JWT authentication with identity provider tokens, which matter when user lists change often.
- Role separation: Organization and team RBAC in Enterprise lets you separate platform administrators from application owners.
- Network restriction: IP allowlisting, available in Enterprise, limits where the admin interface can be reached from.
Confirm the number of users who need admin access, since that determines whether the free SSO tier is enough, and confirm which identity features your current license includes.
Secret custody covers the master key, provider API keys, database credentials and the encryption material your deployment uses. LiteLLM documents that its master key must begin with sk- and should be kept in a secret manager. LiteLLM Enterprise adds direct integrations with AWS, Azure, Google, HashiCorp Vault and CyberArk, so provider credentials can be read from the vault your security team already audits rather than from deployment files.
- Master key:
LITELLM_MASTER_KEYshould be generated, stored and rotated through a secret manager, with access limited to the platform team. - Provider credentials: Each provider key should be scoped to the least spend and model access it needs, and rotated on a schedule your security policy sets.
- Vault integration: Enterprise secret manager integrations can avoid placing credentials in plain configuration, which reduces exposure if a container image or manifest leaks.
- Kubernetes secrets: If you use Kubernetes without an external vault, confirm encryption at rest for Secret objects in your cluster.
- Revocation path: Test how quickly you can revoke a virtual key and a provider key during an incident, since the OpenRouter key path runs through the vendor.
Confirm which secret manager your organization standardizes on, and check whether that integration is in the open-source release or the Enterprise license.
No gateway makes an environment compliant by itself. Regulators and certification bodies look for evidence that your controls operate: who can issue keys, how access is reviewed, how logs are retained and how changes are approved. With OpenRouter, you can request the vendor's own documentation, but you still need evidence for your side of the boundary. With a self-hosted LiteLLM gateway, your team produces most of the evidence, including configuration baselines, access reviews and the records of key issuance. Independent auditors assess these controls, and DeployOpen prepares systems, controls and evidence for those assessments without issuing any audit report or certification.
- Access reviews: Export admin and virtual key lists on a schedule and record who approved each change.
- Audit logs: Audit logs are part of LiteLLM Enterprise, so confirm whether your compliance scope requires them before you choose the license tier.
- Change records: Keep configuration in version control and link each release to a reviewed change.
- Provider evidence: Collect each provider's data processing terms and subprocessor lists from your own accounts.
- Backup and restore: Show that PostgreSQL backups and key restoration work, because auditors ask for tested recovery, not only a backup schedule.
Ask your auditor which control evidence they expect for AI gateway traffic and confirm the audit log features your license provides before scoping the assessment.
What runs in your infrastructure when you self-host LiteLLM
When you self-host LiteLLM, the LiteLLM proxy is the service your applications call. It exposes one OpenAI-compatible endpoint in front of more than 100 providers, and the proxy itself is stateless: it holds no conversation data of its own. Its state lives in PostgreSQL, which LiteLLM names as the primary supported database. PostgreSQL stores what virtual keys need to work, including per-key, per-user and per-team budgets, rate limits and model restrictions, along with spend records. Plan for PostgreSQL backups, high availability and retention from the start, because a lost spend ledger is a finance and audit problem, not only an operations one. On Kubernetes, LiteLLM recommends one Uvicorn worker per pod and horizontal scaling, so capacity grows by adding pods rather than by adding workers to a pod.
Once more than one proxy instance runs, you also need Redis 7.0 or newer. Redis holds the shared state that instances must agree on: rate limits, router state and cache. Without it, each pod enforces limits on its own view of traffic, so a team's RPM or TPM cap can be exceeded across the fleet. The production guide also suggests keeping LITELLM_MASTER_KEY, which must start with sk-, in a secret manager rather than in manifests or environment files committed to source control. At high volume, LiteLLM recommends batching spend updates to the database every 60 seconds, so spend figures are accurate to within that window. Confirm with your own finance team whether a one-minute lag is acceptable for budget alerts.
The production guide suggests 1 vCPU and 4 GiB memory per worker as a starting point. Treat that as a baseline, not a capacity plan: your request sizes, streaming share, logging callbacks and the number of providers and fallbacks all change the load, so run a load test with your own traffic profile before you set resource requests. Ask your platform team to budget for the components you will operate: the proxy pods, the PostgreSQL service with backups, the Redis service, the secret manager and the observability stack that receives logs. Those are the recurring infrastructure and staffing costs that sit alongside provider invoices, and they are the figures to compare against an OpenRouter platform fee in the cost analysis.
Where LiteLLM needs more work than a hosted router
A hosted router handles uptime, provider onboarding and feature access for you. Self-hosting LiteLLM moves those duties into your team, and some of the features your security or audit team expects sit behind an enterprise license. The cards below cover the work to plan before you cut traffic over.
Upgrading on a fixed cadence
LiteLLM supports the four most recent stable minor versions for both open source and Enterprise deployments, so an older minor release eventually falls out of support. Your team owns the upgrade schedule, testing against your model groups, keys and callbacks, and rollback. The latest release listed on the GitHub releases page at the time of writing is v1.105.1, published October 14, 2026. Check that page before you choose a version to deploy, and plan a recurring window for upgrades.
Enterprise features behind a license
SSO is free for up to 5 users from v1.76.0, and beyond that you need an Enterprise license. SCIM, JWT authentication with identity provider tokens, audit logs, organization and team RBAC, and IP allowlisting are also part of Enterprise. If your security review requires any of these, price the license into the total before comparing against OpenRouter's fees, and ask the LiteLLM sales team to confirm what is included in writing.
Separate license for the enterprise directory
LiteLLM is MIT licensed except for content under its enterprise directory, which carries its own license. Your legal team should read that license file before you fork, patch or redistribute the gateway, especially if you plan to embed enterprise-directory code in internal tooling. Confirm which features you depend on live in that directory, so you know which parts of the code are governed by the separate terms.
Managing provider accounts and keys
With OpenRouter, one account holds the provider relationships and billing. With LiteLLM, your organization opens and manages direct accounts with each provider you use, stores their API keys in a secret manager, and handles rotation, spending limits and invoices for each one. Assign an owner for every provider account, and document who can issue keys to production gateways.
Tracking model availability per provider
OpenRouter lists the models it routes and handles provider changes on its side. Behind LiteLLM, your team must watch each provider for model retirements, regional availability changes and renamed model IDs, then update model groups and fallback order. Build a regular check against each provider's documentation and test fallbacks after every change so a deprecation does not surface as a production outage.
What a move from OpenRouter to a self-hosted LiteLLM gateway involves
The sequence moves model traffic from a hosted router to a gateway you run, with provider accounts, keys and budgets under your control and a rollback path kept open until the switch is proven.
Inventory models, providers and traffic patterns
Your platform or engineering lead lists every model slug the applications call through OpenRouter, the applications behind each one, the monthly request and token volume per model, and any provider preferences or fallbacks currently set in OpenRouter. The output is a spreadsheet that maps each application to its models and tells you which providers must be contracted directly. Done when every calling service appears in the list and the security team has confirmed which data classes each one sends.
Open direct accounts with the providers in use
Procurement, legal and the finance owner open billing accounts directly with each provider you need, such as OpenAI, Anthropic, Google or the cloud provider hosting Bedrock or Azure models. Ask each provider to confirm data retention terms, regional endpoints and rate limits for your account tier, since these differ from what you see through OpenRouter. Done when each provider account is active, its API keys sit in your secret manager, and the contract terms are filed with the owner.
Stand up LiteLLM with PostgreSQL and Redis in staging
Your platform team deploys the open-source LiteLLM proxy in a staging environment in your own infrastructure, with PostgreSQL as the primary database and Redis 7.0 or newer for shared state once more than one proxy instance runs. Keep the master key in a secret manager and follow the production guidance for worker and pod settings before any live traffic arrives. Done when a test request succeeds through the proxy, the database records spend, and a restart does not lose virtual keys or budgets.
Map OpenRouter model names to LiteLLM model groups
An engineer translates each OpenRouter model identifier into a LiteLLM model group, listing the direct provider deployments that serve it and the order in which they should be tried. Check that each group returns responses in the request format your applications already use, and record any model that has no direct equivalent so the application owner can decide on a substitute. Done when every model in the inventory has a group, and a test suite runs against each group with expected outputs recorded.
Issue virtual keys with budgets and model restrictions
The platform team creates a virtual key for each application or team, setting per-key or per-team budgets, request and token limits, and restrictions to the model groups that team is approved to use. Store the keys in the applications' secret stores, not in source code. Done when each key has a documented owner, budget and allowed-model list, and a test call with a restricted key to an unapproved model is rejected.
Recreate fallbacks, routing and retries
The engineer rebuilds the fallback order, retry behavior and routing strategy that OpenRouter currently handles, choosing among LiteLLM's routing options and setting cooldowns after repeated failures. Test failover by disabling a provider deployment in staging and confirming that requests move to the next target without the application code changing. Done when a documented test shows each fallback path working and the latency and error results are recorded for comparison.
Run parallel traffic and compare results
For a chosen set of non-critical workloads, the team sends the same requests to OpenRouter and to the LiteLLM gateway, comparing response quality, error rates, latency and cost per request. Logging must be wired to your observability stack at this point, so you can see the gateway's behavior. Done when the owners of each workload sign off that results match acceptable thresholds they defined in advance, and the cost tracking matches the provider invoices.
Switch base URLs with a rollback path, then decommission
Application teams change their base URL and key from OpenRouter to the LiteLLM endpoint one service at a time, keeping the OpenRouter key and configuration available so any service can be pointed back within a short window. After all services have run on the gateway for an agreed period, the team cancels OpenRouter credit purchases, revokes its keys and archives the configuration and logs. Done when no traffic reaches OpenRouter in your logs, the account is closed or revoked, and the runbook records who owns the gateway going forward.
Questions buyers ask before choosing a self-hosted LiteLLM gateway
How much effort does it take to run LiteLLM in-house?
The effort depends on traffic, availability requirements, your team's skills, and how often you upgrade. A single proxy instance for a small internal group is a different job from a multi-instance gateway serving many product teams with shared budgets. LiteLLM's production guide names PostgreSQL as the primary supported database and requires Redis 7.0 or newer once more than one proxy instance runs, because rate limits, router state and cache must be shared. It suggests 1 vCPU and 4 GiB memory per worker as a starting point, and recommends one Uvicorn worker per pod with horizontal scaling. Ask your team to size these components against your own request volume, then plan for ongoing upgrades. LiteLLM maintains support for the four most recent stable minor versions, so upgrades are a recurring task rather than a one-time setup.
Can our team run a self-hosted LiteLLM gateway without outside help?
A team with Kubernetes, PostgreSQL and Redis experience can run LiteLLM alone, provided someone owns the design decisions and the on-call rota. The harder parts are usually not the first deployment but the operating model: key issuance and rotation, budget policies, provider failover behavior, logging into your observability stack, and upgrades that must be tested before they reach production. Outside help is most useful when your team lacks one of those skills, when the deadline is fixed, or when you need evidence for an assessment. Whichever path you choose, write down who owns each component before go-live.
Does SSO for the LiteLLM admin UI require an Enterprise license?
Not for small teams. The LiteLLM admin UI supports SSO with Google, Microsoft Entra ID (Azure AD), Okta and generic OIDC providers. From v1.76.0, SSO is free for up to 5 users, and an Enterprise license is needed beyond that. Enterprise also adds SCIM provisioning, JWT authentication with identity provider tokens, audit logs, organization and team RBAC, and IP allowlisting. If you need audit logs for an assessment or SCIM for joiner and leaver automation, budget for the Enterprise license and confirm the current terms with LiteLLM before you commit, since licensing can change between releases.
Do provider API keys leave our control with a self-hosted LiteLLM gateway?
No, when you run the gateway yourself. In a self-hosted setup, you create the accounts with each provider, store their keys in your own secret manager, and LiteLLM uses them from your infrastructure. Applications receive LiteLLM virtual keys instead of provider keys, which you can revoke, budget and restrict to specific models. With a hosted router, the platform holds the relationship with the providers and the keys are managed on its side. Confirm where your provider agreements sit, which party is the customer of record, and how keys are rotated. LiteLLM recommends keeping its master key in a secret manager, and Enterprise adds integrations with AWS, Azure, Google, HashiCorp Vault and CyberArk.
What skills and tools does the team need to operate LiteLLM?
Plan for these components and skills:
- Kubernetes or another container platform, with one Uvicorn worker per pod and horizontal scaling
- PostgreSQL as the primary database, which virtual keys, budgets and spend tracking require
- Redis 7.0 or newer once you run more than one proxy instance
- A secret manager for
LITELLM_MASTER_KEY, which must start with sk-, and for provider keys - An on-call owner who can read gateway logs and respond to provider outages
- Monitoring and logging, since LiteLLM can send logs to Langfuse, Datadog, OpenTelemetry and Prometheus
Ask each engineer who will touch the gateway which of these they already run in production.
What does a fixed-scope implementation engagement cover for LiteLLM?
DeployOpen plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a LiteLLM engagement, a written scope typically defines the deployment target, database and Redis design, key and budget model, provider and fallback configuration, logging integration, backup approach and handover documentation. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. Ongoing support, upgrades and continuous testing are available as retainers. DeployOpen does not publish a price list, and a quote follows from the written scope. DeployOpen prepares systems and evidence for compliance assessments but does not issue audit reports or certifications.
What should we ask a vendor or partner in a LiteLLM quote?
Ask for a written scope that lists each component, the environment it runs in, and who operates it after go-live. Request the number of proxy instances, the PostgreSQL and Redis topology, how backups and restores are tested, and how upgrades are staged across the four supported minor versions. Ask which features assume Enterprise licensing, such as SCIM, audit logs and team-level guardrails, so the license cost is part of the comparison. Ask what is one-time work and what recurs, including retainers, license renewals and cloud resources. Finally, ask how provider keys, master keys and virtual keys are issued, stored and rotated, and what evidence you receive for your assessment.
Questions to answer before you pick a gateway
Start with contracts and keys, because they decide the rest. Confirm internally who must hold the provider agreements and API keys for OpenAI, Anthropic, Gemini, Bedrock or Azure, and whether finance and procurement accept direct provider billing in place of one OpenRouter invoice. With LiteLLM, your organization holds those accounts and the virtual keys issued to applications, so every provider agreement has to be opened, owned and rotated by your team. Ask OpenRouter for its current terms on data handling, credit purchases, BYOK fees and regional routing in writing, and ask LiteLLM directly for the terms that apply to its enterprise directory, since its open-source MIT code and the enterprise license are separate documents with separate obligations.
Next, fix your residency and audit requirements. If prompts or completions must stay in a named region, your gateway, its database and its logs must run there too, and the providers you route to must be contracted for that region. Decide whether you need SSO for more than five users, which the LiteLLM admin UI offers free from v1.76.0 only up to five users before an Enterprise license applies, and whether audit logs or SCIM provisioning are mandatory for your assessors. If they are, price the Enterprise license into your comparison from the start. Also count your teams and expected request volume, since those drive how many virtual keys, budgets and proxy instances you need, and whether Redis becomes necessary once more than one instance runs.
Finally, test whether your team can operate the stack. LiteLLM's production guide names PostgreSQL as the primary database and requires Redis 7.0 or newer for shared rate limits and router state once you scale out, and virtual keys depend on PostgreSQL. Confirm that someone on your platform team can handle backups, upgrades and failover for both, and that Kubernetes operations are in place if you plan to run multiple pods. If that capacity does not exist, decide whether to hire, extend the team, or bring in an implementation partner under a written, fixed scope before you commit to self-hosting.
How DeployOpen helps you evaluate or move to LiteLLM
DeployOpen is a Chainscore Labs company that plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a LiteLLM gateway, we start with the written scope of which providers, keys, budgets and logging your teams need, then build and hand over the service in your environment.
How we scope
- We inventory the OpenRouter models, providers and fallbacks your applications call today.
- We confirm which provider contracts and API keys your organization will hold directly.
- We agree the written scope, deliverables and timeline before any build work starts.
How we build
- We deploy LiteLLM with PostgreSQL and, for multiple proxy instances, Redis in your infrastructure.
- We configure virtual keys, team budgets, rate limits and model groups mapped to your inventory.
- We test routing, fallbacks and spend tracking against your traffic patterns before cutover.
How we hand over
- We deliver runbooks for key rotation, database backups, upgrades and provider account changes.
- We document configuration, access controls and logging so your team can operate the gateway.
- We prepare evidence for your compliance reviews, while independent auditors issue any formal reports.
How we support
- Ongoing support, upgrades and continuous security testing are available as a retainer.
- Retainers can cover version upgrades for the LiteLLM releases your organization adopts.
- Engagements are fixed-scope, so any change to providers or features is agreed in writing first.
