How LiteLLM virtual keys replace scattered provider keys
Teams that call model providers with shared or personal API keys usually have no single view of who is spending what, and no way to stop one application from consuming the whole budget. Virtual keys issued by the LiteLLM proxy answer this problem. Each key carries its own budget, TPM and RPM limits and a list of permitted models, and applications call the gateway with that key instead of holding a provider credential. The gateway keeps the real provider keys, so they no longer need to be copied into every repository, container or developer laptop.
Virtual keys need a PostgreSQL database to store key records, budgets and spend. The gateway holds the real provider credentials in its configuration, and the master key that administers the proxy must start with sk- and should live in a secret manager, not in a YAML file or an environment variable pasted into a deployment manifest. Confirm where your secret manager sits in the network path and who can read the master key, because anyone holding it can issue keys and raise budgets. Installation and the database setup are covered in the implementation plan topic.
The YAML that defines a key is the easy part. The hard parts are design decisions you must make before issuing anything. Decide who owns each key: a team, a user, or a single application, and who may request changes. Decide how budgets nest, so that a team budget is not silently exceeded by the sum of its keys and so that temporary increases have an approver and an expiry. Decide which models each key may call, and keep that list in step with the approved provider list so a new model does not appear in a key by accident.
Reporting is the final decision. Spend data is only useful if finance, platform owners and team leads agree on which tags, teams and keys map to cost centers, and on how often spend is reviewed. Ask your platform team to confirm how spend updates reach the database at your expected volume, since LiteLLM's production guidance describes batching spend writes at scale. Plan a test with a staging key and a known budget to confirm that limits are enforced and that the numbers in reports match the provider invoices.
Single sign-on for the admin UI, the choice between open source and Enterprise governance features, and the routing and fallback settings that decide which provider serves each request are separate topics in this series. Read the linked LiteLLM documentation on virtual keys and the Enterprise features before you finalize your key model, and confirm with LiteLLM which capabilities your chosen edition includes.
Which LiteLLM settings control keys, budgets and spend
Access, limits and cost reporting in LiteLLM come from a small set of connected components: the proxy, the administrative master key, virtual keys, users and teams, budgets, rate limits and the stores behind them. Each card below explains one component, what it depends on and what to confirm before you rely on it for chargeback or access control.
Proxy server and master key
The LiteLLM proxy is the AI gateway that applications call with the OpenAI request format, and it is where keys, budgets and limits are enforced. The master key, which must start with sk-, is the administrative credential that creates and manages virtual keys and teams. Store it in a secret manager, limit who can read it, and confirm how the version you run exposes admin functions.
Virtual keys
A virtual key is a LiteLLM-issued credential that stands in for provider API keys, so applications never hold direct provider secrets. Each key is the unit to which budgets, rate limits and model restrictions attach. Plan issuance per team or application, and confirm in the virtual keys documentation for your version how keys are created, rotated and revoked.
Users and teams
Users and teams give keys an owner and a grouping for spend reporting and team-level budgets. Map them to your business units or cost centers before issuing keys, because renaming or restructuring later changes how historical spend reads. Confirm whether your edition supports team-level role controls, since Enterprise adds organization and team RBAC.
Per-key, per-user and per-team budgets
Budgets cap spend at three levels: individual keys, users and teams, and each level can be set independently. Overlapping budgets mean the first limit reached blocks requests, so decide which level is authoritative for each application. Confirm the budget currency, the reset period and how spend is calculated for each provider in the version you run.
TPM and RPM limits
TPM (tokens per minute) and RPM (requests per minute) limits protect shared provider quotas and stop a single application from consuming all capacity. They are set per key, user or team, and enforcement across several proxy instances depends on shared state. Test limits with realistic traffic and confirm the counting window and behavior on limit breach in the routing and virtual keys docs.
Temporary budget increases and model restrictions
Temporary budget increases let an owner raise a limit for a defined period without reissuing a key, which is useful for launches and incident response. Model restrictions limit a key to approved models or aliases, so a leaked key cannot reach models outside its approval. Record who may grant increases, and confirm how expiry and model lists are configured in your version.
PostgreSQL and Redis dependencies
Virtual keys require PostgreSQL, which stores keys, teams and spend. Redis, at version 7.0 or newer, becomes necessary once more than one proxy instance runs, because it shares rate limit counters and router state. Size both stores before go-live, back up PostgreSQL, and confirm the spend update batching interval your version uses under load.
Which budget and key model fits each team
This table compares the decisions a platform owner makes before issuing the first LiteLLM virtual key: how keys map to applications and teams, where budgets sit, how overruns are handled, how model access is restricted, and how limits are shared across proxy instances. Read each row as a trade-off, with the operational cost listed in the right-hand column.
| Decision | When it fits | Operational cost |
|---|---|---|
| One key per application | Few applications with distinct owners, and spend needs to be traced to a single service | More keys to rotate and track as applications multiply. A leaked key exposes only that application's budget and models, so blast radius stays small. |
| One key per team | Several applications share a team budget and one owner approves spend | Fewer keys to manage, but a leaked key exposes every application the team runs. Spend reporting is per team, not per service. |
| Budget set on the key | A specific workload needs a hard ceiling regardless of who uses the key | Each key needs its own budget value and review. Changes to one workload do not move the others. |
| Budget set on the team or user | Spend should follow an accountable owner across several keys | Reporting and cap decisions sit at the owner level. Individual keys can still exhaust a shared pool, so pair with per-key limits where needed. |
| Hard cap | Finance requires spend to stop at a fixed ceiling and overruns are unacceptable | Teams hit a stop mid-cycle and need an escalation path. Confirm the exact behavior at the budget limit in your LiteLLM version. |
| Temporary budget increase | Launches, batch jobs or incidents need short bursts above the normal budget | Someone must approve each increase and set an end date. Without a review step, temporary increases quietly become permanent. |
| Model allowlist per key | Applications need access to a fixed set of approved models | Every new approved model must be added to each affected key. Mistakes block traffic or open unapproved models. |
| Model allowlist per team | Teams share one approved catalog and new keys inherit the same model access | Simpler to administer, but a team-wide change reaches every key in the team at once. Review the catalog on each model addition. |
| Redis-backed shared limits | More than one proxy instance enforces TPM, RPM or budget counters | Redis becomes an operated dependency with its own sizing, failover and monitoring. LiteLLM's production guide requires Redis 7.0 or newer once several instances run. |
| Single-instance counters | Only one proxy instance runs, typically in a non-production or pilot setup | Counters reset or diverge if a second instance is added later. Moving to shared limits later requires a planned change window. |
How to build virtual keys, budgets and spend reporting in order
This sequence builds the access model for a LiteLLM gateway in the order the proxy depends on it: ownership and model scope first, then the database and keys, then limits and reporting. Each step closes with a check you can observe, such as a rejected call, a logged spend record or a changed budget. The result is a tested set of keys mapped to teams and a spend report the business owner can use.
Define teams, applications, owners and model scope
Before any key exists, list each team, the applications it runs, a named owner for each application and the models that owner may use. Ask the owner to confirm the approved model list against the providers and private models your platform team has connected. Keep this as a written table, since every key, budget and report in later steps traces back to it. Done when each application maps to one team, one owner and one approved model list, and the owners have signed off the table.
Configure PostgreSQL and the master key
Deploy the proxy with a PostgreSQL database, because virtual keys, budgets and spend records require it. Store the master key, which must start with sk-, in your secret manager and inject it into the proxy at runtime. If you plan to run more than one proxy instance, add Redis 7.0 or newer now, since shared rate limits depend on it. Done when the proxy starts, reads the key from the secret store, accepts an admin request made with the master key and refuses an unauthenticated request.
Create team and user records
Create a team record for each team from step one, then user records for the people who will manage keys or view spend. Set each team's budget and approved models at the team level so that keys issued under the team operate within that ceiling. Confirm that a user outside the team cannot see its keys in the admin UI, and that a changed team budget appears on the team record. Done when the records match your table and each test user sees only what their role allows.
Issue a test key with a model restriction
Create one virtual key for a test application under a team, restrict it to a single approved model and set a small budget that is easy to exhaust. Send a call to the permitted model, then send a call to a model outside the key's list. The second call should be refused. Record the key ID, team and budget in your test log so later steps can compare results. Done when the permitted call succeeds and the restricted call is rejected.
Confirm requests are attributed correctly
Send a request with the test key, then find its spend record in the database or admin UI. Check that the record names the same key, user and team as your table, and that the model and cost fields are populated. Repeat with a second key on the same team to confirm both keys roll up to one team total. If the record is missing, confirm the request went through the proxy, since direct provider calls leave no record. Spend updates may be batched, and LiteLLM's production guide suggests batching every 60 seconds at scale, so allow that interval before concluding a write failed.
Set TPM and RPM limits and confirm rejection
Set tokens-per-minute and requests-per-minute limits on the test key, using the values the owner agreed in step one. Send requests until a limit is reached and confirm the proxy returns a rejection instead of forwarding the call. Check the log entry for the rejection and confirm the refused call did not add to the spend total. If you run more than one proxy instance, repeat the test across instances to confirm the limit is shared, since that depends on Redis. Done when both limits produce observable rejections.
Test a temporary budget increase and its expiry
Grant the test key a temporary budget increase with an expiry time, then send a call that would have failed under the original budget. Confirm the call succeeds while the increase is active, then confirm the key is refused again once the expiry passes. Record the change and its expiry in your change record so the approval trail is complete. Done when the budget returns to its original value after expiry and the refusal is visible in the logs.
Confirm spend totals and schedule the export
Check that spend totals appear per key, per user and per team in the database or admin UI, and that the three totals reconcile with the test calls. Then set a recurring export of spend data to finance or the business owner, in the format and frequency they agree to. Ask the owner to confirm the export arrives and that its totals match the proxy's records. Done when the first export matches the database totals for the test calls and the owner has confirmed receipt.
How key and budget design changes across single node, HA and Kubernetes
A single proxy instance is the simplest shape. One process handles every request, so rate limits for virtual keys (TPM and RPM) can be enforced from the proxy's own view of traffic, and spend is written to PostgreSQL as requests complete. Budgets, model restrictions and temporary budget increases work the same way they do in larger shapes; only the plumbing underneath changes.
What changes for this shape:
- Rate limits are tracked in the single instance without a shared store, so limits are exact for that one process.
- Spend writes go directly to PostgreSQL, which is the primary supported database. Confirm how often your deployed version writes spend at your expected request volume.
- Workers and sizing should follow the production guide's suggestion of 1 vCPU and 4 GiB memory per worker. Count workers on the host, not just the instance.
- Redis is not required by the production guide until more than one instance runs, so it is optional here.
- Failure impact is total: if the single instance or its database is unavailable, every key stops working.
Before go-live, confirm that your budgets are reset on the schedule your finance team expects, and that the model restriction lists match the approved catalog.
Once more than one proxy instance sits behind a load balancer, per-instance counters no longer add up to the real traffic for a key. The production guide requires Redis 7.0 or newer in this shape for shared rate limits, router state and cache. Without shared state, a key with a 60 RPM limit could effectively receive that limit on each instance, so the limit has to be enforced from a common store.
What changes for this shape:
- Rate limits move to Redis so every instance counts against the same TPM and RPM totals for a key.
- Spend write load rises with instance count. At scale, the production guide recommends batching spend updates to the database every 60 seconds, so reports lag by up to that interval.
- Router state such as cooldowns and failover decisions is shared through Redis, which keeps instances from routing differently around the same provider failure.
- Sizing should be repeated per instance, using the suggested 1 vCPU and 4 GiB memory per worker as the starting reference.
- Redis availability becomes part of the key path. Confirm the failure behavior of your deployed version when Redis is unreachable.
Before go-live, test one key against all instances during a load test and confirm the combined limit holds and the spend report reflects the 60 second batching you accepted.
On Kubernetes, the proxy usually runs as a deployment with horizontal scaling, so instance count changes with load. LiteLLM recommends running one Uvicorn worker per pod and scaling out with more pods, rather than adding workers inside each pod. This keeps resource requests predictable and lets the scheduler and autoscaler control capacity.
What changes for this shape:
- Workers per pod should be one, with capacity added by replicas. Each replica then counts as one instance for shared limits.
- Rate limits and router state rely on Redis, because the number of pods changes during autoscaling events.
- Spend writes scale with replica count, so the 60 second batching guidance applies at scale and should be reviewed when autoscaling ranges widen.
- Resource requests should start from the production guide's suggestion of 1 vCPU and 4 GiB memory per worker, then be tuned against measured load in your cluster.
- Master key handling matters more here: keep LITELLM_MASTER_KEY, which must start with sk-, in a secret manager and inject it into pods rather than storing it in manifests.
Before go-live, confirm the readiness and liveness probes, the autoscaling bounds and the Redis and PostgreSQL connection limits, since each new pod opens its own connections.
What drives the effort of setting up keys and budgets
This table lists the factors that change how much work it takes to issue virtual keys and budgets in LiteLLM, with a way to keep each one contained. Read each row as a question to settle before build starts, since the answers set how many keys, budgets and reports the team must maintain. The main takeaway is that most effort comes from scope decisions made early, not from the proxy configuration itself.
| Driver | Why it changes effort | How to keep it down |
|---|---|---|
| Number of teams and applications | Each team or application that needs its own key and budget adds a key to issue, rotate and report on. Keys and budgets multiply with every new application. | Issue keys per application family rather than per microservice, and give each team one budget that its applications share. |
| Number of approved models and providers | Every model a key may call must be listed, priced and tested for routing. More providers mean more credentials and more model names to map. | Publish one shared model list with fixed aliases, and restrict keys to that list instead of granting access model by model. |
| Cost-centre hierarchy for budgets | If budgets must roll up to an existing finance structure, the team needs to map keys to cost centres and keep that mapping current through reorganizations. | Map budgets to a small number of cost centres that finance already reports on, and confirm the mapping with finance before creating keys. |
| Identity provider integration for user-level keys | Per-user keys need an identity source, onboarding and offboarding steps, and SSO configuration. Admin UI SSO and JWT or SCIM options are separate pieces of work and licensing. | Start with team and application keys, and add user-level keys only for named use cases. Confirm which SSO and provisioning features your edition includes. |
| Number of environments (dev, test, production) | Each environment usually needs its own keys, budgets and limits, so the same team can have several keys and separate spend records. | Keep development and test budgets low and separate from production, and generate environment keys from one template. |
| Spend reporting format required by finance | A custom report layout, currency rule or allocation method requires extra queries and checks each cycle. Ad hoc reporting also creates recurring effort. | Agree one fixed export with finance, with the same columns and period, and schedule it instead of building reports on request. |
| Approval workflow for temporary increases | Temporary budget increases need a request, an approver, an expiry date and an audit trail. Without a defined process, each request becomes a manual exception. | Define one request form, a named approver for each team, and a default expiry period. Confirm the approval steps with your own change process. |
| Key rotation policy | The frequency of rotation sets how many keys must be replaced, how many applications must be updated, and how often secrets reach configuration systems. | Rotate keys on a schedule that matches your security policy, and keep keys in a secret manager so applications pick up new values without code changes. |
How to test that budgets and limits hold under load
Before a team relies on LiteLLM virtual keys for spend control, it should prove each control with a test that can fail. The core cases are: a key cannot call a model outside its allowed list, a key stops when its budget is reached, TPM and RPM limits reject excess traffic, a temporary budget increase applies and then expires, and a revoked key fails on its next request. Write each case with the key, the model list, the limit value and the expected HTTP response before running anything. Run the same cases against each budget scope in use (per key, per user and per team), because a control that works on one scope can be misconfigured on another.
Run the tests in staging with a non-production provider account. Create a dedicated provider project or account with a low spending cap and a small set of models, and point a staging proxy at it. Load tests can exhaust real provider quotas, generate real charges and send test prompts to production logs, so keep staging separate from the production database and master key. Confirm with your provider how quickly usage appears in its billing data, because LiteLLM's own spend records and the provider's invoice will not match exactly in the short term. Note the LiteLLM version under test. The LiteLLM releases page lists v1.105.1 as the latest release at the time of this writing, but confirm the version you deploy against the release notes before you start.
The multi-instance cases are where budgets most often drift. Run two or more proxy instances against one PostgreSQL database and a shared Redis, send requests through the load balancer, and then stop one instance mid-test. Spend recorded by the surviving instance should land on the correct team and key, and the total should match the sum of the requests that returned successful responses. Without shared Redis state, RPM and TPM counters can be enforced per instance, so a key may get more throughput than its limit. Confirm this behavior in your topology rather than assuming it.
Spend batching needs its own restart test. LiteLLM's production guidance describes batching spend updates to the database every 60 seconds at scale, so a process killed between batches may leave unwritten spend. Send a known number of requests with known token counts, kill the proxy process during a batch window, restart it, and reconcile the database rows against the provider usage for the same period. Record the gap as an accepted loss or fix it before go-live. Also decide the tolerance for loss with finance, since it determines whether batching at the default interval is acceptable for chargeback.
Decide what the caller sees when a budget runs out. The request should fail with a clear error code that the application can handle, not a generic 500 or a silent fallback to a more expensive model unless you have chosen that behavior deliberately. Test the exact status code and error body your version returns, and confirm that application retry logic does not hammer the gateway after a budget error. Then test alerting: confirm whether a budget breach triggers a notification to the team owner, which channel receives it, and who acts on it. If the gateway does not send that alert in your version, plan an external check on spend against each budget and document who owns the response.
How to protect virtual keys, the master key and spend data
Virtual keys, the master key and spend records all grant or reveal access to models that cost money and send business data to providers. The cards below cover who can administer the gateway, how keys are issued and rotated, what each key may call, and where sensitive values must never appear. Confirm which governance features your LiteLLM license covers before you design around them.
Admin access and the master key
The master key is the highest-privilege credential on the proxy and must start with sk-. Keep it in a secret manager, not in configuration files or container images, and limit who can read it from that store. Restrict admin UI accounts to the people who manage keys and budgets. Ask whether your license covers team RBAC and audit logs before you rely on them for separation of duties.
Showing and storing virtual keys
A virtual key is the credential a team or application uses to call the gateway, so treat its creation as a handover event. Plan to display each generated key once, deliver it through your secret distribution process, and store only what the application needs. Agree with the owning team who holds each key and where it lives, so a lost key is reissued rather than recovered.
Rotation and revocation runbooks
Write separate runbooks for scheduled rotation and emergency revocation of virtual keys and the master key. Each runbook should name the trigger, the owner, the order of steps (issue the new key, update the consuming application, then disable the old one), and how to confirm no traffic still uses the old value. Test the emergency path before go-live, because budgets and spend history depend on the key and team mapping staying correct.
Least-privilege model lists
Restrict each virtual key to the specific models its application needs instead of granting the full model catalog. For example, an illustrative support chatbot key would list only the approved chat model and its fallback, not image or embedding models. Review model lists whenever a provider or private model is added, because new models should not inherit access by default.
Provider credentials behind the gateway
Provider API keys stored in the proxy are the most valuable secrets in the design, because anyone who reads them can bypass budgets and logging entirely. Keep them in the secret manager, scope each one to the minimum provider project or account, and rotate them on a schedule separate from virtual keys. Confirm with your cloud and provider teams how each credential is created and who can change it.
Logging without prompts or key values
Request and spend logs are useful for chargeback, but they can capture prompt content and key values if verbose logging is enabled. Decide which fields are recorded, such as key alias, team, model and token counts, and confirm that prompts, responses and secrets are excluded or masked. Check the logging and callback settings in your version, and review log destinations for access controls and retention.
Network exposure of the proxy
The proxy and its admin interface should only be reachable from networks that need them, such as application subnets and an administrative network. Decide whether IP allowlisting is available under your license or should be enforced at the load balancer or network layer. Keep the database and Redis off public networks, since spend records and rate-limit state are sensitive in their own right.
How to report spend back to the business and keep keys current
Spend reporting only works if the gateway keeps producing trustworthy records after go-live, so the first job is to decide what the operations team watches and who acts on it. Set alerts for budget exhaustion on virtual keys, teams and users, because a key that hits its budget will start rejecting requests and the owning team needs to know before a release or batch job fails. Track rejected requests separately from provider errors, since a spike in budget or rate limit rejections points to a governance decision rather than an outage. Review spend trend per team each week in the first months, and watch for failed spend writes in the proxy logs and database, because missing spend rows understate cost and quietly break the report that finance relies on. Confirm the exact metric names, log messages and spend fields for your version in the LiteLLM docs before you build dashboards on them.
What to back up The PostgreSQL database is the system of record for virtual keys, budgets, team and user assignments and spend records, so it needs the strictest backup and restore plan in the deployment. Define the recovery point you can accept for spend data and test a full restore into a separate environment, then confirm that a restored instance issues the same keys and enforces the same budgets. Keep LITELLM_MASTER_KEY in your secret manager with its own backup and rotation record, because losing it can lock administrators out of the proxy even when the database is intact. Back up configuration files and model definitions in version control so a rebuilt environment can reproduce routing without manual reconstruction.
Upgrades change the database schema and sometimes the way budgets and spend are calculated, so treat each one as a change that needs a retest. LiteLLM maintains support for the four most recent stable minor versions for both open source and Enterprise deployments, so plan a regular cadence that keeps you inside that window and record the version you run in your change log. Before a production upgrade, run the database migration against a copy of production data, then check that existing keys still authenticate, that budgets and TPM and RPM limits still trigger at the expected thresholds, that restricted model lists still block unapproved models, and that spend increments after a test call. Repeat the budget test after every upgrade, not only after major releases, and keep the results with the change record.
Documentation that auditors and finance will ask for Write down who owns each virtual key, which application and team it serves, who approved its budget and models, and when it was last rotated. Keep a register of budget approvals with the approver, the amount, the period and any temporary budget increases, because temporary increases are easy to forget and later appear as unexplained spend. Set a monthly spend export as a formal control: name the owner, the day it runs, the fields exported, the reconciliation check against provider invoices and the place the file is stored. Confirm which export options and spend fields your LiteLLM version provides before you define the template.
From spend records to chargeback or showback A chargeback or showback report groups spend records by team, application and key, then maps those groups to cost centers and applies the agreed price or allocation rule. Showback shows teams their usage without billing them, which is often the right first step while budgets are still settling. Finance usually needs the period, the cost center, the model and provider, the total spend, the budget in force, any overrides or temporary increases and the reconciliation difference against provider invoices. Agree the allocation rules, including how shared keys and unused budget are handled, before the first monthly run, so the numbers do not change once teams start disputing them.
What buyers ask about LiteLLM keys, budgets and spend work
How much does a LiteLLM virtual key and budget rollout cost?
Cost follows the written scope, which depends on how many teams, applications and providers receive keys and how much reporting finance expects. DeployOpen does not publish a price list, so a quote is produced from that scope. The main cost components are discovery and design of the team and budget structure, configuration of virtual keys, budgets, limits and model restrictions, connecting spend data to the report format finance uses, testing, and handover. One-time work covers the build. Recurring costs sit mostly in infrastructure for PostgreSQL and the proxy, plus any support retainer you choose. Ask each bidder to separate one-time from recurring work and to state what the handover includes. Confirm with LiteLLM which features fall under the enterprise directory license, since the open-source license covers the rest of the code.
How long does a LiteLLM spend tracking project take to deliver?
Duration depends mainly on the number of teams and applications that receive keys and on how much reporting the finance and business owners need from spend data. Each team needs a named owner, agreed model access, and budget and rate limits, and each report needs a defined format before it can be built and checked. A pilot with one team and a basic cost report takes a very different path from a company-wide rollout with chargeback by cost center. No duration is given here because it changes with those inputs. Ask for a timeline in the written scope that lists phases, the teams in each phase and the reporting outputs, and check it against your change calendar and approval cycles.
What does our team need to provide before a LiteLLM key project starts?
Plan to provide named owners for each team, a list of approved models with an approver for each, the finance reporting format, and a PostgreSQL database that the proxy can reach. Team owners decide which applications get keys, what budget and TPM and RPM limits each key carries, and who can request a temporary budget increase. Security or architecture leads confirm which providers and private models each team may use. Finance defines the fields, periods and cost allocation it needs. If you plan more than one proxy instance, you also need Redis 7.0 or newer for shared rate limits and router state. Confirm network access, change windows and where the master key will be stored early in the project.
Can our platform team set up LiteLLM virtual keys on its own?
Often yes for a single cluster and a small number of teams, provided the team already runs PostgreSQL and Kubernetes or similar platforms and has time to own the configuration. The harder parts are rarely the proxy itself. They are agreeing on team structure, budget policy, model approvals and a spend report that finance accepts. Teams often find that testing budgets and limits under realistic load takes longer than expected, and that rotating keys and reviewing spend become ongoing work after go-live. Ask whether your team has capacity for each phase in a written plan, who signs off on policy, and who operates the system afterward. A partner can take on the build while your team keeps ownership of policy decisions.
Do we need SSO to give each user their own LiteLLM key?
Not necessarily. Virtual keys, per-user budgets and per-team budgets are proxy features that the documentation describes separately from sign-in. SSO matters mainly for signing into the admin UI with Google, Microsoft Entra ID, Okta or a generic OIDC provider. Since v1.76.0, SSO is free for up to 5 users, and an Enterprise license is needed beyond that. If you want team membership to follow identity provider groups, the documentation places SCIM and JWT authentication in the Enterprise edition, so confirm the exact feature set for your version before you design around it. Count the admin users who need console access, because that number decides whether the free tier fits your plan.
What should we confirm in writing before a LiteLLM engagement starts?
Confirm the scope, deliverables, acceptance criteria and timeline in one written document before work begins. DeployOpen engagements are fixed scope, so the agreed providers, teams, keys, budgets, model restrictions and reporting outputs should be listed explicitly. Also confirm the environment where the proxy runs, who holds the master key and provider credentials, the PostgreSQL and Redis versions, and the LiteLLM minor version to deploy from the four supported minor versions. Put in writing who approves model access, how budget changes are requested, what testing proves the limits work, and what support follows go-live. If you need incident response times, ask for them in writing, because LiteLLM's optional Enterprise SLAs are separate from the open-source project and carry their own terms.
How do TPM and RPM limits behave when several LiteLLM proxy instances run?
Shared counters need shared state, so once more than one proxy instance runs, Redis 7.0 or newer is required for shared rate limits, router state and cache. Spend is written to PostgreSQL, and LiteLLM recommends batching spend updates to the database every 60 seconds at scale, so spend reports may lag real usage by up to that window. Engineers should send traffic for one key through two instances at the same time and confirm that TPM and RPM limits are enforced as configured. Keep LITELLM_MASTER_KEY, which must start with sk-, in a secret manager. On Kubernetes, LiteLLM recommends one Uvicorn worker per pod and horizontal scaling. The production guide suggests 1 vCPU and 4 GiB memory per worker as a starting point, so size the pods from your own load tests.
How DeployOpen scopes, builds and hands over LiteLLM key and budget work
DeployOpen plans, implements, documents and supports self-hosted LiteLLM inside the client's own infrastructure, including the virtual keys, budgets and spend tracking that govern who can call which models. Each engagement starts from a written, fixed scope so that teams, keys, limits and reporting are agreed before any configuration begins.
How we scope
- Inventory the teams and applications that will receive virtual keys and the approved models each one may call.
- Agree per-key, per-user and per-team budgets, TPM and RPM limits, and the rules for temporary budget increases.
- Define the spend reports the business needs and confirm the PostgreSQL database that stores key and spend data.
How we build
- Configure key hierarchies, model restrictions and budget limits in the proxy, using the approved models list from scoping.
- Test that requests beyond a budget or rate limit are refused and that keys cannot reach models outside their scope.
- Verify that spend records match test traffic against the provider usage pages before any application receives a key.
How we hand over
- Deliver written configuration records covering keys, owners, budgets, model access and the spend reporting method.
- Document the key issuance, rotation and budget change procedures so that your operators can run them without DeployOpen.
- Walk your platform and finance owners through the spend reports and the steps to request a limit change.
How we support
- Offer ongoing support, upgrades and testing as a retainer, separate from the fixed-scope build.
- Retest key limits and spend tracking before proposed LiteLLM upgrades reach your production environment.
- Confirm with your team which support tier and response commitments apply, since these depend on your agreement and edition.

