vLLM multi-tenant isolation and cache salting

When one vLLM server must serve several tenants safely

Serving several internal teams or external customers from one vLLM server needs two things at once: an architecture decision about where tenant boundaries sit, and per-tenant cache_salt values that keep prefix cache reuse inside each tenant. Neither is enough alone. A shared server with salting still exposes the endpoints, metrics and admin routes to every caller, and a well-separated architecture still leaks timing information if all tenants share one prefix cache without salts. This section explains when teams need this pattern and what it costs to run.

Teams usually need this pattern for three reasons. The first is shared capacity: a few large accelerators are often better used by several teams than split into idle silos. The second is customer data separation, where contracts or regulators require that one customer's prompts and cached context never influence another customer's responses. The third is cost sharing, where a platform team charges business units for the capacity they consume. Before designing anything, confirm which tenants will share a model, which data classes each tenant sends, and who funds the accelerators. Those answers decide how strict the boundary must be.

vLLM is open source under the Apache-2.0 license and has no paid edition, so the license does not appear as a line item. The money goes to accelerator capacity, the surrounding infrastructure such as networking, storage, proxies and monitoring, implementation work, and the people who operate the service. Implementation costs are mostly one-time: architecture design, gateway or proxy configuration, isolation testing and documentation. Operating costs recur: accelerator hours, staff time, monitoring, and the upgrade work that follows vLLM's new minor releases, which arrive about every two weeks. When you request a quote, ask for the tenant model, the isolation tests included, the upgrade cadence covered and the assumptions about accelerator sizing.

Plan the controls before the first tenant onboards. Generate cache_salt values in the platform, using random secret values such as 256-bit strings, and never derive them from user names or other guessable identifiers. Keep the vLLM server behind a reverse proxy that allowlists only the endpoints tenants need, because the --api-key option does not protect every route, including /metrics, /health and the pause and abort endpoints. Keep multi-node traffic on an isolated network, since vLLM's security documentation says inter-node communication is insecure by default. Prefix caching was also the subject of CVE-2025-46570, so confirm your deployed version and test the timing behavior in your own environment.

Per-tenant instances, a shared gateway or both: choosing the isolation model

This table compares three ways to separate tenants on vLLM: one instance per tenant, one shared instance behind a gateway, and a hybrid that gives sensitive tenants dedicated capacity. Read each row as a trade-off, then test the choice against your own threat model before you commit to a design. The main takeaway is that a shared instance saves accelerator capacity but makes the gateway and cache controls the isolation boundary.

Factor	One vLLM instance per tenant	One shared vLLM instance behind a gateway	Hybrid: dedicated instances for sensitive tenants
Isolation strength	Strongest: separate processes, KV cache and prefix cache per tenant, so cache timing cannot cross tenants	Weaker: tenants share the engine and prefix cache, so cache_salt values and gateway controls must hold	Strong for sensitive tenants, weaker for the shared pool, which still needs salts and gateway controls
GPU utilization	Lower: each instance holds its own weights and idle capacity, so accelerators are often underused	Higher: one set of weights and batching serves many tenants	Middle: shared pool gets high utilization, dedicated instances sit partly idle
Cost sharing	Costs are charged per tenant, and each tenant pays for its own accelerator capacity	Costs are pooled and allocated through per-tenant budgets and usage tracking	Pooled cost for most tenants, with a separate cost line for dedicated instances
Operational load	Highest: each instance needs its own upgrades, metrics, scaling and restarts	Lowest: one deployment, one upgrade path and one set of dashboards	Moderate: two deployment patterns to maintain and test
Where tenant identity is enforced	At the network or proxy layer in front of each instance, since the engine has no tenant concept	At the gateway, through per-tenant virtual keys, budgets, model allowlists and salts set on each request	At the gateway for shared tenants and at the network boundary for dedicated tenants
Prefix cache exposure	Limited to one tenant, so the timing side channel (CVE-2025-46570) does not cross tenants	Requires a per-tenant cache_salt on every request, enforced by the gateway, not by the client	Shared pool needs cache_salt enforcement, dedicated tenants do not
Gateway option (LiteLLM)	Optional, used only for routing and logging	MIT-licensed core, virtual keys need PostgreSQL, and Redis 7.0 or newer once several proxy instances run	Required for the shared pool, with the same PostgreSQL and Redis needs
Confirm before choosing	Accelerator quotas, cost model and the number of instances your team can operate	Whether the gateway itself is in the threat model, and who can read or change salts and keys	Which tenants count as sensitive, and whether the regulator or contract requires dedicated hosts

How cache_salt closes the prefix cache timing side channel

Prefix caching lets vLLM reuse computed attention states when two requests begin with the same prompt text, which saves time for shared system prompts. On a multi-tenant server, that reuse becomes a timing side channel, tracked as CVE-2025-46570. An attacker who sends crafted prompts can infer whether another user's prefix is cached by measuring time to first token, because a cache hit returns the first token sooner than a miss. The optional cache_salt request parameter closes this path by changing what counts as a cache match.

The mechanism is specific. vLLM mixes the cache_salt value into the hash of the first KV cache block, so requests with a different salt, or no salt, produce different hashes for identical prompt text and cannot share cached prefixes. Requests with the same salt still share cache entries, which is the intended behavior within one tenant. Isolation therefore depends on the salt staying secret and unique per tenant: an attacker who knows a victim's salt can still probe that victim's cache. Confirm with the vLLM release you run, through its documentation and your own tests, that the salt applies to every endpoint that uses the prefix cache.

Use random secret salts, such as 256-bit values generated by a cryptographically secure random source. The vLLM security docs advise this, and user names are a poor choice because they are predictable and often visible to other users or appear in logs. Generate one salt per tenant unless your policy requires a separate cache for each user. Keep salts in a secret manager and out of application code and version control, and limit read access to the gateway service. Rotating a salt invalidates that tenant's cached prefixes, so schedule rotation and expect a temporary rise in time to first token.

Map each authenticated tenant to its salt on the server side at the gateway. The client should never choose or send the salt, because a client-supplied value lets one tenant copy another tenant's salt. Strip any cache_salt field that callers send, then inject the tenant's stored value after authentication succeeds. Log changes to the mapping table and test the design: send the same prompt from two tenants and confirm the second shows no cache-hit timing advantage, then confirm that changing headers or body fields cannot reach another tenant's salt. Run these checks in your own environment, with your gateway, vLLM version and traffic pattern, because timing results vary with hardware and load.

Setting up tenant isolation and cache salts in order

This sequence takes a shared vLLM server from one undifferentiated endpoint to one where each tenant has a fenced entry point, a secret cache salt and its own limits. Each step ends with a check you can run and record before the next step begins.

Inventory tenants and data sensitivity
Start by listing every team or customer that will send requests to the server, the data each one submits and the prompts that could contain sensitive information. Record which tenants must never see each other's cached prefixes, which are allowed to share, and which models each tenant may use. Assign the owner who approves each tenant. The output is a written table with one row per tenant. You know this step is done when every tenant that will call the server appears in the table with a sensitivity level and an approved model list.
Choose the isolation model
Decide between one vLLM instance per tenant or one shared instance fronted by a gateway that separates tenants. Per-tenant instances give stronger process and GPU separation at higher hardware cost. A shared instance uses the cache salt and gateway controls for separation and needs a stricter review of the proxy and metrics exposure. Record the decision and the reason for it, including which tenants, if any, must stay on separate instances. Confirm with your security owner that the chosen model meets your internal policy for the tenant sensitivity levels from the previous step.
Place a reverse proxy in front of vLLM
Deploy a reverse proxy between clients and vLLM and configure it as an allowlist. Pass only the user-facing inference paths, such as the OpenAI-compatible completions, chat and embeddings endpoints your tenants use, and deny everything else by default. Do not rely on the vLLM --api-key option alone, because the vLLM security documentation notes it does not protect every route. Check it from outside the network: request an allowed path and confirm it succeeds, then request an administrative path such as /invocations, /pause, /resume or /abort_requests and confirm the proxy returns a denial.
Map each tenant to a secret cache salt
Generate a random secret cache_salt for each tenant and store it in your secret manager, not in code, configuration files or tenant-visible fields. The vLLM security documentation advises random secret values such as 256-bit strings rather than user names or other guessable identifiers. Have the gateway attach the correct salt to each tenant's requests so clients cannot choose or read it. To verify, send two identical prompts with the same salt and observe a cache hit, then send the same prompt with a different salt and confirm the second request does not reuse the first request's cached prefix, for example by comparing time to first token and cache usage metrics.
Add per-tenant keys and limits at the gateway
Configure a distinct API key for each tenant at the gateway, bind each key to its approved model list, and set request and token rate limits and budgets sized to the tenant's agreed use. If you use LiteLLM or another gateway, confirm that its key store and rate limit state are backed by a database that survives restarts and is shared across gateway instances. Test each key by calling a model outside its list and confirming the call is refused, then exceeding the rate limit in a controlled test and confirming throttling. Keep the cache salt server-side so a tenant key never exposes it.
Restrict metrics and admin endpoints to internal networks
The vLLM /metrics endpoint exposes request counts, throughput, latency and KV cache use, and it is not protected by the API key option. Limit access to it, and to health and admin routes, to your monitoring and operations network segments only. Tenants should never reach these paths. From an external test host, request /metrics and the administrative endpoints and confirm each returns a denial at the proxy or network layer. From an approved monitoring host, confirm the Prometheus scrape still succeeds. Record both results as the sign-off evidence for this step.
Verify isolation before tenants go live
Run a final test pass across all tenants before any production traffic is allowed. Confirm that each tenant key reaches only its approved models and paths, that cross-tenant prompts produce no cache hits, that the salts are retrieved from the secret manager at runtime, and that blocked routes stay blocked after a configuration reload. Store the results with the configuration version so a later change can be compared against the baseline. Repeat the cache test after each vLLM version upgrade, because caching behavior is part of what you are validating.

Security controls around the multi-tenant vLLM server

A shared vLLM server has several exposure points beyond the inference path itself: the endpoints it serves, the metrics it publishes, the traffic between its nodes, and the salts that separate one tenant's cache from another's. The cards below cover the controls a security team should review, who owns each one, and what to verify before tenants share the server.

Endpoint allowlisting at the proxy

The vLLM --api-key option does not cover every route, including /invocations, /pause, /resume, /abort_requests, /metrics and /health, and the vLLM docs warn against relying on it alone. Place a reverse proxy in front of the server that forwards only the endpoints meant for tenants, such as /v1/chat/completions and /v1/completions. Ask the platform team to confirm in testing that administrative routes return errors from outside the cluster.

Cache salt generation and rotation

The cache_salt parameter separates prefix cache entries between tenants, but only if the salt is secret and unguessable. The vLLM docs advise random secret values such as 256-bit strings rather than user names or other guessable identifiers. Security engineering should own generation and storage in a secret manager, and the platform team should agree on a rotation plan that accounts for the cache entries that expire when a salt changes.

Protection of the metrics endpoint

The /metrics endpoint publishes Prometheus data on request counts, token throughput, latency histograms and KV cache use. Those values can reveal tenant activity and timing patterns that support the prefix cache timing side channel (CVE-2025-46570). Restrict /metrics to the monitoring network or the scrape job's identity at the proxy, and confirm that dashboards shared with tenants do not expose per-tenant series.

Inter-node traffic and multi-node authorization

The vLLM security docs state that communication between nodes in a multi-node deployment is insecure by default, with no authorization or encryption. Place the nodes on an isolated network that tenants and the proxy cannot reach, and confirm that Ray or multiprocessing traffic stays inside it. Where the isolated network is not enough, the infrastructure team should plan encryption and authentication for that traffic, which vLLM itself does not provide in that mode.

Audit logging at the gateway

vLLM does not record which tenant sent a request, so attribution depends on the layer in front of it. The gateway or proxy should log the caller identity, the endpoint, the model, the timestamp and the response status for each request, and store those logs outside the inference host. Security operations should own retention and review, and should test that a salt or key change can be traced to the change record.

Operating tenant isolation after go-live

Tenant isolation on vLLM is a set of recurring tasks, and each one needs a named owner before go-live. The tasks are reviewing each release, checking the proxy allowlist, monitoring per-tenant behavior through metrics, rotating salts and keys, and retesting the isolation controls after every change. A single engineer can hold several of these roles, but the register should show one accountable name for each task. Without that, the controls tend to drift as the platform changes around them.

The most frequent task is release review. vLLM ships new minor releases about every two weeks, for example v0.31.0 on October 5, 2026, so the owner of this task should expect a steady stream of changes. For each release, read the release notes and the security documentation, then look for new endpoints, changed authentication behavior, altered defaults and any change to how prefix caching or cache_salt is handled. Confirm the new endpoints against the reverse proxy allowlist before the version reaches production. Any endpoint that is not on the list should stay unreachable from tenant traffic until someone has reviewed it.

Monitoring is the second task. vLLM exposes Prometheus-format metrics at /metrics, covering request counts, token throughput, latency histograms and KV cache use. The monitoring owner should build dashboards and alerts that compare each tenant's traffic against its own baseline, since a sudden change in cache use or latency for one tenant can signal probing. Confirm which labels your vLLM version attaches to these metrics, because tenant identity may need to come from gateway logs instead. Keep /metrics off the user-facing path, and scrape it only from the internal monitoring network.

Rotation is the third task. Each tenant salt should be a random secret, such as a 256-bit value, held in the organization's secret manager and not in code or shared configuration. Changing a salt means the tenant's previously cached prefixes no longer match, so expect a cold cache and a temporary rise in time to first token for that tenant. The security owner should schedule rotations, record the date of each change and coordinate them with the tenant. API keys and gateway credentials follow the same pattern, with their own owner and log of changes.

The fourth task is retesting after each change. Whenever the vLLM version, proxy configuration, gateway policy or salt scheme changes, the security owner should rerun the isolation checks. These include confirming that a request carrying another tenant's salt does not show a timing pattern that reveals cached prefixes, and confirming that unauthenticated requests to /metrics, /pause, /resume and /abort_requests are refused at the proxy. Record the results and keep them with the change record. DeployOpen can prepare these tests and the supporting evidence, but independent auditors assess controls and issue any reports.

Limits to confirm in your environment before tenants share a server

Shared vLLM serving only holds up if the controls around it match the versions and features you actually run. The items below are the assumptions most often carried over unchecked from a design document into production. The right answers depend on your data classification and regulatory scope, so confirm each one against your own requirements.

cache_salt behavior in your vLLM version

Confirm that the vLLM version you will run accepts the cache_salt parameter and that the CVE-2025-46570 mitigation is present in that release. Read the security documentation for the exact build, not an earlier design note. Test that two tenants sending identical prompts with different salts do not share cached prefixes, and that a request with a missing or reused salt behaves as your policy requires. Record the version and the test result in your change record before any tenant is onboarded.

Per-tenant salt generation and storage

Decide where each tenant's secret salt is generated, stored and rotated, and who can read it. The vLLM security guidance favors random secret values, such as 256-bit values, over user names or other guessable identifiers. Confirm that the gateway or application layer injects the salt server-side so that tenants cannot choose or read another tenant's value. Ask who owns rotation and what happens to cached prefixes when a salt changes.

Gateway coverage of every inference path

Verify that the gateway's per-tenant budgets, token limits and request limits apply to every route a tenant can reach, not only the chat completions path. The vLLM server exposes several OpenAI-compatible endpoints, including completions, responses, embeddings and audio. Some vLLM operational endpoints are not protected by the --api-key option, so the proxy must allowlist only the routes meant for users. Test each route with a tenant key and confirm that the limit is enforced.

Enterprise features needed and licensed

List the gateway features your controls depend on, such as SSO beyond the free user limit, audit logs, role-based access control, IP allowlisting and secret manager integrations. Confirm which of these sit in the enterprise license terms and which are open source, and read the license file for the components you deploy. Ask the vendor to confirm the license scope in writing for your user count and deployment model before you design around a feature.

Shared GPU memory and scheduling latency

Measure whether tenants sharing one accelerator meet their latency expectations under realistic concurrency. Long prompts from one tenant can occupy the KV cache and delay requests from others, and scheduling behavior depends on the batch mix. Run load tests that mirror each tenant's prompt lengths and request rates, and record time to first token and token throughput. If a tenant's objective cannot be met on shared capacity, plan a dedicated instance for that tenant.

Metrics and logs that separate tenants

Confirm that the metrics and logs you use for billing, capacity and incident response can attribute activity to a tenant. The vLLM server exposes Prometheus metrics at its metrics endpoint, but those metrics describe the server as a whole, so per-tenant attribution usually comes from the gateway. Check that the gateway logs record tenant identity for each request, that log access is restricted, and that retention meets your policy.

Questions buyers ask about vLLM tenant isolation

Does vLLM have a license fee for multi-tenant use?

No. vLLM is Apache-2.0 licensed and has no paid edition, so the cost comes from the accelerators it runs on, the surrounding infrastructure, implementation work and the people who operate it. Gateway costs depend on the product you choose. If you add LiteLLM, its core is MIT licensed, but code under its enterprise/ directory uses a separate license. Its admin UI SSO is free for up to 5 users from v1.76.0, and an Enterprise license is needed beyond that. Ask each quote to separate one-time items (design, build, testing, handover) from recurring items (accelerator capacity, hosting, licenses, support retainers, and staff time for salt management and upgrades).

How long does vLLM tenant isolation take and what drives it?

The duration depends on the scope you agree in writing, and no duration figure is given here. The main drivers are the number of tenants, the isolation model (one vLLM instance per tenant, or one shared instance behind a gateway with per-tenant cache_salt values), how deeply identity is integrated with your provider, and how thoroughly isolation is tested. Each added tenant, identity boundary and cross-tenant test case adds design and verification work. Ask for the work split into phases, with a milestone and sign-off for each, and confirm what is in and out of scope before work starts. Most engagements are fixed-scope, so the timeline is part of the written agreement.

What do we need to provide before vLLM isolation work starts?

Collect these five inputs first, because the design depends on them:

Tenant list: each team or customer, the models it uses, its expected request patterns, and whether it needs a dedicated instance.
Data classification: which tenant data is sensitive or regulated, and which prompts may share a cache.
Identity provider: the system, the protocols it supports, and how groups map to tenants.
Network layout: where vLLM, the gateway and clients sit, and which paths are allowed between them.
Accelerator inventory: GPU models, counts, memory and how nodes are grouped.

It also helps to share the vLLM version you run now and how it is deployed, such as bare metal, Docker or Kubernetes.

Can our internal team handle vLLM tenant isolation alone?

Yes, if the team can test isolation, operate the proxy and manage the salts. Testing means confirming that one tenant cannot reach another tenant's endpoints, that metrics and health routes are not exposed to users, and that cached prefixes stay within their salt. Operating the proxy means allowlisting only the user-facing endpoints, because the --api-key option does not cover /invocations, /pause, /resume, /abort_requests, /metrics or /health. Salt management means generating random secret values and storing them in a secret manager. The team also needs a routine for vLLM upgrades, since the project ships new minor releases about every two weeks. Outside help is most useful for the first design and an independent test pass.

What should a vLLM tenant isolation quote contain?

A quote should follow from a written scope, and DeployOpen does not publish a price list, so the quote is built from that scope. Ask for the following:

The isolation model and the number of tenants covered.
The components in scope, such as vLLM instances, any gateway, the endpoint allowlist, salt handling, metrics access and identity integration.
The testing performed, including cache timing and endpoint reachability tests, and the evidence you receive.
The deliverables, such as design documents, runbooks and handover sessions.
Milestones and acceptance criteria.
The client inputs listed above, and what happens if they arrive late.
One-time and recurring items shown separately.
Exclusions, such as model work or accelerator procurement.

Will DeployOpen issue an audit report for our vLLM setup?

No. DeployOpen does not issue audit reports, attestations or certifications. It can design and build isolation controls, prepare configuration and test evidence, and document how the setup works. An independent auditor or certification body then assesses it and decides what satisfies a given framework. Before work starts, ask your auditor which evidence they expect, such as configuration records, test results, access lists and change history, and put that evidence in the scope. Mapping vLLM controls to specific regulatory requirements is a separate scope item, and the mapping should be checked against the requirement text your assessor uses.

Does cache_salt stop prefix cache timing attacks in vLLM?

It reduces the risk by limiting prefix cache sharing to requests with the same salt, which addresses the timing side channel tracked as CVE-2025-46570. It does not replace the other controls. The vLLM security documentation advises random secret salts, such as 256-bit values, because a guessable value such as a user name lets an attacker target a tenant's cache. Store salts in a secret manager, set them at the gateway so clients cannot choose or read other tenants' values, and test that a request carrying another tenant's salt gets no cache hit. Repeat these tests after each vLLM upgrade, because cache behavior can change between releases.

How DeployOpen scopes, builds and tests vLLM tenant isolation

DeployOpen treats vLLM multi-tenant isolation as an architecture decision first, then a configuration task. We start by deciding whether each tenant gets its own vLLM instance or shares a gateway, and then set per-tenant cache_salt values against the prefix cache timing side channel. Every step runs inside infrastructure you control.

How we scope

Written scope names the tenants, models, endpoints and accelerator pools covered by the isolation design.
We record whether each tenant gets a dedicated instance or shares a gateway, with the reasons documented.
Scope states how cache_salt values are generated, stored, rotated and assigned to each tenant's requests.

How we build

We configure per-tenant cache_salt handling using random secret values, not user names, as the vLLM docs advise.
The reverse proxy is set to allowlist only user-facing endpoints, because --api-key alone does not cover every route.
Internal node traffic is placed on an isolated network, since vLLM's multi-node communication is insecure by default.

How we hand over

Handover includes a written isolation design, configuration files and a record of the tenant-to-salt mapping process.
Test evidence covers cross-tenant timing checks on time to first token and confirmation that restricted routes reject tenant traffic.
Runbooks cover salt rotation, adding a tenant, and reviewing metrics at /metrics for unexpected cache or latency patterns.

How we support

Ongoing support covers reviewing vLLM releases and retesting isolation controls before any version upgrade.
Continuous testing and periodic re-verification of tenant boundaries are available as a retainer.
Before engagement, confirm with your team which compliance controls the isolation evidence must support.