Why vLLM security hardening starts at the network, not the API key
Hardening a vLLM deployment starts with the network layer, not with the API key. In practice that means isolating inter-node traffic on a private network, placing a reverse proxy in front of the API server that forwards only the endpoints meant for users, restricting which hosts and ports can reach the server at all, and deciding deliberately where model files come from. An API key is one control among these, and the vLLM project does not present it as a complete answer. The vLLM security documentation states that communication between nodes in a multi-node deployment is insecure by default, with no authorization and no encryption, and that it must be protected by placing the nodes on an isolated network. This matters for any team that spreads a large model across machines for pipeline parallelism or uses Ray or the built-in multiprocessing backend. The inter-node channel is a separate exposure from the public API, and it needs its own network boundary. The --api-key option is narrower than many teams assume. According to the vLLM security docs, it protects endpoints under /v1, /v2, /inference and /cohere, but not /invocations, /pause, /resume, /abort_requests, /metrics or /health. The docs warn against relying on the key alone and recommend a reverse proxy that allowlists only the endpoints intended for users. Before going live, the team should list every route the running server answers, mark each as user-facing or operational, and confirm which of them the key actually covers in the vLLM version they run. The work is harder than it looks for three reasons. The deployment has several ports: the API port, inter-node communication, metrics and health checks. It has several roles: end users, applications, monitoring systems, platform operators and, in multi-node setups, the nodes themselves. And the project ships new minor releases about every two weeks, so a hardened configuration needs a repeatable patch process, or it will drift. Each of these questions should be answered in writing before the first production request.
Which vLLM components and settings carry the risk
A vLLM deployment exposes more than its inference API. The server, its authentication option, metrics, multi-node traffic, documentation routes and model files each widen the attack surface in different ways, so a hardening plan needs to account for every one of them. The cards below list what each component exposes and what your team should check before users reach it.
vllm serve API server and OpenAI endpoints
The vllm serve process exposes OpenAI-compatible routes, including completions, chat completions, responses, embeddings and audio transcription and translation. Applications written for the OpenAI API can call it directly, which also means any client that can reach the port can run inference on your accelerators. Check which routes are reachable from each network segment, and confirm that only the endpoints your users need are published through the reverse proxy.
The --api-key option and its coverage gaps
The --api-key option protects the /v1, /v2, /inference and /cohere paths, but it does not cover /invocations, /pause, /resume, /abort_requests, /metrics or /health. The vLLM security documentation warns against relying on this option alone. Confirm the exact routes your version protects by testing each path without a key, and plan authentication at the proxy layer for the rest.
The /metrics endpoint
The /metrics endpoint publishes Prometheus-format data on request counts, token throughput, latency histograms and KV cache use, and it is not covered by --api-key. Those figures can reveal traffic patterns, model usage and capacity limits to anyone who can reach the port. Block it at the reverse proxy or firewall, and allow only the Prometheus server that scrapes it.
Inter-node channel in multi-node runs
In multi-node deployments, the vLLM security documentation states that node-to-node communication is insecure by default, with no authorization or encryption. This channel carries work between Ray or the multiprocessing backend across machines. Place all nodes on an isolated network segment, restrict the relevant ports with host and network firewall rules, and verify from outside that segment that these ports are unreachable.
The --enable-offline-docs option
The --enable-offline-docs option serves API documentation without depending on external content delivery, which suits air-gapped environments. Whether the option is set or not, the documentation routes are part of the HTTP surface and should be reviewed with the other endpoints. Decide whether the documentation is needed in production, and if it is not, confirm that the route is blocked at the proxy.
Model and chat template files
Models and chat templates can be loaded from local disk or pulled from a model hub, and a chat template changes how prompts are rendered for every request. Treat these files as code: record their source and revision, verify checksums where the hub provides them, and restrict write access to the model directory. Confirm which templates your models require, and review any template passed with --chat-template before it reaches production.
Which hardening option fits your deployment and how much it costs to run
This table compares the main hardening choices for a vLLM deployment, from the reverse proxy and gateway to metrics exposure, multi-node backend, offline docs and model pinning. Read each row by asking which option matches your network layout and which ongoing work your team would take on. The main takeaway is that most choices trade a small one-time configuration effort for recurring review work, and the cheapest option is not always the one your segmentation requires.
| Option | When it fits | Operational cost |
|---|---|---|
| Nginx or similar reverse proxy with an endpoint allowlist | Teams without an API gateway that need to expose only user-facing /v1 routes, with /invocations, /pause, /resume, /abort_requests, /metrics and /health blocked at the proxy | Low one-time configuration. Recurring work is reviewing the allowlist after each vLLM minor release, since new routes may appear |
| Existing API gateway enforcing the same allowlist | Organizations that already run a gateway with centralized logging, rate limits and access policy, so vLLM routes are added to a control plane they already operate | Little new infrastructure. Cost sits in gateway rule maintenance and in confirming that the gateway, not vLLM, is the only path to the server |
| Keep /metrics on an internal scrape network | Environments where Prometheus or another collector runs on a segment that application clients cannot reach, so metrics stay off user-facing paths | Low. Requires network policy or firewall rules and a scrape path that your monitoring team already runs |
| Proxy /metrics behind authentication | Teams whose collector sits outside the serving network, or who must scrape from a separate account or cluster and cannot use a private segment | Moderate. Adds credential management for the scraper, proxy configuration and a rotation process for the scrape identity |
| Ray backend for multi-node runs | Multi-node deployments that already operate Ray and can place all Ray nodes on an isolated network, since inter-node traffic is unauthenticated and unencrypted by default | Higher. Adds a Ray cluster to operate, its own upgrade path and segmentation controls for its ports, plus staff familiar with Ray |
| Multiprocessing backend for multi-node runs | Multi-node deployments that want fewer moving parts and can confine all node-to-node traffic to a small, tightly firewalled segment | Lower operating overhead, but segmentation work remains mandatory because node communication is unauthenticated by default. Confirm current backend behavior for your release |
| Keep offline docs enabled | Internal deployments where engineers need the local API reference during development or troubleshooting, and the server is not reachable from untrusted networks | Negligible runtime cost. Confirm whether docs routes are exposed through your proxy allowlist |
| Disable offline docs (--enable-offline-docs off) | Air-gapped or production environments that want the smallest exposed surface, with API references kept in internal documentation instead | Negligible. Requires a documented alternative for engineers who relied on the local reference |
| Pin model files to reviewed revisions | Regulated deployments that must show which model weights were reviewed and approved, with changes going through a documented review and rollout | Moderate recurring cost. Each new model revision needs review, a test run and a change record, and staff time grows with model count |
| Pull the latest model revision at startup | Teams that prioritize fast access to newer model revisions and accept that weights can change without a new review | Low upfront effort, but it weakens provenance evidence and makes incident reconstruction harder when behavior changes unexpectedly |
How to harden a vLLM deployment in sequence
This sequence takes a vLLM deployment from an unknown exposure surface to one where only intended users reach the OpenAI-compatible API, control and metrics endpoints stay internal, and model files and patches are traceable. Each step closes with a check you can run or observe before moving on.
Inventory every listening port and endpoint
Start by listing each node, each process that listens, and each HTTP path it serves, including the vLLM API server, any Ray or multiprocessing ports used for multi-node runs, and sidecars such as exporters. Confirm which paths are meant for users (under /v1) and which are operational (/metrics, /health, /pause, /resume, /abort_requests, /invocations). Done when you can produce a per-node table of bound addresses and ports, and each entry has an owner and a purpose. Run ss -tlnp or your platform equivalent on every node and compare the output against the table; any listener not in the table must be explained or stopped.
Isolate multi-node traffic on a private segment
The vLLM security documentation states that communication between nodes in a multi-node deployment is insecure by default, with no authorization or encryption, so nodes must sit on an isolated network. Your infrastructure or network team places the inter-node traffic on a dedicated subnet or VLAN with no route from user networks or the internet. Confirm the design with the network owner before cutover. Done when a test host on the user network cannot open a connection to any inter-node port, and a test from one GPU node to another on the private segment succeeds.
Write host firewall rules for proxy and peers only
On each node, write default-deny inbound rules and allow only the reverse proxy address to reach the vLLM API port, and only known peer addresses to reach the inter-node ports identified in step one. Keep management access through a separate, named path. Apply the rules through your configuration management so they survive restarts. Done when a port scan from an unapproved host shows every vLLM and inter-node port as filtered or closed, and from the proxy host the API port responds. Keep the rule file in version control so the next reviewer can diff it.
Allowlist user paths at the reverse proxy
Place a reverse proxy in front of the vLLM API server and configure it to forward only the /v1 paths your applications use, such as chat completions, completions, responses, embeddings, and any audio endpoints you run. Explicitly deny /metrics, /health, /pause, /resume, /abort_requests and /invocations to clients outside the operations network, and deny by default for any path not on the list. Health checks for your load balancer should come from an internal address. Done when a request to each denied path from an external client returns a refusal at the proxy and never reaches vLLM, and an allowed chat completion request succeeds. Check the proxy access log to confirm the denied requests were recorded.
Set an API key and store it in a secrets manager
Start vLLM with the --api-key option as a second layer behind the proxy. The vLLM documentation says this key protects /v1, /v2, /inference and /cohere but not the operational endpoints listed above, and warns against relying on it alone, so the proxy allowlist stays in place. Generate the key, store it in your secrets manager, and inject it at process start rather than writing it into unit files or images. Done when a request to an allowed /v1 path without the key is rejected, the same request with the key succeeds, and a search of your configuration repositories and container images finds no plaintext copy of the key.
Confirm model provenance and record checksums
Identify where each model file comes from, and use only a source your organization controls, such as an internal artifact store that mirrors approved releases. Record the revision identifier and a checksum for every weight, tokenizer and configuration file, and store the record alongside the deployment manifest. Confirm the licence terms with your legal team before loading. Done when recomputing checksums on the serving nodes matches the recorded values, and the deployment manifest names the exact model revision that each running server loaded.
Decide the offline docs and interactive surface policy
Decide whether the server should expose its interactive API documentation. vLLM provides an option to enable offline docs for air-gapped environments, and the choice should follow your written policy rather than defaults. Record the decision in the deployment standard and make the configuration match it. Done when a request to the documentation path returns the result your policy requires from both an internal and an external client, and the server start command in the deployment manifest shows the setting explicitly.
Tie patching to the vLLM release cadence
vLLM publishes new minor releases roughly every two weeks, so a patch process needs a fixed owner, a review window and a rollback path. Subscribe the owning team to the vLLM GitHub releases, pin the version in the deployment manifest, test each candidate in a staging environment with the same proxy, firewall and key settings, and roll out only after the checks above pass again. Done when the pinned version, staging test record and last rollout date are stored together, and a dry run of the process on the latest release has been completed and signed off by the owner.
What drives hardening effort and how to keep it down
This table lists the factors that change how much work a vLLM hardening project takes, how each one moves effort, and the design choice that keeps it contained. Read each row against your own topology, then use the reduction column to decide what to standardize before scoping the work.
| Driver | Effect on effort | How to keep it down |
|---|---|---|
| Nodes and multi-node backend (Ray or multiprocessing) | Multi-node runs send inter-node traffic that is insecure by default, so every node pair needs isolation and each backend opens its own internal ports. Ray adds more ports and services than the built-in multiprocessing backend. | Place all vLLM nodes on one dedicated isolated network segment with a shared firewall template. Confirm the ports your chosen backend uses before writing rules. |
| Number of models | Each model adds chat template checks, endpoint tests and access rules. Models with different licenses or owners need separate review. | Standardize one serving configuration pattern and one test suite that every model runs through. |
| Model sources per model | Each source (hub download, internal registry, fine-tuned artifact) needs its own provenance and integrity checks, and each one adds a trust decision. | Keep one model provenance record per model that lists every source, checksum and approval, and restrict downloads to approved sources. |
| Environments (dev, staging, production) | Each environment needs its own firewall, proxy, key and evidence set, and drift between them creates findings at assessment time. | Build all environments from the same infrastructure templates and proxy configuration, changing only values such as endpoints and keys. |
| Kubernetes or VM deployment | On Kubernetes, network policy and service exposure live in cluster objects. On VMs, they live in host firewalls and the network. The control points differ, so the review scope differs. | Choose one deployment model for the hardening baseline, and document where network policy is enforced for each cluster or host group. Confirm which Kubernetes options your team runs before scoping. |
| User-facing applications routed through the proxy | Each application needs its own allowlisted paths, authentication decision and rate limits. Applications that call metrics or control endpoints add exceptions to review. | Use one reusable reverse proxy configuration with a common endpoint allowlist, and add applications by identity and route rather than by copying configuration. |
| Air-gapped constraints | Without internet access, some external checks such as hosted scanners, online key validation and remote documentation cannot run, so teams need offline substitutes. vLLM offers an offline docs option for these environments. | Decide early which checks run offline, such as local image scanning and internal package mirrors, and record the substitutes in the test plan. |
| Compliance evidence requirements | The required evidence (configuration exports, access logs, provenance records, patch history) defines how much must be collected and retained. Each control framework asks for different records. | Generate evidence from the same configuration and logging pipeline used in operations, and map each record to the control it supports. Ask your assessor which evidence formats they accept. |
How hardening differs for single node, multi-node, Kubernetes and air-gapped vLLM
On one server, vLLM serves its API, metrics and control endpoints from the same host, so the main job is deciding what can reach the process at all. The --api-key option is only a partial control. It protects the /v1, /v2, /inference and /cohere paths but not /invocations, /pause, /resume, /abort_requests, /metrics or /health. The vLLM security documentation recommends a reverse proxy that allowlists only the endpoints meant for users.
- Reverse proxy allowlist: Expose only the OpenAI-compatible paths your applications call, and deny everything else, including /metrics and the pause and abort controls.
- Host firewall: Restrict the serving port so that only the proxy host or the application subnet can reach it, and block direct access from other hosts.
- Bind address: Confirm the server listens only on the interface the proxy uses, then verify with a scan from outside the host.
- Metrics access: Scrape /metrics from the monitoring host over a private path that the public proxy never forwards.
Check that from a client outside the approved network, each blocked path returns a denial and the approved inference paths still respond.
Pipeline parallelism across nodes means worker machines exchange activations and coordination traffic with each other. The vLLM security documentation states that communication between nodes is insecure by default, with no authorization or encryption, and that it must be protected by placing the nodes on an isolated network. That isolation belongs in the baseline design, and the single-node proxy and firewall controls still apply to the entry node.
- Dedicated inter-node network: Place the nodes on a segment that carries only cluster traffic, with no route from user networks or the internet.
- Firewall between segments: Allow only the traffic the cluster framework needs between nodes, and deny the rest by default.
- Backend confirmation: Confirm whether your version uses Ray or the built-in multiprocessing backend, because the processes and ports you must allow depend on that choice.
- Single entry point: Serve the API from one entry node behind the same proxy allowlist used on a single host.
- Node inventory: Record which hosts belong to the cluster, since any unlisted host on the segment joins a trust zone with no authorization.
Check by scanning the inter-node segment from a host outside the cluster and confirming that no cluster port answers.
On Kubernetes, the host firewall is replaced by network policies and service exposure. The vLLM Kubernetes documentation lists several deployment options at a high level, including native manifests, Helm, the vLLM production-stack, AIBrix, KServe, KubeRay, LeaderWorkerSet, llm-d and NVIDIA Dynamo. The option you choose determines which controllers, services and pods exist, so hardening starts with an inventory of what each one exposes.
- Service type: Keep vLLM services cluster-internal, and place an authenticating gateway or ingress in front of them.
- Network policies: Limit ingress to the proxy and monitoring workloads, and limit pod-to-pod traffic for multi-node groups to the workload's own pods.
- Metrics and control paths: Keep /metrics and the control endpoints out of ingress rules that serve users.
- Namespace separation: Run model-serving workloads in their own namespace under a default-deny policy.
Confirm the policy syntax and enforcement behavior against your cluster's network plugin and documentation, since both differ by platform and by the operator you run. Then test the policies from a pod in each allowed and denied namespace.
In an air-gapped network, vLLM cannot pull models, packages or container images from the internet during operation, so every input must arrive through an approved path. The controls shift from blocking outbound traffic to deciding where each artifact comes from, how it is verified and how it reaches the isolated network. Patch handling becomes an internal process rather than a download step.
- Internal model source: Host approved model files on an internal store, and record a checksum for each file at intake. Provenance is the first control.
- Offline docs decision: The --enable-offline-docs option exists for air-gapped environments. Decide whether the API documentation is served from inside the network, and test that it loads without internet access.
- Chat templates: Models need a chat template to serve chat completions. Stage the template with each model and do not depend on a download at startup.
- Internal patch channel: vLLM ships minor releases about every two weeks, so mirror approved images, packages and security advisories on a fixed internal schedule.
- Staged updates: Test each mirrored release in an internal staging environment before production.
Check that every model, template and container image has a documented origin, a recorded checksum and a named owner who approves updates.
Which endpoints and channels to block, allow and verify
A vLLM server exposes more than the routes users need, and its API key does not cover all of them. The cards below separate what to allow, what to block at a reverse proxy, how to isolate node-to-node traffic, and how to verify model files, secrets and denied requests.
Allowlist the user-facing /v1 routes
Expose only the OpenAI-compatible routes applications call, such as /v1/completions, /v1/chat/completions, /v1/responses and /v1/embeddings, plus audio transcription and translation only if your models serve them. If the proxy forwards everything, any client that reaches the port can call routes you never reviewed. Confirm the exact route list against the vLLM version you pin, because routes change between releases.
Block the unauthenticated control routes
The --api-key option does not protect /invocations, /pause, /resume, /abort_requests, /metrics or /health. Without a proxy rule, an unauthenticated client could pause or resume inference, abort other users' requests, or read operational data. Deny these paths at the reverse proxy for all external callers, and test each one from outside the cluster after every configuration change.
Isolate inter-node traffic
The vLLM security documentation states that communication between nodes in a multi-node deployment has no authorization or encryption by default. Place the nodes on a dedicated network segment with firewall rules that admit only cluster members, and keep the Ray or multiprocessing ports off user networks. Ask your platform team to confirm which ports the chosen backend opens before writing the rules.
Verify model file provenance and integrity
A model file loaded from an untrusted source can carry malicious code or altered weights, and a chat template can change model behavior. Record where each model came from, pin the revision, and compare checksums against the source before loading. Restrict which hosts the serving nodes may download from, and keep model storage read-only to the serving process where possible.
Handle the API key and proxy credentials
Store the vLLM API key and any proxy credentials in a secrets manager, not in container images, shell history or configuration files committed to source control. Rotate them on a schedule and after staff changes. Because the API key covers only some routes, confirm that the proxy authenticates the callers you intend, and that clients never receive the server key directly.
Log and review denied requests
Blocked requests show where clients or scanners are probing the server, and they are evidence during an incident. Configure the proxy to log denied paths, source addresses, timestamps and the identity used, and send those logs to your central store. Review denial patterns regularly, and confirm that logs do not capture prompts or API keys.
How to verify vLLM hardening works and fails safely
The most useful test of a vLLM hardening plan is an outside-in one: run the checks from a network position that a real user or attacker would occupy, then confirm that each control refuses what it should. Configuration review alone is not enough, because a proxy rule can look correct on paper and still leave an endpoint reachable through a path nobody tested. Start by listing every control you intended to add, then write one pass or fail check for each. The checks below are the minimum set for a vLLM server that sits behind a reverse proxy with an allowlist.
First, confirm that the allowed /v1 routes respond through the proxy. Send a request to /v1/completions, /v1/chat/completions, /v1/responses and /v1/embeddings using a client on a user network, and confirm each returns a normal response. Then confirm the routes that must be refused do not answer from user networks. Test /metrics, /pause, /resume, /abort_requests and /invocations directly and through the proxy, including with a valid API key. The vLLM security documentation notes that the --api-key option does not protect several of these paths, so a key-authenticated request to /pause should still fail at the proxy. Record the status code and body for each refusal, since a 200 response with an empty body is still a failure.
Second, confirm that inter-node traffic is unreachable from outside the isolated segment. For multi-node runs, the vLLM security documentation states that node-to-node communication is insecure by default, so the test is whether ports used by the cluster backend can be reached from a host that is not a cluster member. Run a port scan from the user network and from a jump host outside the segment, and compare the results with the port list your Ray or multiprocessing configuration actually uses. Your network team should confirm which ports those are, because the list can change with the backend and version you deploy.
Third, test failure behavior. Send requests with no API key, a wrong key and a malformed Authorization header, and confirm the proxy and server both refuse them without leaking configuration, stack traces or version details. Then stop the vLLM backend while the proxy is running, and stop the proxy while the backend is running. In each case the user should see a generic error from the proxy, and the raw server port should not become reachable as a fallback. Also confirm the proxy does not forward requests to the backend when its own health check fails. Record what you saw in each failure state, since this is where configuration drift most often appears.
Fourth, verify model provenance. Compare the checksum or repository revision of every model file on the serving host with the values recorded at approval time, and confirm the chat template matches the one reviewed. Repeat this after every model update and after every vLLM upgrade, because a new release can change how the server loads files. Finally, treat all of these checks as a regression suite. Run them after each upgrade, since vLLM publishes new minor releases about every two weeks, and any endpoint or default that changes can reopen a path. A DeployOpen security test or an independent assessor can run this suite, and you should ask either one to deliver the raw results, not only a summary.
What vLLM security hardening costs, takes and needs from your team
What drives the cost of vLLM security hardening?
The main cost is people and time, because vLLM has no license fee under its Apache-2.0 license. Expect cost in four areas: engineering hours to design the reverse proxy, network segmentation and firewall rules; testing time to confirm that blocked endpoints stay blocked and approved users still get through; infrastructure for a staging environment and any logging or monitoring tools; and ongoing patch work, since vLLM ships new minor releases about every two weeks. Accelerator hardware is a separate cost tied to the serving platform itself. Recurring costs are patch reviews, re-testing after upgrades and access list reviews. Ask for a quote that lists each component separately, names who performs it and states what you receive at handover.
Can our internal team harden vLLM without an outside partner?
Yes, if the team has network and reverse proxy skills and time set aside for testing. The core work is placing multi-node traffic on an isolated network, putting a proxy in front of the API that allows only user-facing endpoints, and writing firewall rules that keep control and metrics paths internal. The harder part is usually verification: proving that each blocked path refuses requests and each approved path still works after every change. Teams without proxy experience, or without spare capacity for that testing, often find a fixed-scope engagement easier to plan. Whichever route you choose, decide in advance who will own the proxy configuration after handover.
Why is the vLLM API key alone not enough protection?
The vLLM security documentation warns against relying on the --api-key option alone. That option protects the /v1, /v2, /inference and /cohere endpoints, but it does not protect /invocations, /pause, /resume, /abort_requests, /metrics or /health. An unauthenticated caller could pause or abort running work, or read operational metrics, without holding a key. The documentation recommends a reverse proxy that allowlists only the endpoints meant for users. Multi-node communication has a separate issue: the docs state it is insecure by default and must be isolated on a private network. Plan for both controls.
What do we need to provide before vLLM hardening begins?
Four inputs make the work faster and more predictable. First, network diagrams showing where each vLLM node, gateway, load balancer and client sits. Second, an endpoint inventory listing every vLLM route in use and which callers reach it. Third, a source list for each model file, including where it comes from and how it is downloaded. Fourth, a named patch owner who receives security notices and schedules updates. Access to a staging environment and to the people who approve firewall changes also matters. Missing diagrams or inventories add discovery time, and that time is paid for whichever party does the work. Confirm this list in writing before scoping begins.
How long does vLLM security hardening take to complete?
The duration depends on the scope, so ask for a schedule tied to each deliverable rather than a single estimate. The main drivers are the number of nodes and models, how many network zones need isolation, whether Kubernetes ingress must be reworked, how many endpoints need allowlist decisions, and how long your change approval process takes. Multi-node clusters, missing inventories and slow firewall approvals lengthen the work. A single node, an accurate network diagram, an existing authenticating gateway and a staging environment that matches production shorten it. Ask the partner which of these assumptions the timeline depends on, and what happens if one changes.
Does DeployOpen issue a security certification for vLLM?
No. DeployOpen does not issue audit reports, attestations or certifications. It prepares systems, controls and evidence, such as configuration records, endpoint allowlists, network documentation and test results, that your compliance team can present. Independent auditors or certification bodies assess that evidence and decide the outcome. Confirm early which framework your assessor uses and which evidence they expect, because their requirements determine what should be collected. Map the control IDs from your framework to the evidence list before work starts, so nothing needed for the assessment is missing at the end.
What does a fixed-scope vLLM hardening quote include?
A fixed-scope quote follows from a written scope. Most DeployOpen engagements are fixed-scope, meaning scope, deliverables and timelines are agreed in writing before work starts. The scope should list the vLLM nodes and endpoints covered, the proxy and firewall changes, the testing approach, the documentation handed over and the items you provide. DeployOpen does not publish a price list, so the quote comes from that written scope. Ask that the quote separate one-time work from recurring items such as patch reviews or retainer support, and that it state clearly what is excluded.
What to monitor, patch, back up and document after hardening
Hardening only holds if someone checks it after go-live, so the first decision is which signals the team will watch and who receives them. Four matter most for this deployment. Denied requests at the reverse proxy show whether clients are reaching paths outside the allowlist, or whether a misconfigured application keeps retrying blocked routes. Any successful request to /metrics from outside the monitoring network is a finding, because the --api-key option does not cover that endpoint and the proxy is the only control in front of it. Changes to firewall rules or proxy configuration should raise an alert linked to an approved change record, not just a log entry. Finally, watch for model file hash drift. Recompute checksums for weight, tokenizer and chat template files on a schedule and compare them with the provenance record. A mismatch means a file changed outside an approved process.
Patching needs a rule set agreed before the first advisory arrives. vLLM ships new minor releases about every two weeks: v0.28.0 on August 26, v0.29.0 on September 9, v0.30.0 on September 22 and v0.31.0 on October 5, 2026. A fixed monthly patch window would fall behind quickly. Decide in advance which classes of fix justify an out-of-cycle update, such as a vulnerability in an endpoint your proxy exposes, a flaw in authentication or request parsing, or a dependency fix that touches the serving path. Everything else can wait for the scheduled test cycle. Check the release notes for how each release labels security fixes, since those labels determine when an emergency change is triggered. Pin the version in deployment manifests so an automatic image pull cannot change the running server without a change record.
Back up three things separately from the serving hosts: proxy configuration, firewall rules and the model provenance record. Proxy configuration and firewall rules belong in version control or a configuration store with logged access, so the last approved state can be restored and compared against the live system. The provenance record lists each model artifact with its source, license, checksum and approval reference. It is the baseline for hash drift checks, and losing it means losing the ability to show what was served. Store copies where the serving cluster cannot modify them. Test a restore of each item at least once on a separate host, and record what was missing and how long recovery took. Confirm who owns each backup and where the keys used to encrypt them are held.
Four documents should exist and stay current: a network diagram showing the isolated node network and the proxy and firewall boundaries, an endpoint allowlist, a change log and test results from hardening checks and each upgrade. The allowlist should name every path the proxy forwards to users, such as the /v1 routes your applications call. It should also state that /metrics, /pause, /resume, /abort_requests and /invocations are not forwarded to users, and that internal health probes reach the server only from the cluster network. Update the diagram and allowlist in the same change as the configuration, not afterward. Auditors and internal reviewers usually ask for these first. DeployOpen prepares this evidence for compliance assessments, but independent auditors issue the assessment itself. Regression testing for each vLLM release is covered in the vLLM upgrades and support topic.
How DeployOpen scopes, hardens and hands over vLLM security work
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, including vLLM. For vLLM security hardening, the work starts with a written scope that names the network, endpoints, model files and patch process to be covered, then moves through build, testing and handover.
How we scope
- Map every vLLM node, API server and multi-node link, and confirm which ones sit on an isolated network.
- List the endpoints users need, and the control, metrics and health endpoints that must stay private.
- Agree in writing which model files, sources and provenance checks are in scope, and who approves them.
How we build
- Place the vLLM API behind a reverse proxy that allowlists only user-facing endpoints, with authentication enforced at the proxy.
- Apply firewall rules so inter-node traffic and the metrics endpoint are reachable only from approved hosts.
- Test each blocked path from outside the network to confirm that unauthenticated requests fail as designed.
How we hand over
- Deliver configuration files, firewall and proxy rules, and a record of the test results with their scope.
- Document the model provenance procedure, the version pinning approach and the steps for applying vLLM patches.
- Walk your operations and security staff through the runbook and the open items agreed in the written scope.
How we support
- Engagements are fixed-scope, so changes after handover are agreed in writing before work starts.
- Ongoing support, upgrades and continuous testing are available as a retainer, if your team wants them.
- Confirm with your own security and compliance leads which evidence they need, since DeployOpen does not issue audit reports.
