vLLM air-gapped deployment

Start with the model weights, because vLLM downloads models from Hugging Face by default and an isolated host has no route to fetch them. Every model you serve has to be obtained outside the boundary, checked, and moved in through an approved transfer process, under the same change control you would apply to any other software entering the environment. Decide who may request a model, who approves it, and where the approved copy is kept. The vLLM Docker guide mounts the Hugging Face cache at /root/.cache/huggingface, so plan the staged location to match the path your serving process reads from. Confirm the exact file set with the model's publisher, including weight shards, configuration files, tokenizer files and any chat template the model needs. The chat completions endpoint requires a chat template, either built into the model or supplied with --chat-template, so check that requirement in the vLLM documentation for your version before the transfer starts.

Container images and Python packages form a second transfer stream that is easy to overlook. The vLLM server runs from a container image or a Python environment, and both resolve dependencies from public registries and package indexes unless you change where they come from. Mirror the image into an internal registry and place the wheels in an internal package index or a local wheelhouse. Record the image digest, package versions and hashes so the set that arrives inside matches the set you approved outside. Pin to a release your team has tested. vLLM ships new minor releases about every two weeks, for example v0.31.0 on October 5, 2026, so an unpinned pull can change the software between two transfers. Settle who may add packages to the approved list and whether any dependency resolution is ever allowed inside the boundary.

Verification is the control that makes the transfer defensible. NERC CIP-013-2 requires electric utilities to address, in their supply chain procurement processes, how they verify the software integrity and authenticity of vendor software and patches. If you are not an electric utility, map the same idea to the framework your auditors use. In practice, verify the publisher's checksums or signatures for each artifact, scan the container image again on the inside, and compare hashes after the artifacts cross the boundary. Treat the vLLM cache with particular care. vLLM loads cache contents without integrity checks and warns against copying cache contents from untrusted sources, so do not drop cache directories from outside into the directory set by VLLM_CACHE_ROOT. Move artifacts into their verified locations only after they pass these checks.

Turn each transfer into a record you can hand to an auditor. A transfer manifest should list every artifact with its name, version, source, hash, approver, date and destination path. Then prove the staged set works before go-live. Start vLLM on the isolated network with outbound traffic blocked, confirm that the model loads from local paths, and confirm that the chat endpoint answers. A network error at that point usually means a file is missing from the manifest. Plan the process to repeat, since releases arrive every two weeks and each new version or model may bring different files. Ask any implementation partner to write the transfer procedure, verification steps and manifest template into the agreed scope, so your team owns the process after handover.

Treat vLLM's cache directories as trusted local state that never crosses a trust boundary. The vLLM security documentation assumes that cache directories rooted at VLLM_CACHE_ROOT are private and trusted, because cache contents load without integrity checks. A tampered cache file is therefore loaded as if vLLM had produced it. The default root is under the home directory of the account running the server, at ~/.cache/vllm. In a closed network, the practical rule is that you do not copy cache contents from a laptop, a shared drive or another team's host, even when the source sits inside your boundary. Move model weights and container images through your staging process, and let the target host build its own cache.

Decide where the cache lives before the first start. Set VLLM_CACHE_ROOT to a dedicated path on a volume owned by the service account, and restrict write access to that account. Remove group and world write permissions from the directory tree. If the server runs in a container, mount the cache on a volume that no other container shares unless that container runs inside the same trust boundary. The Docker deployment guide mounts the Hugging Face cache at /root/.cache/huggingface, which is a separate store with its own location. It needs the same ownership and access rules, and it should appear in your runbook alongside the vLLM cache path, since both are easy to overlook during an audit.

Confirm how to disable usage statistics and any other outbound telemetry for the exact vLLM version you deploy. The configuration switch can change between releases, and this topic does not verify one for every version, so check the installation and configuration documentation and the release notes for your version. Also list the other outbound paths the server might use, such as model downloads, package index access during image builds, and any dependency that reports usage. Record each disabled setting in your configuration baseline and add it to the review for every upgrade, so that a new release cannot silently reintroduce an external call.

Test the deployment with the network blocked to find outbound attempts you did not expect. Run the server in a test environment with egress denied at the network layer, load the model from staged files, and send representative requests to each endpoint you plan to expose. Application logs may not show silent connection failures, so also review firewall deny logs and DNS query logs during the run. A clean test is evidence for the configuration you tested, not proof for every code path. Repeat the test after each vLLM upgrade and after any change to the model, image or cache location.

The first thing to settle is whether your vLLM deployment spans more than one node at all. The vLLM security documentation states that communication between nodes in a multi-node deployment is insecure by default, with no authorization and no encryption, and that the nodes must be protected by placing them on an isolated network. Inside an air-gapped boundary, that isolation is the starting point and still needs a design decision behind it. Treat every node-to-node path as an internal interface that needs an owner, a documented purpose, and a list of ports and hosts allowed to reach it. If your topology does not need multiple nodes, keeping the model on one host removes this entire class of exposure and simplifies your evidence.

Parallelism choice determines what crosses the wire. vLLM uses tensor parallelism within a node and pipeline parallelism across nodes, so inter-node traffic is shaped by the pipeline stages and the data they pass between hosts. For multi-node runs, vLLM relies on either Ray or its own multiprocessing backend. Each option brings different process management, dependencies and network requirements, and the chosen backend determines which services must be reachable between hosts. Confirm with the vLLM parallelism and scaling documentation which backend and transport fit your topology, and whether data parallel deployment applies to your mixture-of-experts model. Record that decision before writing firewall rules, because the rules follow from it.

Encryption is where the default falls short. The documentation describes inter-node traffic as unencrypted by default, so the isolated network alone may not satisfy your control set. Where policy requires encryption of traffic between nodes, plan an external layer that you can validate and operate, such as a FIPS-validated transport layer if your hosts run in FIPS mode. Ask the platform team to confirm which layer terminates the connection on each side, who manages its certificates or keys, and how the layer is retested after upgrades. Do not assume that vLLM's own configuration covers this requirement, and record the answer in the design file before the first multi-node test.

Verification and operations complete the section. Test the isolation rather than assuming it: from each node, attempt connections to every other node on the ports your backend uses, and confirm that nothing else is reachable. Capture the results with the configuration baseline so an assessor can see the boundary as built. Monitor for new listeners after upgrades, since backend and port behavior can change between minor releases. Assign an owner for the inter-node path, and include it in change review whenever the vLLM version, the parallel layout or the network segmentation changes.