Why an air-gapped or government network needs an Azure OpenAI alternative
An agency, defense contractor, critical infrastructure operator or regulated body that cannot send prompts to a shared cloud service needs its language model workloads to stay inside its own boundary. The short answer is open-weight models served on vLLM inside that boundary, with the gateway, logging and evaluation tools chosen because they run offline. Azure OpenAI can be the right choice for some workloads, so the first task is to test its authorization, region and model availability against your own requirement, not to assume it fails.
The gap this page addresses is specific. Azure Government clouds do not offer Global Standard deployments, offer fewer models, and usually make one model version available at a time. For a team that must keep prompts in a disconnected or sovereign enclave, that combination changes how often applications must be revalidated and how much control the team has over which weights answer a request. Ask the cloud provider, in writing, which models, deployment types and versions are available in the specific government region and impact level you need, and which of them carry the authorization your system boundary depends on.
Self-hosting moves those decisions to your side of the boundary. You choose the weights, you decide when they change, and you can keep the same OpenAI-compatible interface that vLLM exposes so applications need fewer changes. The trade-off is evidence: your team, or your implementation partner, must document the model source, the transfer of weights into the enclave, the serving configuration and the update path for an assessor. Confirm early with your authorizing official or compliance lead which control baseline and evidence format they expect, because that determines how much of the work is document preparation rather than engineering.
What the government and air-gapped rules require from the AI stack
Government and critical infrastructure networks decide what an AI stack may hold, where it may run and how it is checked before anything reaches production. The cards below set out the requirements that most often shape a self-hosted vLLM design, and what your own team should confirm with the authorizing official before the design is fixed.
NIST SP 800-53 control baselines
SP 800-53B defines low, moderate and high baselines that select 149, 287 and 370 controls and enhancements respectively. Your system's categorization decides which baseline applies, and that baseline determines how the gateway, the vLLM hosts and the logging path must be documented. Ask your authorizing official which baseline and which release of SP 800-53 they will assess against, and map each component of the inference stack to the controls it inherits or must implement itself.
FedRAMP scope for AI services
FedRAMP covers cloud products and services that handle federal information on behalf of an agency. A system used only for one agency's operations on cloud infrastructure, and not offered as a shared service, falls outside that scope. Whether a self-hosted model in your own government cloud tenant is in scope therefore depends on how the service is offered and who uses it, so confirm the boundary with your agency's FedRAMP contact before you assume either outcome.
CJIS encryption and scanning
CJIS Security Policy v6.1 took effect on 26 June 2026 and requires FIPS 140-3 certified modules or FIPS 197 AES with at least 256-bit keys for criminal justice information in transit and at rest outside physically secure locations. It also requires vulnerability scanning at least monthly. Model weights, prompt logs and vector stores that contain CJI all fall under these rules, so check the cryptographic modules in your vLLM, gateway and tracing components before loading any CJI.
CUI handling with NIST SP 800-171
Under 32 CFR 2002.14(h)(2), federal agencies use NIST SP 800-171 when they set requirements for controlled unclassified information on non-federal systems, unless the CUI category's own authority prescribes other safeguards. A self-hosted inference stack that prompts with CUI inherits those requirements at the gateway, the logs and the operator access layer. Confirm with the data owner which CUI categories may enter prompts and whether retrieved documents or traces are themselves CUI.
CMMC cloud storage conditions
Under 32 CFR 170.17(c)(5), a defense contractor pursuing CMMC Level 2 may store or process CUI in a cloud service only if that service is FedRAMP Authorized at the Moderate baseline or higher, or meets equivalent requirements. Non-cloud providers are assessed within the contractor's own scope. A self-hosted model on hardware you own usually changes which assessment boundary applies, so ask your C3PAO or CMMC advisor how the inference host, the gateway and any hosted tracing service are classified.
NERC remote access and software integrity
Where the reader operates BES Cyber Systems, NERC CIP-005-7 requires Interactive Remote Access to pass through an Intermediate System with encryption and multi-factor authentication, and CIP-013-2 requires procurement processes that verify software integrity and authenticity. Each model file, container image and Python package that crosses an isolated boundary falls under that check. Confirm with your compliance team whether the AI stack touches BES Cyber Systems at all before adopting these controls.
Which open-source options keep inference inside the boundary
This table compares the open-source components needed to serve, front and observe open-weight models without an outside connection. Read each row for licensing, offline capability and operating burden, then confirm the items in the last column against the vendor's current documentation and your own environment before you choose.
| Component | Role in the boundary | Licensing and offline fit | What to confirm |
|---|---|---|---|
| vLLM | Serves open-weight models behind OpenAI-compatible endpoints inside your network | Apache-2.0 with no paid edition; offline use depends on pre-staged weights and container images | Accelerator support for your hardware, and that /metrics and /health are blocked at a reverse proxy |
| Open WebUI | Offline chat front end using Ollama or an OpenAI-compatible API such as vLLM | Open-source; the docs state it can run entirely offline with Ollama or OpenAI-compatible APIs | Confirm the license terms of the release you approve, and that no telemetry or update check reaches outside hosts |
| LiteLLM AI Gateway | Single endpoint with virtual keys, budgets, rate limits, fallbacks and logging to Prometheus or OpenTelemetry | MIT license; enterprise directory has its own license, and SSO, SAML and audit logs sit in LiteLLM Enterprise | Which gateway features you need, and whether SSO and audit logging require the enterprise license |
| Langfuse | Tracing and evaluation storage for prompts and completions | Open core: MIT outside ee directories, commercial license inside them; some advanced features need an enterprise key | Needs PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage, all inside the boundary; confirm which features the key unlocks |
| Opik | Tracing, evaluations and prompt management for model outputs | Apache-2.0; the README states the full platform can be self-hosted without a commercial license | Storage and database dependencies in your deployment, and whether your accreditation scope covers the chosen components |
| Arize Phoenix | Open tracing and evaluation built on OpenTelemetry and OpenInference | Elastic License 2.0, not OSI open source; it restricts offering the software as a hosted or managed service | Legal review of the license against your procurement policy before approval |
| LangSmith self-hosted | Tracing and evaluation deployed to your Kubernetes cluster | Requires an Enterprise plan and license key from LangChain, so licensing is a contract decision | Whether the deployment runs without outbound calls to LangChain services in your setup |
| Portkey hybrid gateway | Gateway runs in your VPC while the control plane stays in Portkey's VPC | The gateway is MIT licensed, but hybrid mode calls out over HTTPS to the vendor control plane | Whether a fully disconnected enclave can run without the control plane, which the hybrid design does not provide |
How the deployment differs by environment
On a classified or disconnected network with no transfer path, nothing reaches the enclave from the internet at run time. Every model file, container image, Python package, tokenizer and documentation asset must be brought in deliberately, verified and then kept inside the boundary. Installation and upgrades become controlled imports rather than routine pulls, and the vLLM server must be configured so it never attempts to fetch model artifacts from outside.
- Weight and image transfer: Model weights, the vLLM container image and its dependencies are staged on approved media and checked before they enter the enclave. Plan for a repeatable import procedure, not a one-off copy.
- Internal registries: Mirror the container registry and package indexes inside the boundary so cluster builds and rebuilds never depend on public endpoints.
- Offline licensing: Confirm with each model publisher and with your legal team whether the model license permits this use and whether any license check phones home. Vendor licensing for offline use is a separate question for any commercial component you add.
- Accreditation scope: Ask the accrediting authority whether the inference servers, the gateway and the logging stack belong to the system boundary, and which evidence they expect for each import.
- Multi-node traffic: vLLM's security guidance states that communication between nodes in a multi-node deployment is insecure by default, with no authorization or encryption. Place those nodes on an isolated network, and confirm the isolation with your security team.
Confirm the transfer procedure, the mirror inventory and the accreditation boundary before any hardware arrives.
A sovereign cloud region controlled by the agency or a designated operator still runs on shared cloud infrastructure, but the data, keys and administrative access stay under the agency's rules. Here the main work is proving who can reach the model servers and where the traffic travels, rather than rebuilding the transfer process from scratch. Pulls from public registries may be allowed in some regions and forbidden in others, so the answer must come from the region's own policy.
- Region and operator control: Confirm which region, operator and administrative personnel can reach the GPU instances, storage and control plane, and document that answer for the authorization package.
- Registry mirroring: Decide whether images and weights are pulled from a mirror inside the region or staged from approved media, and confirm the mirror is inside the authorized boundary.
- Accreditation scope: Confirm whether the cloud provider's existing authorization covers the infrastructure you use, and whether the model servers still need separate evidence. Do not assume inherited controls cover your application layer.
- Node isolation: If you run multi-node vLLM, the inter-node traffic remains insecure by default, so use a dedicated subnet or equivalent isolation and restrict what can reach it.
- Gateway exposure: Only the endpoints meant for users should be reachable from outside the cluster, which means a reverse proxy that allowlists them, not the bare
--api-keyoption alone.
Ask the operator to state, in writing, which personnel and tooling can reach the inference nodes and what evidence they will hand over.
In a government-owned data center, the agency controls the racks, the network and the personnel, so the design is closer to a traditional on-premises build. The advantage is direct control over isolation and hardware. The burden is that the agency also owns every operational control, from physical access and power to patching the GPU drivers and the orchestration layer.
- Physical and network isolation: Decide which VLANs or physical segments host the GPU servers and how they connect to the users and the gateway. Keep the multi-node traffic on its own isolated network.
- Cluster build without internet access: Kubernetes clusters can be bootstrapped from pre-pulled control plane images through a private registry, so plan that image list and mirror it before the build.
- Ownership of patching: Assign who applies driver, kernel, vLLM and gateway updates, and how each update is imported and tested in the same way as the original build.
- Evidence production: Agree which configuration baselines, scan results and access logs the accreditor will expect, and who collects them on a schedule.
- Hardware and drivers: Confirm the GPU models and driver versions against the vLLM installation requirements for that accelerator before purchase.
Confirm the accreditation boundary, the isolated network design and the patch ownership model with the accreditor and the data center team before procurement.
How vLLM exposes endpoints and why --api-key is not enough
The --api-key option on vllm serve protects only the endpoints under /v1, /v2, /inference and /cohere. It does not protect /invocations, /pause, /resume, /abort_requests, /metrics or /health. The vLLM security documentation warns against relying on the key alone. In a classified or disconnected enclave, that gap matters: a caller who can reach the API server can pause inference, abort in-flight requests or read operational data without presenting the key. Treat the key as one layer and put a reverse proxy that allowlists only the user-facing endpoints in front of the server, denying every other path by default.
Keep metrics off the user network. vLLM serves Prometheus-format metrics at /metrics, covering request counts, token throughput, latency histograms and KV cache use. These figures reveal workload patterns and capacity, which a security authority will want treated as sensitive. Expose /metrics only to a monitoring network with its own scrape path, and confirm with your security team how that network is segmented from the enclave's user zone. Also check which health probes your orchestrator needs, since /health is not protected by the key and should be reachable only from the cluster's control path.
Documentation access is the last item to decide. The --enable-offline-docs option lets the API documentation load without calling external content delivery networks, which matters on a disconnected network. Test the served docs page from a host with no outbound route before accepting the configuration. Record each setting, proxy rule and network path as accreditation evidence, and re-test them after every vLLM upgrade, because the set of endpoints can change between minor releases.
How to move weights and deploy vLLM across a disconnected boundary
This sequence takes approved open-weight models from a vetted supply chain into a disconnected environment, and ends with a documented baseline that an authorizing official can review.
Approve model and weight versions first
Before anything crosses the boundary, the security, compliance and platform owners agree on the exact model names, weight revisions, vLLM release and container image digests to be carried in. Map this approval to your supply chain control, such as the software integrity and authenticity expectations in NERC CIP-013-2 if you operate under that standard, or the equivalent procurement control in your own framework. The output is a signed bill of materials listing each file, its source, its checksum and the person who approved it. Confirm with your assessor which artifact types they expect to see.
Stage weights and images on approved media
Download the approved weights, the vLLM container image, Helm charts and Kubernetes control plane images on a connected staging host, then verify every checksum against the bill of materials. Copy the artifacts onto media approved for your boundary, using your own transfer and malware scanning procedure. The step is complete when the checksums on the far side match the staging record and the transfer log is filed with the change ticket. Ask your security team which scanning and media controls apply to your classification level.
Mirror the registry and pre-pull cluster images
Load the staged images into a private registry inside the boundary, then build the Kubernetes cluster from those images rather than public registries. Kubernetes supports preparing control plane images ahead of time with kubeadm and pointing kubeadm init at a private registry using its image repository option, so the cluster never reaches the internet during bootstrap. Your platform team owns this work, and the output is a cluster that starts with no outbound pulls. Confirm the exact kubeadm and Kubernetes versions with your platform vendor before building.
Deploy vLLM behind a reverse proxy and gateway
Deploy vLLM with the Helm chart or native manifests from the vLLM Kubernetes documentation, and place a gateway such as LiteLLM in front of it for virtual keys, budgets and logging. vLLM's own --api-key option does not cover every endpoint, and its documentation recommends a reverse proxy that allowlists only the paths meant for users, so the proxy should block administrative and metrics routes from application networks. The step is done when an allowed client call succeeds through the gateway and a blocked path returns a denial. Record the proxy rules as configuration under change control.
Wire metrics and logs to an internal stack
Point the vLLM Prometheus endpoint and gateway logs at your internal monitoring and log stack, which must also run without internet access. Offline guidance exists for monitoring of the same boundary: Wazuh publishes an offline installation guide and can use an offline vulnerability repository, and Security Onion supports airgap installs with updates delivered on new ISO media. Confirm with the vendors which of their components you will run and how their rule and signature updates reach the boundary. The step closes when dashboards show request counts, token throughput, latency and KV cache use from a test load.
Run acceptance tests with your own prompts
Run a fixed test set built from your real workload prompts against the candidate models, scoring outputs for accuracy, format, refusal behavior and latency under expected concurrency. Keep the test set, scoring rubric and results in a version-controlled repository inside the boundary so any later model change is compared against the same inputs. Involve the business owners who will sign off on quality, because only they can say whether an output is acceptable. The step is complete when each model has a recorded pass or fail result against the agreed thresholds.
Document the baseline for the authorizing official
Assemble the configuration baseline, bill of materials, network diagram, proxy rules, access model, monitoring coverage, test results and the approved change procedure into the package the authorizing official reviews. DeployOpen prepares this evidence for compliance assessments, but independent auditors and the authorizing official make the determination. The package is ready when every control claimed in the system security plan points to a stored artifact. Ask the authorizing official which templates and control baselines they require before you start writing.
How weight transfer and version upgrades work after go-live
Every model version, container image and Python library that enters an air-gapped boundary should pass through the same verified path, and that path should be written down before go-live. Build artifacts on an connected staging host, record a SHA-256 hash for each model weight file, image tarball and wheel, and store the manifest alongside the transfer media. Inside the boundary, recompute the hashes before anything is loaded, and keep the previous model directory, container image and requirements lock file as a rollback copy until the replacement has passed your acceptance evaluation. Ask your security team which signing or attestation method the accreditation authority expects for software crossing the boundary, since NERC CIP-013-2 frames the same question for software integrity and vendor patches in the energy sector and similar supply chain checks apply in federal and defense assessments.
The harder decision is cadence. vLLM ships new minor releases about every two weeks: v0.28.0 on August 26, v0.29.0 on September 9, v0.30.0 on September 22 and v0.31.0 on October 5, 2026, according to its GitHub releases. Two-week intervals are far faster than most accreditation and change-control cycles, so an isolated deployment should not track the release feed. Instead, set a fixed review window, for example quarterly, in which the team selects one release, runs its regression evaluations on your own prompts, reviews the release notes for security fixes, and submits a change request. Security fixes may justify an out-of-cycle transfer, so define that trigger in advance. Confirm with your accreditor whether a change to the serving stack requires a new authorization decision or only a documented significant change review.
The same discipline appears in mature offline products. Security Onion's airgap documentation describes updating an isolated install with the soup tool from newer ISO media rather than pulling packages from the internet, and GitLab's offline environment guidance covers self-managed instances that receive updates through transferred packages. Use these as models for your process: a media-based update, a staging rehearsal on a mirror of the production boundary, and a dated record of what was installed where. Before you commit to a schedule, ask your integrator and your vLLM operators to show the transfer manifest, the rollback procedure and the evidence file for each upgrade, because those documents are what an assessor will request first.
What assessors and accreditation teams ask to see
Assessors and authorizing officials judge a self-hosted model service by its boundary, its inventory, its protections and its change history, not by a vendor's certificate. The cards below list the evidence packages that accreditation teams usually request, and each one maps to a question they will ask about the vLLM service. DeployOpen prepares this evidence but does not issue audit reports, attestations or certifications; independent assessors and authorizing officials make those decisions.
System boundary and data flow
Draw the boundary around every host that touches prompts, completions, logs and model weights: the GPU nodes, the gateway, the observability stack, the object store and the admin workstations. Show every data flow across that boundary, including the transfer path for weights and container images into the isolated network. Confirm that no component calls an external endpoint at runtime, and record how that was verified. Assessors compare this diagram against the actual network configuration, so keep both under the same change control.
Component inventory with versions and licenses
List every software component with its exact version, source repository, license and owner, including vLLM, the inference container image, the gateway, the database and the operating system packages. Record the license of each item so that copyleft or enterprise-only directories are identified before deployment rather than during the assessment. Confirm which features are unavailable under the open-source license, such as enterprise SSO or audit logs in some gateways. The inventory is the baseline for every later vulnerability scan and upgrade record.
Encryption configuration for CJI or CUI
Where the data includes criminal justice information or controlled unclassified information, document how it is protected in transit between the gateway and vLLM and at rest in logs, caches and object storage. The CJIS Security Policy version 6.1 expects FIPS 140-3 validated modules or AES with 256-bit keys outside physically secure locations, so record the cryptographic modules actually in use and their validation status. Also record how inter-node traffic in multi-node vLLM runs is encrypted and isolated, because vLLM does not provide that protection by default. Confirm these settings with the authorizing official rather than assuming them.
Vulnerability scan results
Keep scan reports for the operating system, container images, Python dependencies and the GPU driver stack, produced with an offline vulnerability repository because the network has no internet access. Match the scan cadence to the baseline your authorizing body applies, since CJIS Security Policy version 6.1 requires vulnerability scanning at least monthly. Track each finding to a disposition: patched, mitigated with a documented compensating control, or accepted by a named owner. Assessors look for the closure record as much as the scan itself.
Access control and endpoint allowlist evidence
vLLM's --api-key option does not protect every route, including /metrics, /health and /invocations, so the documented approach is a reverse proxy that exposes only the endpoints meant for users. Provide the proxy configuration as an allowlist, the role mapping for each client, and the test results showing that blocked paths return denial. Include evidence that administrative and monitoring endpoints are reachable only from named management hosts. Keep the allowlist file under version control so every change has an author and a reason.
Weight and image provenance records
Record where each model weight file and container image came from, its checksum at import, the person who approved it, and the transfer media and date used to carry it across the isolated boundary. Verify software integrity and authenticity before import, since the supply chain control in NERC CIP-013-2 is an example of what utilities are asked to show for vendor software. Keep the signed or hashed manifest alongside the model card and license for each weight set. Without this chain, an assessor cannot tell whether the model running today is the one that was evaluated.
Change records for upgrades
vLLM publishes new minor releases about every two weeks, so an unmanaged environment drifts quickly. For each upgrade, keep the release notes reviewed, the test results on your own evaluation prompts, the rollback image, the approver and the date the change entered production. Show how a security patch moves through the same record as a feature release. Ask the authorizing official in advance which changes need a formal significant-change review, because that decision sets how much evidence each upgrade needs.
What do buyers ask before running this in a government network?
What does running open-weight models on vLLM in a government network cost?
The cost has four parts: accelerator hardware, the surrounding servers and storage, the people who build and operate the stack, and the accreditation evidence the boundary owner needs. vLLM itself has no paid edition, so its license adds nothing to the bill. Hardware is usually the largest recurring item and scales with model size, concurrent users and how many GPUs each model needs for tensor parallelism. Accreditation effort is mostly one-time but can be large in a regulated boundary. When you request a quote, ask for each component on its own line, including hardware lead times, the monitoring and gateway stack, and any hours assigned to documentation and testing.
How long does a classified or disconnected deployment take?
The duration depends on the boundary, the hardware procurement path, the accreditation process and how many models and gateways are in scope. This page does not give a timeline, and no researched figure supports one. Ask for a timeline in the written scope, with separate milestones for hardware delivery, weight transfer into the enclave, accreditation evidence, testing on your prompts and cutover. Ask which of those steps depends on your authorizing official or an external assessor, because those are the steps most often outside the implementer's control.
What does our team need to provide for the engagement?
You need to name the people who own the system boundary, the accreditation contacts who will review the evidence, and the hardware access the work depends on. Boundary owners decide which network segments, identity sources and logging feeds are in scope. Accreditation contacts decide which controls the evidence must address, for example controls from NIST SP 800-53 Rev. 5 selected through the applicable baseline in SP 800-53B. Hardware access means physical or change-controlled access to the GPU servers, the isolated network and a route for transferring signed packages and model weights across the boundary. Confirm each of these roles before work starts.
Can our own engineers run this without outside help?
Yes, if your team already operates Kubernetes or bare-metal GPU servers, can manage isolated networks and has people who can produce accreditation evidence. vLLM is open source under Apache-2.0 and exposes OpenAI-compatible endpoints, so application changes are usually small. The difficult parts are the security configuration, which requires a reverse proxy that allowlists only user-facing endpoints, the protection of node-to-node traffic in multi-node runs, and the evidence trail. Many teams combine internal operations with an outside implementation partner for the first deployment. DeployOpen plans, implements, hardens, documents and supports these platforms inside your own infrastructure, under a fixed scope agreed in writing.
Are open-weight models good enough for our government workload?
Only your own tests can answer that. Public benchmarks do not show how a model handles your forms, classifications, citizen correspondence or technical documents. Build an evaluation set from real, approved samples inside the boundary, score candidate open-weight models against the outputs your current service produces, and record the results as part of the acceptance evidence. Include failure cases such as refusals, formatting errors and long-context questions. Tracing and evaluation tools such as Langfuse or Opik can store those results under your control, and the test set should be version-controlled with the model it validated.
How do model and vLLM upgrades pass through an isolated network?
Each upgrade is a controlled import: a new vLLM release or model weight set is obtained outside the boundary, checked for integrity and authenticity, transferred across the boundary and tested before it serves users. vLLM ships new minor releases about every two weeks, so you need a policy that decides which releases you adopt, how often, and who approves them. NERC CIP-013-2 is an example of the supply-chain expectation: verify software integrity and authenticity of vendor software and patches before it crosses the boundary. Ask the implementer for a written upgrade procedure, a rollback plan and a record of each version in the accreditation package.
What to confirm with the vendor, the standard and your own environment
Treat every licensing, accreditation and hardware claim in this page as open until you have checked it. Confirm the license terms of each component before you build the disconnected bundle. vLLM is Apache-2.0 licensed according to its repository, but the gateway, tracing and evaluation tools in your stack may not be. Ask each vendor in writing whether offline or air-gapped use needs a commercial license, which features sit behind an enterprise key, and whether an offline license file works without outbound license checks. Vendor pricing pages in this category often place air-gapped deployment in paid editions, so do not assume a free tier covers it.
Next, establish the authorization status of your hosting environment and the control baseline that applies to your system. Ask your authorizing official or compliance office which baseline your system is assigned in NIST SP 800-53B and whether your CUI scope under 32 CFR Part 2002 or your CJI scope under the CJIS Security Policy applies to prompts, retrieved documents and completions. Those scopes decide which cryptographic modules, scanning cadence and access controls you must evidence. Confirm whether the enclave you plan to use already holds an authorization that covers the new model-serving components, because adding a component can change the boundary.
Last, verify the accelerator support for your exact hardware before you order or transfer anything. The vLLM installation documentation lists CUDA, ROCm, Intel XPU and CPU targets, and third-party hardware plugins cover other accelerators, so check the specific GPU model, driver stack and operating system against the release you intend to pin. Ask your hardware vendor which driver and container runtime versions they support in a disconnected build, and run a test load in a staging enclave on the same model family you plan to use. Record the results as evidence, because a model that loads on a development machine may not run on the accredited hardware.
How DeployOpen deploys vLLM under government network constraints
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, including isolated and disconnected networks. For air-gapped and government deployments of open-weight models on vLLM, we start with the network boundary, the approved software path and the evidence the assessors will need, then build and hand over against that written scope.
How we scope
- Written scope names the enclosures, networks and accounts where vLLM and the gateway will run, and who may reach them.
- We map the model weights, container images and Python packages that must cross the boundary, and the approved route for each.
- Scope lists the compliance frameworks your assessor applies, so the controls and evidence we prepare match what they will review.
How we build
- We stage the vLLM server, gateway and monitoring stack behind a reverse proxy that exposes only the endpoints users need.
- vLLM's --api-key option does not cover every endpoint, so we allowlist routes and isolate multi-node traffic on a separate network.
- We pull and verify images and weights offline, then test the deployment with your own prompts and access rules before go-live.
How we hand over
- Handover includes runbooks for transferring signed weights and images across the boundary, plus the checks performed at each transfer.
- We document configuration, network diagrams, access roles and the control mapping that supports your accreditation package.
- Upgrade procedures cover how a new vLLM minor release is validated offline, scanned and promoted into your environment.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer, scoped in writing with the same boundary rules.
- Retainer work can include re-running vulnerability scans, reviewing vLLM security advisories and applying approved patches.
- DeployOpen prepares systems and evidence for assessments but does not issue audit reports or certifications; your assessor makes that determination.
