vLLM Docker deployment on a single GPU host

When a vLLM Docker deployment fits a single GPU host

A vLLM Docker deployment runs the official vllm/vllm-openai image on one host, and it suits a single node that serves one model or a small set of models to internal applications. This fits a team that needs a language model available to its own systems from hardware it controls, and that does not yet need requests spread across several machines. The container starts the vLLM server, so the host needs a working GPU driver, a container runtime with GPU access, and enough accelerator memory for the model weights plus the key-value cache, which grows with concurrent requests. Before committing, confirm in the vLLM installation documentation that your GPU generation and driver stack are supported for the release you plan to pin.

vLLM is licensed under Apache-2.0 according to its GitHub repository, and it has no paid edition. The software line item is therefore zero, which can mislead budget planning. The cost of running it comes from GPU or other accelerator capacity, the server and storage around it, the implementation work to size and harden the host, and the people who operate it through upgrades and incidents. Accelerator capacity is usually the largest recurring component, whether you own the hardware or lease it. Ask any implementation partner to state the accelerator assumptions, the hours for setup and hardening, and the operations tasks included in the quote.

The server started with vllm serve exposes OpenAI-compatible endpoints, including /v1/completions, /v1/chat/completions, /v1/responses and /v1/embeddings, along with audio transcription and translation routes. Applications written against the OpenAI API can change their base URL and API key and keep most of their code. That lowers integration effort, but it also means every product team will send traffic to the same host. List which routes each application calls. Chat completions also require a chat template, which the model must provide or receive through the chat template option, so test that path before any application depends on it.

A single host is a reasonable target when a load test on your own model and hardware meets the latency and concurrency your applications need. Plan the exit path now. Multi-node runs use tensor parallelism within a node and pipeline parallelism across nodes, and the vLLM security documentation states that communication between nodes is insecure by default, so those nodes must sit on an isolated network. The API key option also does not protect every route, including /metrics and /health. The proxy in front of the container should therefore allow only the endpoints users need and keep administrative and metrics routes internal.

Before approving the design, run four checks. Confirm the GPU and driver combination against the installation documentation. Load test the chosen model at the concurrency you expect. Send test requests to each route the proxy should block and confirm they fail from user networks. Record the exact image tag you pin, because vLLM ships new minor releases about every two weeks, and an unpinned image can change behavior between restarts. Write each result into the written scope so the partner and your team share the same acceptance criteria.

Container options and when each fits

This table lists the container and server settings a single-host vLLM deployment decides on, what each one controls, and when it fits. Read each row before the first run, then confirm exact option names and defaults against the current vLLM and Docker documentation for the release you plan to pin. The image tag, API key and port binding decide what can be reached and how the deployment changes over time, so they deserve the most review.

Setting	What it controls and when it fits	What to confirm
Image tag or pinned version	Selects the vLLM release in the container. A fixed tag keeps restarts repeatable. Fits production; floating tags suit test hosts.	Release notes for the pinned tag, and the host driver and CUDA versions it requires.
GPU device exposure	Makes host GPUs visible to the container and controls which devices it can use. Fits one GPU host or a fixed subset of cards.	Docker's GPU device option, the NVIDIA Container Toolkit setup, and vLLM's supported accelerator list.
Shared memory allocation	Sets the size of /dev/shm inside the container. Multi-GPU runs often need more than Docker's default, so set it deliberately.	Docker's shared memory option and vLLM's guidance on shared memory for multi-GPU runs.
Model cache volume	Stores downloaded model weights outside the container so restarts do not re-download them. Fits hosts with fixed disk and backup rules.	The cache path used in the image, and the disk space your model list needs.
Hugging Face token	Passes credentials for gated models into the container. Avoid baking it into image layers or shell history. Fits gated or private repositories only.	The environment variable name vLLM reads for the token and which secret handling methods are supported.
API key	Requires a key on /v1 endpoints. Routes such as /metrics and /health are not covered, so add a reverse proxy allowlist.	The current list of protected routes in the vLLM security docs and the allowlist your proxy enforces.
Port binding	Maps a host port to the server port inside the container. Binding to localhost or a private interface limits exposure. Fits access through a proxy.	The vLLM host and port options, the Docker publish syntax, and your host firewall rules.
Chat template	Supplies the prompt format for the chat completions API when the model does not include one. Fits base or older models; check the model card first.	The --chat-template option in vLLM's online serving docs and whether your model ships a template.

Model cache volumes, gated model tokens and version pinning

The model cache should live on a persistent volume mounted into the container, so model weights download once and survive container restarts. Without that mount, every start pulls the full weights again, which adds startup time, uses bandwidth, and fails whenever the model hub is unreachable or the revision has changed. The volume is a recurring cost in disk space that grows with every model and every revision you keep. Ask the team to measure the weight size of each model on the shortlist, then add headroom for at least one additional revision during an upgrade and for test models that will be removed later. Confirm the cache path that the image's Hugging Face libraries use for the tag you intend to pin, and confirm the variable that redirects it, because a mount at the wrong path leaves the cache inside the container layer.

Gated models need a token, and the token needs a deliberate delivery path. Use a read-only token issued to a service account dedicated to this host, granted access only to the gated repositories your team has approved. Deliver it at runtime through a file-based or orchestrator secret that the container reads when it starts, then exposes to the process as an environment variable. Keep it out of the Dockerfile, including ENV and ARG instructions, because those values persist in image history. Do not copy a token file into the build context, and do not pass the literal value with a docker run flag typed at the shell, since that lands in shell history and in the process list. Rotate the token when people leave the team or when the host is retired, and confirm the token types the model hub currently offers before you choose one.

Pinning matters because a floating tag such as latest can change under you between one restart and the next. Pin the image by digest where your registry supports it, and pin the model by its repository revision, which is a commit identifier, so that the same weights load every time. Record both in the same change record, so a rollback restores the engine and the weights together. The cadence makes this a standing task. vLLM ships new minor releases about every two weeks, with v0.28.0 on August 26, v0.29.0 on September 9, v0.30.0 on September 22 and v0.31.0 on October 5, 2026. A pin that is never revisited will fall behind quickly, and a team that tracks every release will spend steady time on testing. Choose a documented cadence before the first production start.

The upgrade plan turns pinning into a routine. Stage each candidate image on a non-production host with the same model revision, the same flags and the same chat template, then run a fixed set of prompts, check the OpenAI-compatible endpoints your applications call, compare latency and throughput against the current pin, and confirm the Prometheus metrics at /metrics still scrape cleanly. Read each release's notes for changed or removed flags and for changes to default behavior. Keep the previous image digest available so rollback is a single change. Ask any implementation partner to price this as a recurring activity: testing time, staging capacity, and the number of upgrade cycles per year, since those are the drivers of ongoing cost. Confirm who approves promotion to production and who owns the rollback decision before the first cycle.

When a custom image is worth building

Each card covers one reason to move from the official vllm/vllm-openai image to a custom build, along with what that fork costs to maintain. Before building, check whether a mounted file, an environment setting or a runtime flag solves the problem. Every custom layer adds rebuilds, dependency checks and CVE scanning to ongoing operations.

Extra Python packages in the image

Trigger: your serving code, a tokenizer helper or a logging library needs a package the official image does not include. The cost is a Dockerfile layer that must be rebuilt against each new vLLM release, and the project ships new minor releases about every two weeks. Pinned packages can conflict with the torch and CUDA versions vLLM depends on, so resolve the full dependency set in a test build. Scan the added layer for CVEs on each rebuild. Question to ask a partner: how will you test the pinned set against each vLLM release before we adopt one?

Chat template missing from a model

Trigger: a model ships without a chat template, so the chat completions endpoint cannot format messages. vLLM accepts a template through the --chat-template option, so confirm whether a mounted template file covers the case before you add an image layer. If the template must be baked in, the layer is small but still ties you to every rebuild and to the model revision it was written for. Scan that layer like any other. Question to ask a partner: can you show whether a runtime option or mounted file works for this model before we accept a custom image?

Plugins for non-NVIDIA accelerators

Trigger: the hardware is outside the NVIDIA CUDA path and needs a third-party hardware plugin, which lives outside vLLM's main repository. AMD ROCm and Intel XPU have their own images or build paths covered in sibling topics, so confirm the right base image for your hardware in the installation docs before writing a Dockerfile. A plugin build carries the heaviest maintenance: it depends on the driver stack, the plugin's own release cycle and the vLLM version, and each combination needs rebuild and CVE review. Question to ask a partner: which plugin version is tested against which vLLM release, and who owns that matrix?

Internal CA certificates

Trigger: the host must reach a model mirror, package proxy or artifact store signed by a private certificate authority. Adding the CA bundle to the image is a small layer, but it bakes trust into every container built from that image, so a certificate rotation forces a rebuild. Where a bundle can be mounted at runtime, keep the certificate out of the image and reduce the rebuild frequency. Scan the image after each change, as with any other layer. Question to ask a partner: how will we rotate the CA, rebuild on that schedule and store the bundle securely?

Setup sequence for a vLLM Docker deployment on one host

This sequence takes one GPU host from a bare operating system to a vLLM server that answers OpenAI-compatible requests on the interface you intend. Each step ends with a check your team can run before moving on, so problems surface while the change is still small.

Confirm the accelerator and driver stack
Your platform or infrastructure engineer checks the GPU model, the driver version and the CUDA stack against the support matrix in the vLLM installation documentation for the release you plan to run. Record the results in the change ticket, because the container image carries its own user-space libraries but still depends on the host driver. Done means every item in the matrix matches the host and the GPU is visible to the operating system with its driver loaded. If the host falls outside the documented matrix, stop here and get a decision from the vendor or your implementation partner before going further.
Install the container runtime and GPU support
The host administrator installs a container runtime and the GPU container toolkit for that runtime, then grants the operator account permission to run containers. Confirm the setup by running a test container that lists the GPUs it can see. The test passes when the same devices and driver version appear inside the container as on the host. If the GPU list is empty inside the container, fix the runtime's GPU configuration before pulling the vLLM image, because a missing device later looks like a model problem.
Create the model cache volume
Create a dedicated volume or host directory for downloaded model files and mount it into the container at the cache path the image uses. Place it on storage with enough free space for the weights you plan to serve, and set ownership so the container user can write to it. Verify by starting a throwaway container that writes and deletes a test file in the mounted path, then confirming the file appears on the host. A persistent cache also keeps model weights in place across container restarts and image rebuilds, so the server does not download them again each time.
Supply the gated model token securely
If the model is gated, accept its terms with the account that will issue the access token, then pass the token to the container from a secrets store or a restricted environment file. Keep it out of shell history, container inspection output and committed compose files. Limit read access on the file to the operator account. Verify by checking that the token does not appear in the container configuration or the shell history, and that the model download starts without an authorization error when the server launches in the next step.
Pin the image tag and model revision
Replace any floating latest tag with the specific vllm/vllm-openai image tag you validated in the installation check, and record its digest. Pin the model to a specific revision so an upstream change cannot alter the weights your applications were tested against. Store both values in the deployment record. Verify by starting the container with the pinned values and confirming the startup log reports the expected vLLM version and the model revision you recorded. Repeat this check whenever either value changes.
Start the container on the intended interface
Launch the pinned image with GPU access, the cache volume, the token, the model identifier and the revision, and set the shared memory size the workers need. Bind the API to the interface you intend: loopback for local testing, or a private address behind a reverse proxy for shared use. Do not expose a public address without the proxy and authentication in place. Verify with the startup log showing the model loaded and the server listening on the bound address, and with a socket listing on the host confirming the port is open only where you planned.
Check the endpoint with a model list
From an allowed client, call the models endpoint and confirm the served model name matches the deployment record. Then send one small completion or chat request to confirm generation works end to end. If chat requests fail, check that the model has a chat template or that one was supplied at launch. Verify that both calls return successful responses. Keep the metrics and health routes behind your proxy allowlist, because the API key option does not cover them.

How to verify the endpoints before users connect

Before any application points at the server, confirm that the chat completions endpoint returns a correct answer for the exact model and configuration you plan to run. Send a request to /v1/chat/completions using the served model name, and check the response structure, the finish reason and the token counts. Chat requests depend on a chat template. vLLM needs the model to provide one, or you must supply one with --chat-template. If neither exists, the completions endpoint may work while chat fails, so a text-only smoke test does not prove the chat path is ready. Confirm the template against the model card or the upstream model documentation, and keep the template file under version control with the rest of the deployment. If you plan to expose /v1/embeddings, test it separately. An embedding request is only meaningful when the model you loaded is an embedding model and your vLLM version supports it for that model. Check the vector dimension, the response shape and the input limits against what your client library expects. Compare the output for a fixed input with a reference value from a trusted run, so that a silent change after an image update is caught before users see it. Skip this check if embeddings are out of scope for the first release, and say so in the change record. The /metrics and /health endpoints need a separate test. The --api-key option does not protect them, and the vLLM security documentation recommends a reverse proxy that allowlists only the endpoints meant for users. From outside the network, send unauthenticated requests to /metrics, /health and the administrative routes such as /pause and /resume, and confirm the proxy blocks each one. From inside the network, confirm that the Prometheus scrape job and the load balancer health check can still reach /metrics and /health on the internal address. Metrics cover request counts, token throughput, latency histograms and KV cache use, which makes them useful for capacity decisions, but they also reveal workload patterns to anyone who can read them. Load test with prompt lengths that match your expected traffic, not only short greeting-style prompts. Build a test set from representative, anonymized requests, including the longest prompts you expect to accept, and run concurrency levels that approach the planned peak. Watch time to first token, end-to-end latency, error rates, queue depth and KV cache use in the Prometheus data during the run. Record where latency rises sharply or requests start failing. That ceiling is the figure to compare against your capacity plan, and the test should be repeated after any change to the model, the image or the serving flags. Ask the team that will operate the server to confirm which limits they will alert on. Finally, record the pinned image digest, the model revision (the exact commit or version of the weights), the full vLLM command line, the chat template file, the proxy configuration version and the results of the checks above in the change record. vLLM publishes new minor releases about every two weeks, and an image tag can be repointed, so the digest is what lets you reproduce the tested state later. Confirm with your registry how digests are displayed and stored in your environment, and make the change record the reference point for any future upgrade test.

Security and operational considerations for a single host

A single vLLM container on one host has a small attack surface, but several defaults need deliberate handling before users reach it. The cards cover authentication scope, network placement, secrets, operational endpoints, container and image trust, and logs. Check each item in your own environment before go-live.

Limit the API key with a proxy

The --api-key option protects only the /v1, /v2, /inference and /cohere routes. It does not cover /invocations, /pause, /resume, /abort_requests, /metrics or /health, and the vLLM security docs warn against relying on it alone. Place a reverse proxy in front of the container that forwards only the user-facing endpoints your applications call, and deny everything else. Test the allowlist by requesting each excluded path from outside the proxy and confirming that it refuses the request.

Keep the port off shared networks

Bind the published port to a private interface or loopback so that only the proxy can reach it. A listener on a public or shared network exposes every endpoint, including the unprotected ones, to anyone who can route to the host. Check the host firewall and any cloud security group, then confirm with a scan from outside the network. Multi-node communication is insecure by default, according to the vLLM security docs, and must sit on an isolated network. A single-host deployment avoids that path. The multi-tenant and FIPS topics cover deeper controls.

Handle gated model tokens as secrets

Gated models need an access token from the model hub at download time. Pass the token through a file or secret manager that the container reads at start, and keep it out of the Compose file, shell history and image layers. Use a read-only token scoped to the models you need, and rotate it when people with access leave. Check that docker inspect output and process listings do not reveal the token, and that the model cache volume is readable only by the service account.

Restrict the metrics endpoint

vLLM serves Prometheus-format metrics at /metrics. They cover request counts, token throughput, latency histograms and KV cache use, which reveal load patterns, capacity limits and traffic timing to anyone who can read them. The API key does not protect this path, so block it at the proxy for user traffic and allow scraping only from your monitoring host. Review the histogram bucket settings as well, since they shape what the endpoint reveals. Confirm that the scrape still works after the change.

Container privilege and image trust

Start the container without privileged mode, drop Linux capabilities it does not need, and avoid mounting the Docker socket or host paths beyond the model cache. Use the official vllm/vllm-openai image with an explicit version tag, and record the image digest your deployment runs so that upgrades happen on purpose. Verify the digest against the registry before each pull, and rebuild custom images from the same pinned base. Read the release notes for each new minor version before you move to it.

Control logs and retention

Container logs are written through the Docker log driver on the host. Depending on the version and log level you run, they can carry request details, errors and configuration values, so confirm what your vLLM version writes at the level you choose. Set a log driver with size limits so the disk does not fill, and restrict read access to the log store and the docker group, since docker group membership grants root-equivalent control of the host. Set retention to match your policy. If you need audit trails, forward logs to a central system over a protected channel and test that a log rotation does not drop entries.

Upgrading and pinning across vLLM releases

The first decision for any single-host vLLM deployment is whether to track new releases at all, and that decision should be made deliberately rather than by pulling the latest image on a schedule. vLLM ships new minor releases about every two weeks. The published examples are v0.28.0 on August 26, v0.29.0 on September 9, v0.30.0 on September 22 and v0.31.0 on October 5, 2026. At that rhythm, an unpinned container tag such as latest can change the serving engine under a production model several times within a quarter. Most teams should pin an exact version tag of the vllm/vllm-openai image in their compose file or run script, record that tag in change management, and move it only through a planned change. Confirm the tag naming convention for the image you intend to use on the vLLM GitHub releases page before you write the pin into automation.

A new tag should never reach the production host first. Stand up a staging host with the same GPU model, driver stack, CUDA-compatible image base, model weights and cache volume layout as production, then start the candidate tag with the same vllm serve arguments. Run a fixed set of prompts that reflect your real traffic, including chat requests with your model's chat template, embedding or audio calls if you expose them, and long-context requests that stress KV cache memory. Compare output behavior, time to first token, throughput and memory headroom against the current tag. Keep the test prompts and results in version control so the next upgrade has a baseline to compare against. Be aware that a staging host with a smaller GPU or different driver will not reproduce memory failures that matter, so confirm the hardware match before you rely on the results.

Read the release notes for every version between your current pin and the candidate, not only the newest one. Look for changed or removed command-line flags, renamed environment variables, altered defaults for memory utilization, maximum sequence length or shared memory behavior, and any change to the OpenAI-compatible endpoints under /v1, including /v1/completions, /v1/chat/completions, /v1/responses and /v1/embeddings. Check whether your client applications depend on response fields that a release has modified. Also check whether the release changes how the chat template is loaded or whether the model needs a new template passed with --chat-template. Each change found in the notes should map to a line in your run configuration, a test case in staging, or a documented decision to leave the behavior alone. Changes you cannot explain are a reason to delay the upgrade.

Before every cutover, keep a rollback image: the previous known-good tag must remain pulled on the host, and the previous compose file or run script must be saved alongside the new one. Rollback should be a tag change and a restart, not a rebuild. Record the image digest as well as the tag, since a tag can be re-pointed by its publisher. Confirm that the model cache volume and any custom image layers you added for extra dependencies still work with the older tag, because a custom image built on a newer base may not run on the old one. Upgrade work in this area, including image rebuilds, staging tests, release reviews and rollback drills, is covered in more detail on the vLLM upgrades page. If your organization does not have the capacity to run this cycle every two weeks, a retainer can carry release tracking, staging tests and scheduled pin updates over time, with the scope and cadence agreed in writing before work starts.

Limits and what to confirm in your environment

Each card names one thing to confirm before a vLLM Docker host goes into service, and who can confirm it. Some answers come from vLLM's documentation, some from the model card, and the rest only from a startup test on your own hardware.

Accelerator and driver support

Confirm which GPU generations, CUDA versions and host driver versions the image tag you pin supports. The vLLM installation documentation and the release notes for that tag are the sources to check, and your GPU vendor's driver documentation covers the host side. Record the host driver version in your build record so that any driver update triggers a repeat startup test before it reaches production.

Shared memory size for the container

Containers get a small default shared memory allocation, and a model server that passes tensors between worker processes can fail at startup or under load when it is too small. Check the shared memory setting that the vLLM Docker documentation currently recommends for the image tag you run, then set it in your run command or Compose file. Confirm the value on your host by serving realistic traffic, because the recommendation can change between releases.

Model license and gated access

Read the license on the model card and confirm it allows your intended use, including internal production use, serving other teams, and any fine-tuned derivatives. Gated models also need an access request approved by the model owner and a model hub token tied to an account that has accepted the terms. Confirm these points with the model publisher and your legal team, then document who holds the token and how it is rotated.

Chat template status

The chat completions endpoint needs a chat template to format messages for the model. Check the model card and the tokenizer configuration in the model repository to see whether the model ships one. If it does not, supply a template with the chat template option, and confirm that its prompt format matches what the model expects. Verify the result with a test request to the chat completions endpoint before any application depends on it.

GPU memory against model and context

Model weights, the KV cache and runtime overhead must all fit in GPU memory at the context length you plan to serve. Use the model card to find the parameter count and numeric precision, then confirm the fit by starting the server on the target GPU with the maximum model length and concurrency you expect. If startup fails, lower the context length or the memory utilization setting, and record the values that worked as the baseline for capacity planning.

Image or source build for hardware

The official vllm/vllm-openai image covers the common accelerator stacks, but a different GPU vendor, CPU architecture or Python requirement can call for a different image or a source build. Check the vLLM installation documentation for your accelerator and confirm that the published image includes its runtime. If it does not, plan a custom image or source build and test it on the same hardware before you standardize on it. Third-party hardware plugins need their own version review.

Questions buyers ask before hiring help with a vLLM Docker deployment

What drives the cost of hiring help with a vLLM Docker deployment?

The cost comes from four places: accelerator capacity, the surrounding infrastructure, implementation labor and ongoing operations. vLLM itself has no paid edition, so the software line in your estimate is zero.

Your infrastructure provider charges for GPU hosts, storage, networking and backups, usually on a recurring basis.
A partner charges for scoped work: planning, the container build, model access setup, security configuration, testing and handover.
Your own staff time is the cost teams most often leave out.

Ask any quote to separate one-time implementation from recurring operations, and to state which GPU type and host count it assumes. Capacity is often the largest recurring line, so confirm it before comparing offers.

Does vLLM have a paid edition or license fee?

No. vLLM is open-source software licensed under Apache-2.0, and it has no paid edition. Its cost comes from the GPU or other accelerator capacity that runs it, the infrastructure around it, the implementation work and the people who operate it.

Models are a separate question. Each model you plan to serve carries its own license and access terms, and some require accepting terms or using an access token before download. Review those terms for every model in scope, and confirm with your legal or compliance team whether any model restricts commercial use, redistribution or hosting in a particular region.

When you compare quotes, check that the vLLM license is not treated as a cost item, and that model licensing is listed as an assumption with an owner.

How long does a single-host vLLM Docker deployment take to set up?

Duration depends on the written scope, so ask for a timeline in the quote rather than relying on a general estimate. A single host goes through the same stages whatever its size: confirming the host and drivers, pinning the image version, setting shared memory and model cache volumes, configuring model access, running a smoke test, checking the API endpoints your applications call, enabling metrics, and handing over runbooks.

The stages that vary most are model access approvals, driver and container runtime issues on the host, and the security review. A quote should name which stages it covers, which dependencies it assumes your team clears, and what acceptance tests end the engagement. Ask the partner to show the sequence of work and the points where your team must act.

What changes the effort of a single-host vLLM deployment?

Four factors move the effort most.

Number of models: each model needs its own access check, configuration and test pass.
Hardware target: NVIDIA CUDA, AMD ROCm, Intel XPU and CPU backends each carry different driver, image and tuning steps.
API surface: the server exposes completions, chat completions, responses, embeddings and audio endpoints, and each one you expose needs testing. Models also need a chat template before the chat API works, or one supplied at launch.
Security review: the level of access control, network placement and logging decides how much hardening and evidence work is needed.

Air-gapped hosts, custom images with extra dependencies, and multi-tenant use add further work. Ask the partner to list which of these apply before estimating the review and test effort.

What does our team need to provide before the work starts?

The work depends on access and decisions that only your organization can supply, so agree them in writing before the start date.

Hardware access: a GPU host or hosts in your cloud account or data center, with drivers and a container runtime you approve, plus access for the people doing the work.
Model access and tokens: accounts and access tokens for any gated model, and a decision on which model sources are allowed.
Network and identity decisions: which network the endpoint sits on, who can reach it, how users authenticate, and where logs and metrics go.
A named owner: one person who approves changes, accepts deliverables and answers questions during the work.

Without these, schedules slip, and the partner cannot test what your users will actually call.

Can our own engineers deploy vLLM with Docker without a partner?

Yes. The vLLM project publishes installation and serving documentation, and a team can run the official vllm/vllm-openai image on a single GPU host without outside help. Start with a test host that matches your production hardware, so that driver, memory and model issues show up before production.

The team needs working knowledge of Linux administration, GPU drivers, Docker and the NVIDIA container tooling (or the equivalent for your accelerator), basic HTTP API testing, and secrets handling for model tokens. Someone must also own security configuration, including who can reach the server and which endpoints are exposed, and monitoring for the metrics endpoint.

A partner adds value when deadlines, compliance evidence or several models make the internal workload hard to schedule. Decide that based on your team's capacity and the risk of an error in production.

What should a vLLM deployment quote include?

A useful quote states the scope in writing, with deliverables and acceptance tests, before any work begins. DeployOpen works on fixed scope, so the written scope is the basis for the quote.

Scope: the host or hosts, accelerator type, models, image version and API endpoints covered.
Assumptions: GPU capacity, model licenses, network design and identity sources the quote relies on.
Deliverables: the configuration, runbooks, test results and handover materials.
Exclusions: anything outside the listed hosts, models or endpoints, such as custom model training.
Recurring items: infrastructure costs, support, upgrades and any retainer, shown separately from one-time work.
Change process: how additional models or endpoints are priced and approved.

Ask for the timeline, the named roles on both sides and the acceptance criteria as part of the same document.

Is the vLLM api-key option enough to secure the server?

No. The --api-key option protects the endpoints under /v1, /v2, /inference and /cohere, but it does not cover /invocations, /pause, /resume, /abort_requests, /metrics or /health. The vLLM security documentation warns against relying on this option alone.

The documentation recommends a reverse proxy that allowlists only the endpoints meant for users, and that blocks the control endpoints from ordinary clients. Metrics can reveal request volumes and usage patterns, so restrict them to your monitoring system.

In practice, the security review should cover the proxy rules, TLS termination, network placement, token handling and log retention. Ask the partner to show which endpoints are reachable from each network zone, and test the allowlist from outside the trusted network before go-live.

How DeployOpen scopes and delivers a vLLM Docker deployment

DeployOpen plans and delivers self-hosted vLLM Docker deployments inside infrastructure you control, starting from a written scope. The work covers the single GPU host, the vllm/vllm-openai container, cache volumes, and the endpoints your applications call. Most engagements are fixed-scope, with ongoing support available as a retainer.

How we scope

The written scope names the GPU host, driver and container runtime versions, and the vLLM image tag to pin.
Model list is confirmed against the chat template, tokenizer and gated access your team holds before build starts.
Client applications, API keys and the OpenAI-compatible endpoints they call are listed as scope items with owners.

How we build

Host preparation covers shared memory size for the container, a persistent model cache volume and restricted file permissions.
Gated model tokens are injected through a secrets mechanism your organization controls and never baked into images.
Custom images extend the official vllm-openai image only for dependencies the model needs, with each build recorded and tested.

How we hand over

Handover includes the compose or run definition, pinned image digests, environment variables and a documented upgrade procedure.
Test evidence covers a model load, completion and chat requests, and a restart with the cache volume intact.
Your team receives a runbook covering logs, the /metrics endpoint and ports that must stay closed to outside traffic.

How we support

Retainers can cover image updates, tracking vLLM releases that ship about every two weeks, and retesting pinned versions.
Upgrades move between release tags in a staging host first, with the same model, cache and endpoint checks repeated.
Security testing and compliance evidence can be added later as separate scoped work, if your review needs them.