Pages on the Kubernetes stacks, gateways and applications the vLLM documentation lists for running vLLM in production, limited to self-hosting and operations.
Deploying vLLM with the project's production-stack Helm chart: the router in front of model replicas, Grafana observability and LMCache KV cache offloading, and how it compares with plain manifests for a self-hosted inference platform.
Serving vLLM through KServe on Kubernetes, either with the Hugging Face serving runtime or the LLMInferenceService with llm-d, for teams that already run KServe for other models and want one serving control plane.
Running multi-node vLLM on Ray clusters managed by KubeRay: declarative RayCluster and RayService resources, autoscaling, blue/green upgrades and the network isolation that Ray and PyTorch distributed traffic require.
Putting the LiteLLM proxy in front of self-hosted vLLM to issue per-team keys, budgets and rate limits, route and fall back across deployments, and close the gaps in vLLM's own API key protection.
Giving staff a self-hosted chat interface on top of vLLM with Open WebUI: connecting it to the OpenAI-compatible endpoint, authentication in front of both services and capacity planning for interactive users.
Running Dify workflows and agents on self-hosted vLLM: installing Dify's vLLM model provider, pointing it at the vLLM endpoint and keeping both stacks inside the same network boundary.
Scaling vLLM horizontally on Docker hosts without Kubernetes, with several vLLM containers behind Nginx: least-connection balancing, failure detection, a shared model cache and endpoint allowlisting at the proxy.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.