The OpenAI API serves OpenAI's hosted models per token. Pages here cover self-hosting open-weight models on vLLM or Ollama behind your own gateway when data control, residency, deprecations or cost push teams off the hosted API.
Comparing the hosted OpenAI API with open-weight models served on your own vLLM cluster: API compatibility, where prompts are processed and retained, model choice and version control, GPU capacity planning, security hardening of the endpoint, and who is on call when it breaks.
For developers and small teams considering Ollama instead of the OpenAI API: what Ollama's OpenAI-compatible local server covers, where it stops short for multi-user production traffic, its cloud features and how to turn them off, and when to move up to vLLM.
Moving production traffic from the OpenAI API to open-weight models on vLLM: putting a LiteLLM gateway in front so applications keep one OpenAI-compatible endpoint, sizing GPUs, validating outputs against current prompts, shifting traffic gradually and keeping OpenAI as a fallback during cutover.
Cost of ownership by component: per-token API spend, residency uplifts and enterprise data controls against GPU capacity, the gateway, monitoring, implementation and the engineers who run inference, with the utilization and traffic patterns that decide which side is cheaper.
For EU teams weighing OpenAI's European data residency, which needs sales approval and abuse monitoring controls for regional processing, against open-weight models served in their own EU datacenter or cloud region: what each keeps in the EU, GDPR processor questions and operating cost.
OpenAI's Zero Data Retention and Modified Abuse Monitoring need prior approval and do not cover every endpoint. This page covers what is retained by default, what approval changes, and when serving models yourself is the simpler way to guarantee prompts are never stored by a third party.
Running LLM inference where no request can leave the network: open-weight models on vLLM or Ollama, transferring model weights and container images in, offline documentation, endpoint authentication and network isolation, and how updates and model upgrades work without internet access.
OpenAI gives at least six months' notice before shutting down generally available models and less for previews, which forces prompt revalidation on its schedule. This page covers pinning an open-weight model version on your own infrastructure, the work that moves to you, and when that trade is worth it.
Replacing OpenAI's embeddings endpoint with open-weight embedding models served through vLLM's OpenAI-compatible /v1/embeddings: why every stored vector must be re-embedded when the model changes, sizing the re-embedding job, and running it next to an existing vector database.
For teams not ready to leave OpenAI: placing a self-hosted LiteLLM gateway between applications and the API to own keys, budgets, rate limits, logging and fallbacks to self-hosted models, so a later move to open-weight models becomes a routing change instead of an application rewrite.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.