Azure OpenAI or self-hosted vLLM: which fits your Azure OpenAI alternative decision
If your renewal is approaching and your main concern is that you cannot control where prompts are processed or which model version answers them, self-hosted vLLM is the better fit. If you need to keep managed access to OpenAI models with Microsoft running the service, and your data residency terms are already met by a deployment option you can verify, Azure OpenAI remains the simpler choice. Most teams facing a PTU planning question or a model retirement land somewhere between these two, so the decision needs to be made on specific criteria rather than on general preference.
Four factors drive the decision. The first is data processing location: Azure OpenAI deployment types differ in where prompts may be processed, so confirm the exact deployment type your applications use against Microsoft's current documentation and your own residency obligations. The second is model version control: with vLLM you pin the open-weight model and its revision yourself, while Azure OpenAI schedules retirements on Microsoft's cadence. The third is throughput commitments: reserved capacity gives a contracted allocation in Azure, whereas a self-hosted cluster delivers only what your GPUs can serve. The fourth is in-house capacity, meaning whether your team can operate GPU nodes, the vLLM serving stack, a gateway and monitoring, including on-call coverage.
A practical first step is to write down, for each application, its data classification, its required model behavior, its peak token load and its tolerance for a forced model change. Applications that fail on residency or retirement risk are candidates for vLLM behind a gateway such as LiteLLM, which exposes an OpenAI-compatible endpoint so client code changes are usually limited to the base URL and model name. Applications that depend on capabilities you cannot reproduce on open-weight models, or on a team with no capacity to run GPUs, may remain on Azure OpenAI while you plan a phased move. Confirm model capability gaps on your own prompts before committing to either path.
Which option fits which scenario
This table maps common requirements to the option that usually fits better and the condition that makes it true. Read each row against your own constraints, and treat a row as decisive only when its condition describes your environment. Most teams find that one or two rows dominate their decision.
| Requirement | Better fit | Condition under which it holds |
|---|---|---|
| Strict processing location requirements | Self-hosted vLLM | You must name the exact region, facility or network where prompts and completions are processed, and Azure OpenAI deployment types cannot guarantee that location. Confirm the processing options for your tenant with Microsoft. |
| Need for a pinned open-weight model version | Self-hosted vLLM | Your application or validation process depends on one exact model build that you can keep running for longer than the provider's retirement schedule. You also need to test each upgrade before it reaches production. |
| Reliable throughput at peak with PTUs | Azure OpenAI with PTUs | Your peak load is predictable and you can size reserved capacity from measured token volumes. Self-hosting only wins here if your team can match the capacity planning, failover and on-call coverage that the reservation includes. |
| Small or spiky usage | Azure OpenAI pay-per-token | Traffic is low or irregular, so idle GPUs would dominate your cost. Self-hosting becomes competitive only when utilization is sustained enough to keep accelerators busy. |
| Teams without GPU operations staff | Azure OpenAI | No one on your team can own driver upgrades, capacity planning, model loading, monitoring and incident response for accelerators. If you choose self-hosting, plan for a managed partner or retainer to cover those duties. |
| Air-gapped or sovereign networks | Self-hosted vLLM | Your workloads must run disconnected from public cloud services or under sovereign control. Confirm with your accreditation authority which weight transfer, logging and upgrade evidence it expects. |
| Need for OpenAI-specific models or features | Azure OpenAI | Your application depends on OpenAI models, their proprietary tooling or features that open-weight models do not replicate. Confirm each feature against vLLM's OpenAI-compatible endpoints before assuming parity. |
How applications switch with the OpenAI-compatible API
Most applications that call Azure OpenAI can move to vLLM with a smaller change than teams expect, because the vLLM server exposes OpenAI-compatible endpoints such as /v1/chat/completions, /v1/responses and /v1/embeddings. In practice the code change is usually the base URL, the API key and the model name. Azure OpenAI also uses resource-specific endpoints and an api-version parameter, so those values, and any code that builds Azure-style deployment paths, need to be found and replaced explicitly. Inventory every service that reads these settings, including batch jobs, evaluation scripts and internal tools that developers may have copied years ago.
A LiteLLM gateway can sit between the applications and the backends so that teams change one endpoint and one set of keys instead of every client. Its virtual keys, per-key, per-user and per-team budgets, TPM and RPM limits, model restrictions, fallbacks and routing rules let you send traffic to vLLM, to Azure OpenAI during a cutover, or to both. Confirm in the LiteLLM documentation which of these features your chosen version supports and how they behave under your traffic, because routing and failover change where requests land and what they cost.
Compatibility of the request format is not the same as equivalence of behavior. Test tool calling, structured output (JSON schema and response formats), streaming, token limits, refusals and error codes against your own prompts, and compare the results with a fixed evaluation set before any production switch. Pay attention to model-specific chat templates, default sampling settings and reasoning output, which can change parsing. Record the vLLM version you test, since releases ship about every two weeks, and retest when you upgrade.
Where prompts are processed and who controls the stack
With Azure OpenAI, the processing location depends on the deployment type you choose and on what Microsoft's current terms say, so the first step is confirming that for your own deployments. With self-hosted vLLM behind a gateway, the same questions move to your infrastructure, where you decide the location, the identity model, the network boundary and the evidence you keep.
Processing location by deployment type
Azure OpenAI offers several deployment types, and they do not all process prompts in the same place. Global deployments may process prompts in any Azure region, while regional deployments keep processing in the region you select. Confirm in writing with Microsoft which deployment types each of your workloads uses, and record the region that applies to each. Treat the answer as a fact to verify, not an assumption from marketing material.
Self-hosting in your own environment
With vLLM, the inference servers run in your Azure subscription, in your datacenter or in another cloud you choose. You decide which region, account and network hold the GPUs, the model weights and the prompt logs. The main check is whether your data protection requirements and your hosting location agree, including where backups, monitoring data and gateway logs are written.
Single sign-on with Microsoft Entra ID
The LiteLLM admin UI supports single sign-on with Microsoft Entra ID, so administrators can sign in with the identity provider your organization already runs. From v1.76.0, SSO is free for up to 5 users, and an Enterprise license is needed beyond that. Confirm the user count you expect to administer before you plan licensing, and check how application users are authenticated separately from administrators.
Audit logs and RBAC on Enterprise
Audit logs, organization and team role-based access control, and IP allowlisting are part of LiteLLM Enterprise, not the open-source tier. Teams that need to show who changed a key, budget or model route should price this tier into the plan. Ask for the current Enterprise feature list in writing, since the boundary between tiers is the detail that most often changes over time.
Network isolation for vLLM nodes
vLLM's own security documentation states that communication between nodes in a multi-node deployment is insecure by default, without authorization or encryption. Place those nodes on an isolated network. The --api-key option does not cover every endpoint, so put a reverse proxy in front that allowlists only the paths meant for users, and keep the metrics and health endpoints off public networks.
Compliance evidence and who issues it
DeployOpen plans and implements the controls, configuration and evidence that an assessment will examine, including access logs, network diagrams and backup records. DeployOpen does not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those assessments, so ask your assessor early which evidence they will accept and in what format.
Throughput guarantees against PTUs and what self-hosting replaces
Provisioned throughput units (PTUs) buy reserved model capacity that Microsoft manages on your behalf, so your team trades a per-token bill for a capacity commitment and a deployment it does not have to size. Self-hosting on vLLM reverses that split. Capacity planning becomes your job, and the guarantee you get is only as strong as the hardware, configuration and monitoring you put behind it. Before comparing the two, confirm the current PTU terms, reservation length, minimum quantity and what happens to unused capacity directly with your Azure account team and in the Azure OpenAI pricing documentation, since reservation terms change and this page does not quote them.
On vLLM, sizing starts with GPU memory for the model weights plus the KV cache that holds active requests. A model that fits on one GPU is simple to run. Larger models need tensor parallelism across GPUs in one node, and pipeline parallelism when the model spans nodes. Multi-node runs use Ray or vLLM's own multiprocessing backend, and the vLLM security documentation states that node-to-node traffic is insecure by default, so those nodes must sit on an isolated network. For mixture-of-experts models, data parallel deployment is the option to evaluate. Your operations team will also need to watch KV cache usage, request counts, token throughput and latency histograms from the Prometheus-format /metrics endpoint, because those signals show when capacity is running out long before users see errors.
The cost side of this comparison is different from PTUs. vLLM has no paid edition, so the capacity cost is the GPU or accelerator hardware or cloud instances you pay for, plus the infrastructure around them and the engineers who operate, patch and on-call for the stack. Ask for a benchmark on your own prompts, output lengths and concurrency rather than relying on published throughput figures, because the result depends heavily on model choice, quantization and batch behavior. Only then can you compare sustained utilization against the PTU reservation, and decide whether the guarantee you need is worth the reserved capacity or better met by owning the capacity yourself.
What drives the cost of each option
How does Azure OpenAI pay-per-token pricing compare with provisioned capacity charges?
On Azure OpenAI you pay Microsoft for usage, either per token on pay-as-you-go deployments or for reserved provisioned throughput units (PTUs) that you hold whether or not traffic arrives. Per-token cost scales with how much you send, so it is mostly variable. Provisioned capacity is closer to a fixed monthly commitment that stays the same when traffic is low. Ask your account team for the rate for each deployment type, the reservation term, what happens to unused PTUs, and whether Batch or other discounts apply to your workloads. Confirm these in your own contract, since terms differ by agreement and region.
What do you pay for when you run vLLM yourself?
vLLM itself has no license fee, since it is Apache-2.0 licensed with no paid edition. You pay for the accelerator capacity that runs the model (GPU or other accelerator servers or virtual machines), the storage and network around them, and the people who install, operate and upgrade the stack. Those costs recur monthly or hourly, whether or not the hardware is busy. The main variable is how many accelerators your chosen model needs at your target latency and concurrency, so ask for a sizing that names the model, context length and peak load.
Why does GPU utilization decide whether self-hosting costs less?
Idle accelerators cost the same as busy ones, so the cost per request depends on how many hours a GPU serves real traffic. Low, bursty usage often favors per-token pricing, while steady, high-volume traffic can fill reserved hardware and lower the unit cost. Measure your token volume by hour and by peak, not only by month. vLLM exposes request counts, token throughput and KV cache use on its metrics endpoint, which gives the data needed to model utilization before you commit to hardware. Ask any implementation partner to show the utilization assumption behind the quote.
What does the gateway and monitoring layer add to the monthly bill?
A production gateway such as LiteLLM adds its own infrastructure. LiteLLM's production guide names PostgreSQL as the primary supported database, which holds virtual keys, budgets and spend data. Once more than one proxy instance runs, it requires Redis 7.0 or newer for shared rate limits, router state and cache. The guide suggests 1 vCPU and 4 GiB memory per worker as a starting point. Tracing and evaluation tools such as Langfuse add ClickHouse, Redis or Valkey and object storage. Ask for each database's sizing, backup and high availability design, and whether those services are counted in the quote or billed separately by your cloud provider.
Who is paid for on-call, upgrades and model changes after launch?
Running the stack is ongoing work that recurs for as long as you keep it. Your own engineers or a support retainer cover on-call rotations, patching the serving engine and gateway, watching vLLM releases that arrive about every two weeks, rotating keys, and testing replacement models before retiring old ones. On Azure OpenAI, Microsoft carries the hosting operations, but you still pay engineers to validate applications when models change. Ask who owns each task, the response expectations for incidents, and how upgrades are tested and documented. Retainers for ongoing support and upgrades are available from DeployOpen on a written scope.
What is a one-time implementation cost and what should it include?
Implementation is the one-time part of self-hosting. It covers architecture, installing vLLM and the gateway, network isolation for multi-node setups, authentication and reverse proxy rules, model evaluation on your own prompts, monitoring setup, backup, and the migration of applications from the Azure OpenAI endpoint to the OpenAI-compatible API. Documentation and evidence for compliance reviews may also be part of the work. Most engagements are fixed-scope, so ask for written deliverables, acceptance criteria, timelines, assumptions about your environment, and what is excluded, such as hardware purchase or model licensing review.
What should I ask for in a written quote for self-hosting?
Ask for every recurring and one-time line separately so you can compare them with your current Azure OpenAI invoices. A complete quote names the accelerator type and count, the cloud or datacenter where it runs, the storage and network charges, the PostgreSQL and Redis services, the gateway and monitoring licenses (noting that some LiteLLM and Langfuse features need enterprise keys), the support hours, and the assumed token volume and peak load. Ask which costs change if traffic doubles and which items you would still pay if the system is switched off for a quarter.
Does self-hosting remove the cost of model version churn?
It shifts the cost rather than removing it. Azure OpenAI deployments follow Microsoft's retirement schedule, and migrating provisioned workloads costs revalidation time on your side. With self-hosted open-weight models, you choose when to change versions, but you must test and approve each change yourself. Forecast the number of model changes per year, the test set size, and the review effort for each. Ask any partner how regression testing is scoped and whether it is included in the fixed fee or sold as a retainer.
What a move from Azure OpenAI involves
The sequence turns a set of Azure OpenAI deployments into self-hosted open-weight models behind one gateway, with each stage ending in evidence that the replacement is ready to carry the traffic. Teams can stop at any stage if the evaluation shows the workload should stay where it is.
Inventory deployments, models and API versions
The platform team lists every Azure OpenAI resource, deployment, model version, API version and deployment type, including Standard, provisioned and Batch use, and records which applications call each one and with what request shape. Application owners confirm their prompts, tool calls and response formats. The output is a single register of endpoints and their owners, with each provisioned deployment's contract or retirement date noted so the cutover plan has fixed dates to work back from.
Size and benchmark GPU capacity
Engineers choose candidate open-weight models and test them on representative GPU hardware using vLLM, measuring throughput, latency under concurrent load and KV cache behavior against the peak traffic from the inventory. Hardware vendors and cloud providers should be asked to confirm supported accelerators and quotas for the chosen region. The output is a capacity sheet that states which model runs on which hardware at which concurrency, and what headroom remains for growth.
Stand up vLLM behind a secured gateway
Platform engineers deploy vLLM on the chosen infrastructure and place a LiteLLM gateway in front of it, so applications receive one OpenAI-compatible endpoint with virtual keys and budgets. vLLM's own documentation says the --api-key option does not cover every endpoint, so a reverse proxy should allowlist only the routes meant for users, and node-to-node traffic in multi-node runs should sit on an isolated network. The output is a tested deployment where unlisted routes return errors from the proxy and not from the model server.
Evaluate replacement models on your own prompts
Application owners run a fixed set of their own prompts, including edge cases and sensitive content, against Azure OpenAI and each candidate model, then score outputs with the people who own the business outcome. Evaluation criteria cover accuracy, format compliance, refusal behavior and tool calling. Public benchmark rankings do not replace this step. The output is a signed-off scorecard per application that names the approved model and version and the cases where it falls short.
Run parallel traffic with fallbacks
The gateway sends a small share of production traffic to the self-hosted models while Azure OpenAI continues to serve the rest, with fallback routing to Azure OpenAI configured for any failure. Operations staff watch error rates, latency, token spend and output quality through the metrics vLLM exposes and the gateway's logging. The output is a period of recorded production behavior that shows the self-hosted path meets the agreed service levels, along with a documented rollback procedure.
Cut over provisioned workloads before contract dates
Teams move provisioned workloads to the self-hosted endpoints in an order set by the register, starting with lower-risk applications, and keep the Azure OpenAI deployment available until the contract or retirement date has passed for that workload. Each application switches its base URL and model name in configuration, not in code, if its client already uses the OpenAI request format. The output is a change record per application showing the cutover date, the approved model version and the decommissioning date of the old deployment.
Set up upgrades on the vLLM release cadence
Operations staff establish a routine for testing vLLM and gateway releases in a staging environment, rerunning the evaluation set and the capacity benchmark before promoting a change. vLLM publishes minor releases roughly every two weeks according to its release history, so the routine needs a named owner and a schedule that fits the organization's change process. The output is a documented upgrade runbook and a regression suite, and a record of each upgrade with its evaluation results.
When staying on Azure OpenAI is the right call
Staying on Azure OpenAI is the right call when the workload depends on specific hosted models, when Microsoft should operate the capacity, or when the team has no GPU operations skills to staff a serving stack. Frontier hosted models are often the best fit for complex reasoning or multimodal tasks, and a self-hosted open-weight model may not match them on your prompts. Before deciding, run your own evaluation set against the candidate open-weight models and confirm which Azure OpenAI model and version your applications actually call. If the quality gap is material for your use case, the migration case weakens.
Low or spiky request volume is the second strong argument for staying put. Reserved capacity on a provisioned deployment, or pay-per-token Standard deployments, only pay off at sustained utilization, and GPU servers sit idle between peaks. Running vLLM on dedicated hardware also adds on-call duties for patching, scaling, driver updates and capacity planning. Ask your team who would own those tasks at 2 a.m. If the answer is unclear, the operating burden will likely outweigh the savings. Your existing Azure contract, network controls, Entra ID roles, policy assignments and logging may also make Azure OpenAI the lower-friction path for security and procurement review.
A hybrid pattern is often the practical middle ground. A gateway such as LiteLLM can expose one OpenAI-compatible endpoint, route regulated or high-volume traffic to vLLM in your own subscription or data center, and send specialized or bursty traffic to Azure OpenAI. Applications change only their base URL and model alias, and the gateway holds the routing rules, fallbacks and budgets. Decide per workload, record the data classification that justifies each route, and confirm with Microsoft and your compliance team which deployment types and data handling terms apply to the Azure side.
Questions buyers ask before choosing vLLM
How much effort does switching from Azure OpenAI to vLLM take?
The effort depends mostly on how many applications call Azure OpenAI, how many deployments and model versions they use, and how much your prompts depend on model-specific behavior. vLLM serves an OpenAI-compatible API, so the client change is often small when applications already use the OpenAI request format. The larger work is standing up GPU capacity, the gateway, monitoring and identity, then re-running your own evaluations on the replacement model before any traffic moves. Ask for an inventory of deployments, API versions and workloads before estimating anything.
Can our team run vLLM without outside help?
Yes, if your platform engineers already operate GPU nodes, Kubernetes and model-serving stacks and have on-call coverage for them. Teams without that experience usually need help with GPU driver and runtime setup, capacity planning, network isolation for multi-node runs and monitoring. DeployOpen plans, implements, hardens, documents and supports self-hosted platforms in your own infrastructure, and most engagements are fixed-scope with deliverables agreed in writing before work starts. Ongoing support and upgrades are available as retainers.
How long does a migration take?
No general duration applies, because the timeline is set by the number of workloads, the replacement models you evaluate, the GPU procurement lead time in your environment and the approval steps for changes to production systems. Teams with one application and a clear evaluation set move faster than teams with many provisioned deployments and regulated sign-off. Ask any partner for a written schedule tied to a defined scope, and ask what dependencies, such as hardware delivery or security review, sit on the critical path.
What does the work need from our staff?
Plan for people who can own four areas: model serving on GPUs, the gateway layer, observability and security operations. vLLM exposes Prometheus metrics, so someone must define dashboards and alerts. A gateway such as LiteLLM needs PostgreSQL and, once more than one proxy instance runs, Redis 7.0 or newer. Someone also has to own model version upgrades, which arrive on a regular release cadence. Ask whether your staff will run these components or whether the work is shared with a partner under a retainer.
Is vLLM free to license?
vLLM is licensed under Apache-2.0 and has no paid edition, so the license itself costs nothing. The cost of running it sits elsewhere: accelerator capacity (GPUs or other supported hardware), the surrounding infrastructure such as storage, networking and gateway databases, implementation work, and the engineers who operate it. If you add LiteLLM, its core is MIT licensed, but code under its enterprise directory uses a separate license, and features such as SSO beyond the free user limit need an Enterprise license. Confirm the license of every component in your stack before you sign.
Can open-weight models match Azure OpenAI quality?
The honest answer depends on your task, and no general claim applies. Open-weight models differ in quality across tasks, languages, context lengths and output formats, and a model that performs well on public benchmarks may still fail your own prompts. Build an evaluation set from real examples of your workloads, score the candidate models and the current Azure OpenAI output on the same cases, and review failures with domain experts. Model choice and version pinning then become decisions you document and control.
How do we secure vLLM endpoints?
The --api-key option protects only some paths, and the vLLM security documentation warns against relying on it alone. Its docs recommend a reverse proxy that allowlists only the endpoints meant for users, so administrative and health paths such as /metrics and /pause stay internal. Multi-node deployments send traffic between nodes without authorization or encryption by default, so isolate those nodes on a private network. Add TLS, network policy, secret handling for keys and audit logging at the gateway, then have the design reviewed by your security team.
How DeployOpen helps evaluate, implement or migrate
DeployOpen plans, implements, hardens, backs up, upgrades and documents self-hosted vLLM and LiteLLM inside your own cloud subscription, data center or other environment you control. The work is delivered under a fixed scope agreed in writing before it starts, with scope, deliverables and timelines set out in that document. For the Azure OpenAI replacement, a typical written scope covers the vLLM serving nodes, the LiteLLM gateway with virtual keys and budgets, network placement of the nodes, the reverse proxy that exposes only user-facing endpoints, and the migration of applications to the OpenAI-compatible API. Ask any provider, DeployOpen included, to list each of these as a numbered deliverable so you can see what is in and out.
Security testing covers the AI system and its endpoints, not only the infrastructure. DeployOpen runs security testing of applications and APIs, cloud and infrastructure, and AI systems and agents, including red teaming. For this topic, that means testing the vLLM endpoints you choose to expose, checking that unlisted paths such as /metrics, /health and /pause are blocked at the proxy, confirming that --api-key is not the only control, and verifying that multi-node traffic runs on an isolated network. Retainers are available for ongoing support, upgrades when new vLLM minor releases ship, and continuous testing after go-live. Agree in writing which of these are recurring and what each covers.
DeployOpen prepares systems, controls and evidence for compliance assessments. It does not issue audit reports, attestations or certifications; independent auditors and certification bodies do that work. The evidence package typically includes data flow diagrams, the location of inference and logs, access and key management records, change records for model and version upgrades, and backup and restore test results. Before you commit, confirm with your auditor which evidence they expect, and ask DeployOpen to map each item to the control it supports.
