Azure OpenAI cost vs self-hosted vLLM: where the break-even sits
vLLM has no license fee: it is Apache-2.0 licensed with no paid edition, so the comparison with an Azure OpenAI bill turns on GPU utilization and staff cost, not on a per-token price. The costs you would carry are the GPU or other accelerator capacity, the surrounding infrastructure (gateway, storage, networking, monitoring), the implementation work to stand it up, and the engineers who operate it. Each of those is paid to a different party: GPU capacity to your cloud or hardware vendor, implementation to a partner or your own team, and operations to your staff or an on-call rota.
On the Azure side, three commercial shapes suit different traffic. Pay-per-token Standard deployments carry no capacity commitment and fit spiky or low volume work. Reserved provisioned throughput units (PTUs) fit steady, predictable load where you pay for reserved capacity whether or not you use it. Batch processing is priced at 50% of the standard rate, which suits work that can wait for results, such as offline evaluation or document backfills. Ask Microsoft for current rates in your region and agreement, because they change and are not stated here.
Self-hosting pays off only when your own GPUs stay busy. Idle accelerators cost the same as busy ones, so the break-even depends on sustained, predictable token volume and on the share of the year those GPUs serve real requests. Before comparing quotes, measure your current peak and average tokens per minute, the batch share of your workload and the hours of on-call cover you would need. Then ask each vendor or your cloud provider for a written quote covering GPU hours, redundancy for a second node, and the people time to operate the stack.
Which cost components sit on each side and when they are paid
This table sets Azure OpenAI pay-per-token, reserved PTU capacity and Batch pricing against the hardware, gateway, monitoring and people that a self-hosted vLLM stack needs. Read each row as a cost line, then check whether it is one-time or recurring and what drives it up. The main takeaway is that the open-source software carries no license fee, so the comparison turns on utilization, hardware and the engineers who operate the stack.
| Cost component | Azure OpenAI side | Self-hosted vLLM side | One-time or recurring |
|---|---|---|---|
| Model usage or capacity | Pay-per-token Standard deployments, reserved PTUs, or Batch at 50% for asynchronous work; confirm current rates in the Azure pricing pages | No model fee; open-weight model weights are downloaded and stored on your infrastructure | PTUs and per-token charges recur; weights are one-time transfer plus storage |
| GPU or accelerator hardware | Included in the service price; you do not buy or host GPUs | GPU virtual machines or on-premises servers sized for the chosen model and concurrency; vLLM runs on NVIDIA CUDA, AMD ROCm and Intel XPU | Purchase is one-time; cloud GPU rental and power or colocation recur |
| Gateway and virtual keys | Azure API management and key handling are part of the Azure resource; extra gateway tooling is optional | LiteLLM open-source proxy (MIT) with PostgreSQL for virtual keys and Redis 7.0 or newer once more than one instance runs; SSO beyond 5 users and some controls need a LiteLLM Enterprise license | Setup is one-time; database, cache and any Enterprise license recur |
| Monitoring and metrics | Azure Monitor and diagnostic logs billed by the platform; confirm retention and volume charges | Prometheus scraping vLLM /metrics and Grafana dashboards, both self-hosted with storage and compute | Initial dashboards are one-time; storage, retention and upkeep recur |
| Implementation and migration | Application changes to Azure endpoints and deployment names, with internal or partner effort | Model evaluation on your own prompts, vLLM and gateway deployment, network isolation and cutover, usually through a written, fixed scope | Largely one-time, but migrations are paid in full before the first production request |
| On-call and operations | Azure carries the service operations; your team handles incidents in the application layer | Your engineers or an outsourced retainer cover GPU failures, node health, scaling, patching and on-call rotations | Recurring; usually the largest cost line when utilization is low |
| Upgrades and revalidation | Model retirements and version changes arrive on the provider's schedule and require revalidation of your prompts | vLLM ships new minor releases about every two weeks, so you choose when to upgrade and must retest; LiteLLM supports the four most recent stable minor versions | Recurring; each upgrade cycle needs regression testing |
| Internal skills | Azure administration and OpenAI API knowledge, usually already present in cloud teams | GPU operations, Kubernetes or Helm, PostgreSQL, Redis and inference tuning, which may need hiring or training | Hiring and training are one-time; retained expertise recurs |
What moves the comparison most
Six drivers decide whether reserved Azure OpenAI capacity or self-hosted vLLM costs less for your workload. Most of them change the fixed cost of running GPUs, so the same token volume can favor different options in different organizations.
Sustained GPU utilization and peaks
Self-hosted GPUs are paid for whether or not they serve tokens, so the cost per token falls as sustained utilization rises. Spiky traffic leaves capacity idle between peaks, which favors pay-per-token Standard deployments or Batch for work that can wait. Measure hourly token throughput over several weeks, including the busiest hour, before sizing either option. vLLM's /metrics endpoint exposes token throughput and KV cache use, which give you the data to do this.
Token volume and model size
Larger models need more GPU memory and more accelerators per replica, so each token costs more to serve on self-hosted hardware even when throughput is high. Smaller open-weight models can serve the same task on fewer GPUs, which lowers the fixed cost. Input and output token counts are priced differently in Azure OpenAI, so separate them when you estimate. Test candidate models on your own prompts before deciding that a larger model is required.
Latency tolerance and commitment type
Reserved PTUs suit steady, latency-sensitive workloads that need predictable throughput. Work that tolerates delay, such as nightly summarization or bulk classification, can use Batch pricing or on-demand capacity instead. The more of your traffic that tolerates delay, the less reserved capacity you need to buy. Ask Microsoft for the current PTU terms and the Batch discount in writing for your region and model, since these change over time.
Availability and multi-node operation
A high-availability target usually means at least two replicas in separate failure domains, plus spare capacity for maintenance and upgrades. Each added replica is more GPU cost, and models too large for one node need multi-node serving. vLLM supports tensor parallelism within a node and pipeline parallelism across nodes, but its security docs say node-to-node traffic is insecure by default and must sit on an isolated network. Include that networking and the spare capacity in the comparison.
Applications and teams sharing the gateway
A gateway shared by many applications spreads fixed infrastructure and on-call cost across more consumers, which improves the self-hosted case. Each team also adds budgets, keys and model access rules to administer. LiteLLM virtual keys with per-team budgets and rate limits require PostgreSQL, and multiple proxy instances require Redis 7.0 or newer. Count the teams that will need separate spend tracking, since chargeback is often the reason shared gateways get built.
Existing platform engineering skills
Self-hosting moves operating work to your staff: GPU driver and CUDA upgrades, Kubernetes manifests, monitoring, certificate rotation and vLLM version upgrades. Teams that already run Kubernetes and GPU nodes add less cost than teams starting from scratch, who should budget for hiring or outside support. Ask whether your on-call rota can cover model serving outside business hours, and price that coverage explicitly, because it recurs for as long as you self-host.
How to build the comparison with your own numbers
This sequence turns the cost components into a comparison built on your own traffic, hardware quotes and staffing, so you can see at what utilization reserved capacity or self-hosting starts to pay off. Each step produces a figure you can check against your invoices and quotes.
Export token volumes from Azure usage data
The cloud or finance team exports usage for the last full billing period from the Azure portal or cost management exports, then records input and output tokens for each Azure OpenAI deployment and each model. Use the deployment name and model version as keys so that traffic can be matched to the applications that generate it. The step is done when you have a table of tokens per deployment per model per day or per hour, with the peak hour identified. If your usage data does not show tokens at this level, ask Microsoft account support or your Azure administrator which metrics and exports are available before you estimate anything.
Separate pay-per-token, PTU and Batch traffic
Split the exported volumes into three groups: Standard pay-per-token calls, calls served by provisioned throughput (PTU) deployments, and asynchronous calls submitted through the Batch API. Batch traffic is billed at a different rate from Standard, so it must be priced on its own line rather than mixed into per-token totals. Ask your Azure account team to confirm current rates for each group in writing, because these figures change and should come from the quote, not from this page. The step is done when each group has its own token total and its own unit price in your sheet.
Size GPU capacity with a benchmark on your prompts
Deploy an open-weight candidate model with vLLM on a test GPU host and run a load test that replays a representative sample of your real prompts, including long inputs and the output lengths you see in production. Measure throughput and latency at your target concurrency, using the Prometheus metrics that vLLM exposes at its metrics endpoint. Repeat the test for average and peak load to get the GPU count for each. The output is a table showing how many accelerators you need for each traffic level, with the model, context length and quantization recorded. Treat the result as specific to that model and workload, and retest when either changes.
Price GPU hours from written cloud or hardware quotes
Request written quotes for the GPU instances you would run, either from your cloud provider for on-demand and reserved commitments or from your hardware vendor for owned servers, including power, rack space, support and replacement terms. Note what is one-time (hardware purchase, implementation, initial hardening) and what recurs (instance hours or amortization, support, power, facility). Ask for the commitment length and what happens to unused reserved hours. The step is done when every GPU option has an hourly or monthly cost, a commitment term and a list of exclusions, all in writing.
Add gateway, monitoring, storage and staffing lines
Add the cost of the components that surround the model servers: the LiteLLM gateway and its PostgreSQL and Redis dependencies, the monitoring and logging stack, object storage for logs or weights, load balancers, and the engineers who run the platform. Staffing is usually the largest recurring line that teams underestimate, so include the on-call rota, upgrade effort for vLLM and LiteLLM releases, and security patching. Record which of these lines are internal salary time, which are vendor or consultant fees, and which are cloud charges. The step is done when each surrounding component has an owner and a monthly figure, or an explicit note that it is excluded.
Model utilization at several levels
Build a sheet that compares the total Azure cost, including reserved PTU capacity where it applies, with the self-hosted total at several utilization levels, such as low, average and peak use. Show the cost of paying for reserved capacity that sits idle when traffic falls, and the cost of on-demand GPUs when traffic is steady. The break-even point is the utilization at which the self-hosted total equals the Azure total for the same work. Mark where your current traffic falls on the curve. The step is done when you can point to the utilization level at which each option becomes cheaper, and state the assumptions behind that level.
Validate the model with a written review
Have the platform owner, the finance approver and the security or compliance lead review the sheet, the benchmark results and the quotes together. The review checks that every number traces to an export, a test log or a written quote, and that the staffing assumptions match the team you actually have. Record any figure that is still an estimate and the date it must be confirmed. The step is done when the sheet has a signed-off version, a list of open items and a named owner for each open item.
How a self-hosted vLLM stack is put together
The cost of a self-hosted vLLM stack sits in five components, and the GPUs are only one of them. vLLM serves the open-weight model on GPU capacity and exposes OpenAI-compatible endpoints such as /v1/chat/completions and /v1/embeddings, so applications written for the OpenAI API can point at it. vLLM has no paid edition, so its license cost is zero, but the accelerator hardware, the node operating system, the network and the people who run them are all yours to fund. vLLM also publishes Prometheus-format metrics at /metrics, covering request counts, token throughput, latency histograms and KV cache use. Those metrics feed the monitoring stack you already run, and that stack is a line item to confirm, not a free add-on.
The gateway is where most of the operating effort lands. LiteLLM sits in front of vLLM and issues virtual keys with per-key, per-user and per-team budgets, TPM and RPM limits, and model restrictions. Those controls need a PostgreSQL database. Once more than one proxy instance runs, LiteLLM's production guide requires Redis 7.0 or newer for shared rate limits, router state and cache, so plan for managed or self-run PostgreSQL and Redis with backups, patching and failover. Ask your team which of these they can operate in-house and what a managed equivalent would cost in your environment.
vLLM's own --api-key option does not protect every endpoint. According to the vLLM security docs, it covers /v1, /v2, /inference and /cohere, but not /invocations, /pause, /resume, /abort_requests, /metrics or /health, and the docs advise against relying on it alone. Their recommended pattern is a reverse proxy that allowlists only the endpoints meant for users. Budget for that proxy, its TLS certificates and its configuration review as a recurring part of the stack, and confirm during design that the metrics and control endpoints are reachable only from your monitoring network.
Questions buyers ask before comparing costs
Is vLLM free to run in our environment?
vLLM is open-source software under the Apache-2.0 license and has no paid edition, so the license itself costs nothing. The money goes to the GPU or other accelerator capacity it runs on, the surrounding infrastructure such as storage, networking and the gateway, and the people who install, secure, upgrade and operate it. Those are the lines to price. Confirm with your own cloud or data center team what GPU capacity, reservation terms and support contracts apply in your region, because those vary by provider and are not part of the software license.
What does self-hosting cost in staff time and ongoing effort?
Staff cost is usually the largest recurring line after hardware, and it covers more than initial setup. The work includes standing up the inference servers, placing a reverse proxy in front of them, wiring metrics into your monitoring stack, patching, and handling on-call for GPU failures and capacity spikes. vLLM ships new minor releases about every two weeks, so the upgrade cycle is continuous rather than occasional. Ask any vendor or implementation partner to break its quote into one-time build work and recurring operations, with the number of engineers covered and the on-call model stated.
When do reserved PTUs beat pay-per-token pricing?
Reserved capacity tends to win when your traffic is steady and high enough to keep provisioned throughput busy for most hours. Pay-per-token pricing wins when usage is spiky, unpredictable or low. Compare the reserved hourly commitment against your measured tokens per hour at peak and average load, including how much headroom you need for growth. Check the current Azure OpenAI pricing page for PTU rates, Standard token rates and the 50% Batch discount terms, since these change and this page does not state them. Also ask whether the reservation is one-time or recurring, and whether it can be scaled down.
How much utilization justifies owning GPUs for vLLM?
Self-hosting pays off only when GPUs stay busy enough that their fixed cost is spread across many useful tokens. The break-even point depends on the model size, the GPU type, the throughput you achieve under your real prompt lengths, and how many hours you must keep hardware on standby for failover. Measure this before deciding: vLLM exposes Prometheus-format metrics at /metrics covering request counts, token throughput and KV cache use, which gives you the observed numbers to model. Build the comparison from those measurements, not from vendor benchmarks run on different prompts.
Can our platform team run vLLM without outside help?
A capable platform team can run vLLM, but the tasks that trip teams up are the ones outside the model server itself. vLLM's security documentation states that node-to-node communication in multi-node deployments is insecure by default, so nodes must sit on an isolated network. Its --api-key option does not protect every endpoint, so a reverse proxy that allowlists only user-facing routes is needed. Teams also need GPU drivers, Kubernetes expertise if they use it, and a gateway such as LiteLLM with PostgreSQL and Redis. Assess these skills honestly against your staff, then decide which parts to keep in-house.
What should a DeployOpen quote include?
DeployOpen does not publish a price list. Engagements are fixed-scope, so a quote follows from a written scope with agreed deliverables and timelines before work starts. Ask that the quote separate planning, implementation, integration with your applications and gateway, hardening, backup, documentation and any compliance evidence preparation. Ask for ongoing support, upgrades and testing to be priced as a retainer apart from the build. State which infrastructure costs are yours to pay directly, such as GPU capacity in your own accounts, so the total is comparable with your current Azure OpenAI spend.
How often do upgrades recur, and what do they involve?
Upgrades recur continuously. vLLM publishes new minor releases roughly every two weeks, and LiteLLM maintains support for the four most recent stable minor versions in its open-source and Enterprise deployments. Each upgrade means testing the new version against your prompts, checking API compatibility, rolling it out and confirming metrics still flow. Ask any provider to describe the upgrade cadence it commits to, the test evidence it produces for each change, and whether upgrades are included in a retainer or billed per event.
Does LiteLLM require a paid license for production use?
The core LiteLLM gateway is MIT licensed, and everything outside its enterprise directory uses that license, so the open-source proxy can run without a license fee. A paid license applies to content under the enterprise directory. LiteLLM's documentation states that SSO is free for up to 5 users from v1.76.0, and an Enterprise license is needed beyond that. Enterprise also adds SCIM, JWT authentication, audit logs, organization and team RBAC, and secret manager integrations. Check the LiteLLM license terms and your user count before you decide whether those features are required.
How DeployOpen scopes a fixed-price vLLM implementation
DeployOpen plans and implements self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For Azure OpenAI cost vs self-hosted vLLM, we start from your current token volumes, PTU or Batch commitments and operating costs, then agree a written scope before any build work starts.
How we scope
- We inventory current Azure OpenAI deployments, token volumes and reserved PTU commitments before modeling any alternative.
- The written scope names which models, gateway features, GPU capacity and monitoring components are in and out of the engagement.
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts; a quote follows from that scope.
How we build
- We deploy vLLM behind a gateway in your environment, with the OpenAI-compatible endpoints tested against your existing application calls.
- We restrict exposed vLLM endpoints with a reverse proxy allowlist, because the API key option alone does not cover every route.
- We benchmark throughput and utilization on your own prompts so the capacity sizing reflects your workload rather than generic assumptions.
How we hand over
- You receive documentation of the architecture, GPU sizing assumptions, cost model inputs and the utilization thresholds we measured.
- Runbooks cover scaling, model upgrades, key rotation, backups of gateway database state and monitoring alerts.
- Handover includes a review session where your engineers operate the stack and confirm each deliverable against the written scope.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers after handover.
- We can track vLLM and gateway release changes and plan upgrade windows that your team approves.
- Ask any provider to confirm in writing which costs stay with your team, including GPU capacity, cloud accounts and on-call staffing.
