Amazon Bedrock cost vs self-hosted vLLM

Does self-hosting vLLM cost less than Amazon Bedrock?

Self-hosting vLLM tends to cost less than Amazon Bedrock only when GPU utilization is sustained and predictable. When traffic is bursty, utilization is low, or the team that operates the cluster is thin, the same hardware and people can cost more per token than Bedrock's pay-per-use pricing. vLLM itself carries no license fee, since it is licensed under Apache-2.0 and has no paid edition, so the comparison is not about software cost. It is about GPU or other accelerator capacity, the surrounding infrastructure, implementation work and the people who run inference. On the Bedrock side, the comparison is token charges on-demand, Flex, Priority or batch, plus any Provisioned Throughput commitment and, if you use it, Custom Model Import billing.

Do not model Bedrock costs from memory or from third-party summaries. Take the current per-token and commitment rates from the Bedrock pricing page, then confirm the rates that apply to your account in your own AWS invoices, because discounts, regions and service tiers change what you actually pay. Split your last three to six months of Bedrock spend by model, pricing tier and commitment, and record your peak and average tokens per minute. Those numbers set the utilization threshold at which a dedicated GPU fleet starts to win. Your GPU instance pricing should come from the same billing source and the same region, so the comparison is like for like.

Then price the costs Bedrock absorbs. Ask your implementation partner for a written quote covering the gateway, such as LiteLLM with PostgreSQL and Redis, monitoring through the Prometheus metrics that vLLM exposes at /metrics, a reverse proxy that restricts endpoints, model rollout and patching, and on-call coverage. Implementation is largely one-time, while GPU capacity, storage, support and staff time recur. Ask for the cost of keeping spare capacity for failover and of re-running evaluations after each vLLM minor release, since those are the costs that most often push the total above the Bedrock estimate. Request a utilization scenario and a break-even calculation built on your own invoices, not a list price.

Which cost components sit on each side

This table places each cost component on the Amazon Bedrock side or the self-hosted vLLM side and marks whether it is paid once or recurs. vLLM carries no license fee, so the self-hosted column is almost entirely capacity and people, while the Bedrock column is mostly metered usage and commitments. Use it to decide which lines to request in a written quote before comparing totals.

Cost component	Amazon Bedrock side	Self-hosted vLLM side	One-time or recurring
On-demand, Flex, Priority and batch tokens	Per-token charges that vary by tier and model, set by AWS pricing for each model	Not applicable; cost shows up as GPU hours and operations instead	Recurring, scales with token volume
Provisioned Throughput commitments	Committed capacity billed whether or not it is used, with terms set by AWS	Dedicated GPU capacity you size and keep warm, which you pay for continuously	Recurring, committed term
Custom Model Import billing	Billed per model copy and minute of active use under AWS terms	Replaced by the GPU instances or accelerators that host your own weights	Recurring while the model is loaded
GPU instances or accelerators	Hidden inside the per-token or commitment price	Your largest line item: instance or hardware purchase, plus idle capacity you must budget for	Hardware is one-time; cloud instances are recurring
vLLM license	Not applicable	Zero; vLLM is Apache-2.0 with no paid edition	None
LiteLLM gateway, PostgreSQL and Redis	Not applicable; AWS handles request routing	Compute for gateway pods, a managed or self-run PostgreSQL, and Redis 7.0 or newer once more than one proxy runs	Recurring infrastructure
Monitoring with Prometheus and Langfuse or Opik	CloudWatch and any third-party tooling you add, billed separately	Storage, compute and retention for Prometheus, plus Langfuse dependencies (PostgreSQL, ClickHouse, Redis or Valkey, object storage) or Opik	Recurring, grows with trace volume
Implementation, migration and guardrail replacement	Mostly configuration of Bedrock features already in your account	Fixed-scope build of serving, gateway, monitoring and evaluation, plus cutover testing; confirm scope and fees in writing	One-time
Upgrades, support and internal skills	Model version changes and retirements handled by AWS within its lifecycle	Ongoing vLLM and LiteLLM upgrades, security patching, on-call coverage and staff who understand GPU serving	Recurring; ask whether support is a retainer

What moves the cost comparison

The gap between Bedrock token pricing and a self-hosted vLLM cluster depends on a few variables that move in opposite directions. Read each driver against your own traffic, model choice and team before comparing totals.

Sustained utilization versus bursty traffic

A self-hosted cluster is paid for whether or not it serves tokens, so its cost per token falls as utilization rises and climbs when traffic is spiky and the GPUs sit idle between peaks. Bedrock on-demand charges only for tokens consumed, which favors irregular workloads. Measure your hourly token volume over several weeks, including the peak hour, before you size a cluster. Ask whether you need capacity for peaks or for the average.

Bedrock tier per workload

Bedrock prices differ by tier: on-demand, Flex, Priority and batch. Latency-tolerant jobs such as offline evaluation or document processing may cost far less on a lower tier or batch than on the standard path, which narrows the gap to self-hosting. Interactive chat that needs predictable latency usually pays more on Priority. Confirm the current per-token rate for each tier and model in your Region in the AWS pricing pages, then split your workloads by tier before comparing totals.

Commitment term and size

Provisioned Throughput converts per-token spend into reserved model units for a chosen term. Commitments lower the unit cost only if the reserved capacity is used, and the term locks in spend even if you move models. Compare the committed model unit cost with the GPU hours you would otherwise rent for the same throughput. Ask AWS for the term options, cancellation rules and what happens to the commitment if the model reaches end of life.

Model size and GPU type

Cost per token on vLLM depends on how many GPUs one model replica needs, the memory left for the KV cache and the GPU generation you rent or buy. A larger open-weight model may need several accelerators per replica, multiplying the hourly cost, while a smaller model on fewer GPUs can be cheaper per token at the same quality bar. Benchmark your own prompts on two or three candidate models and GPU types. Confirm supported accelerators in the vLLM installation guide before you price hardware.

Multi-node networking and isolation

A model too large for one node needs pipeline parallelism across nodes, which adds interconnect hardware and more instances to pay for. The vLLM security documentation states that communication between nodes is insecure by default, with no authorization or encryption, so the cluster must sit on an isolated network. That isolation, along with private links and the inter-node fabric, adds platform cost that Bedrock does not charge you for. Estimate these items from your cloud provider's quotes rather than assumptions.

Existing platform and staff skills

Running vLLM in production means owning GPU drivers, autoscaling, the reverse proxy and allowlist in front of the API, metrics, upgrades and on-call. vLLM publishes minor releases about every two weeks, so someone must test and roll out upgrades on a steady schedule. If your platform team already runs GPU Kubernetes and an observability stack, the added labor is smaller. If not, count the hiring, training or outside support you need before comparing against Bedrock, which absorbs that operating work.

What the gateway and monitoring add to the self-hosted bill

The gateway is the component that most often turns a single vLLM server into a line item that needs a budget owner. The LiteLLM AI Gateway, which is MIT licensed outside its enterprise directory, gives you virtual keys, per-key, per-user and per-team budgets, TPM and RPM limits and spend tracking. Those controls are what replace the per-token metering you pay Bedrock for, so they are not optional if you want cost visibility. The catch is that virtual keys require a PostgreSQL database, and once more than one proxy instance runs, Redis 7.0 or newer becomes necessary for shared rate limits, router state and cache. Both are recurring infrastructure costs with their own patching, backup and high availability work that your team absorbs.

For sizing, the LiteLLM production guide suggests 1 vCPU and 4 GiB memory per worker. Treat that as a starting point to validate under your own traffic, not a capacity guarantee, and ask your platform team to load test the proxy with realistic prompt sizes and concurrency before you commit to a node count. On the monitoring side, vLLM exposes Prometheus-format metrics at /metrics on its API server, covering request counts, token throughput, latency histograms and KV cache use. Those metrics feed the GPU utilization decisions that drive your largest cost line, so scrape them from day one and confirm that the /metrics path is excluded from any public reverse proxy allowlist, as the vLLM security docs recommend.

Some governance features sit in the paid tier, and the bill should show them. SSO in the LiteLLM admin UI is free for up to 5 users from v1.76.0, and an Enterprise license is needed beyond that. SCIM provisioning, audit logs and secret manager integrations for AWS, Azure, Google, HashiCorp Vault and CyberArk are listed under LiteLLM Enterprise. If your auditors expect audit logs or your identity team requires SCIM, add a line for the Enterprise license to the quote and ask the vendor to confirm current pricing and terms in writing. Ask your implementation partner to separate that line from the infrastructure and labor estimates so you can see exactly what the license buys.

Costs teams forget on each side

Bedrock invoices show token prices, but commitments, network paths and add-ons can keep billing after a move. Self-hosting moves those costs onto your own capacity, operations and testing. Check both columns before comparing totals.

Provisioned Throughput terms

Provisioned Throughput is a commitment, so its term can run past the date you plan to cut over. Ask AWS for the exact commitment length, billing start and any early termination terms for each model unit you hold. Model the overlap period, when Bedrock and vLLM are both paid for, as a separate line in the business case. Confirm whether the commitment can be reduced or released before its term ends.

Network egress and endpoint charges

Traffic from Bedrock to your applications can incur data transfer charges, and PrivateLink or VPC interface endpoints bill for each endpoint and the data they process. Keep these in the Bedrock total, because a private connection to Bedrock adds recurring cost that a self-hosted endpoint inside your VPC does not need. Ask your network team to estimate traffic per application and per region before you compare totals.

Guardrail and evaluation add-ons

Bedrock Guardrails and any evaluation features are billed separately from base inference, so a guardrail applied to every request adds a second meter. Confirm which guardrail policies you enable, how each one is metered and whether your test runs are billed. On the self-hosted side, the matching cost is the engineering time to build or integrate those checks, which you should price separately.

Idle GPU capacity

A GPU instance bills for every hour it is running, whether or not requests arrive. Low-traffic periods, nights, weekends and regional gaps leave reserved accelerators idle, and that idle time becomes the largest variable in a self-hosted total. Ask for utilization data from your current Bedrock traffic, then size the cluster for peak load and decide whether to scale down or share nodes outside business hours.

On-call rota for inference

Someone must watch the vLLM service, respond to failed nodes, drained GPUs and stalled requests, and apply upgrades. vLLM releases minor versions about every two weeks, so a version watch and test cycle is recurring work. Count the people on the rota, their hours and the escalation path, then compare that with the operating work Bedrock absorbs. This cost is often the one missing from the original business case.

Reverse proxy for vLLM endpoints

The vLLM --api-key option does not cover /metrics or /health, and the vLLM security docs recommend a reverse proxy that allowlists only the endpoints meant for users. Building, configuring and maintaining that proxy, along with its TLS certificates and access logs, is work you must schedule. Confirm which internal paths your monitoring needs and restrict the rest at the proxy layer.

Model evaluation and security testing

Switching models means evaluating outputs on your own prompts, comparing quality and checking regressions for each model and each vLLM upgrade, which takes engineering time on every cycle. The endpoint also needs security testing for authentication bypass, exposed internal routes and prompt-level attacks before production use. Ask for the scope, the number of test cycles and the retest plan in writing, because these costs repeat with every model change.

How to build the comparison with your own numbers

This sequence turns Bedrock invoices and a GPU quote into a monthly and total cost comparison at several utilization levels, so the decision rests on your own usage rather than on a generic estimate. Each step names who supplies the input and how to confirm it is complete.

Export twelve months of Bedrock usage
Pull twelve months of billing and usage data from AWS Cost and Usage Reports or Cost Explorer, grouped by model, region and pricing tier. Separate on-demand, Flex, Priority and batch token charges, and list any Provisioned Throughput commitments and Custom Model Import charges as their own lines. Input source: your AWS billing account, not a vendor quote. Done when every line item on the invoices maps to one model and one tier and the monthly totals reconcile to the invoices.
Measure peak and average token throughput
From application logs, gateway logs or CloudWatch metrics, measure input and output tokens per second for each model, both at the daily peak and as a monthly average. Record how many hours per day the peak occurs, since that determines how much GPU capacity sits idle. Input source: your own telemetry. Done when you have a peak and an average figure per model and a note on the traffic pattern behind them.
Price GPU capacity from a quote
Size the GPU fleet needed to serve the measured peak with headroom for failover and upgrades, then request prices for that exact configuration from your cloud provider's pricing calculator or from a hardware vendor. Ask whether quotes are on-demand, reserved or committed, and what the commitment term and exit terms are. Input source: vendor quotes. Done when you have a monthly cost for the fleet at both the on-demand and the committed rate.
Add gateway, monitoring and storage lines
Add separate line items for the components around vLLM: the LiteLLM gateway and its PostgreSQL and Redis dependencies, the monitoring stack that collects vLLM's Prometheus metrics, log and trace storage, load balancing and the reverse proxy that limits exposed endpoints. Price each from cloud or hardware quotes, and note which are fixed and which scale with traffic. Input source: vendor quotes for infrastructure, plus your own retention requirements. Done when every supporting component has its own monthly figure.
Estimate operating hours from team records
Use your own ticket history, on-call rota and change records to estimate the hours per month spent on upgrades, patching, capacity changes, incident response and model evaluation for the inference platform. Convert those hours to cost using your internal loaded rates, and state which share is recurring and which is occasional. Input source: your own team's records. Done when you have a monthly operating hours figure with the work it covers listed beside it.
Add one-time implementation and migration costs
List one-time costs: design, deployment, hardening, evaluation of replacement models on your own prompts, cutover work and any partner scope. Request fixed-scope quotes from implementation partners, including DeployOpen, based on a written scope, and ask each to separate deliverables from ongoing support. Input source: written quotes and your internal estimates. Done when one-time costs are a single total with a clear list of what it buys.
Compare cost at several utilization levels
Build a table with Bedrock cost and self-hosted cost per month and over the planning horizon, calculated at several utilization levels such as your measured average, your measured peak and a lower and higher case. Show the break-even point where the self-hosted total falls below Bedrock, and mark which figures rest on vendor quotes and which on your telemetry. Done when a reviewer can change one input and see the effect on the break-even point.

Who runs inference in each option

In the managed model, AWS operates the model hosting capacity, endpoints, patching of the serving stack and model version lifecycle. You pay AWS per token on demand, or through reserved or priority tiers, and your team does not select GPU instances, install drivers or tune batch schedules for the model itself. Your costs are primarily usage-driven and recur with traffic, and your internal effort centers on integration, IAM policy, guardrails and cost monitoring.

Who is paid: AWS bills for tokens, provisioned capacity or custom import units, so spend follows consumption and commitments.
Staffing: a small application and cloud team can run the integration, but you still need people to own IAM, quotas and spend alerts.
Operations absorbed: AWS handles endpoint availability and model hosting, though you rely on its published lifecycle dates.
Limited control: you cannot pin a model version beyond what the lifecycle allows or choose the hardware.
Kubernetes options: none, since you do not operate a cluster for inference.

Before you decide, confirm current per-token rates, commitment terms and model lifecycle dates directly in the AWS pricing and documentation pages for your region and models.

Questions buyers ask before comparing costs

Does vLLM have a license fee?

No. vLLM is licensed under Apache-2.0 according to its GitHub repository, and it has no paid edition. What you pay for is the GPU or other accelerator capacity, the surrounding infrastructure (networking, storage, load balancing and monitoring), the implementation work and the people who operate the service. Bedrock's per-token and capacity charges are a different kind of cost: they bundle hardware, operations and model access into one bill. When you compare the two, separate the license line (zero for vLLM) from the infrastructure and labor lines, which is where the real difference sits.

How long does a cost comparison take and what drives the effort?

The effort depends on how many models you run, how much of your Bedrock usage is on-demand, Flex, Priority, batch, Provisioned Throughput or Custom Model Import, and how well your team already tracks token volume by application. Usage data is usually the slowest input to collect. Ask any provider to scope the comparison in writing, listing the workloads, the traffic profile (peak and average), the data retention requirements and the models in scope, so that the output is a worked model you can rerun rather than a one-off estimate. Duration should be agreed in that written scope.

What skills does the team need to run vLLM and LiteLLM?

Plan for four skill areas: Kubernetes or container operations for GPU node pools, GPU and model-serving tuning (parallelism, memory and quantization choices), API gateway operations including PostgreSQL and Redis, and security engineering for network isolation and authentication. vLLM's own security documentation says node-to-node traffic in multi-node deployments is insecure by default and recommends an isolated network and a reverse proxy in front of user-facing endpoints. LiteLLM's production guide names PostgreSQL as the primary supported database and requires Redis 7.0 or newer once more than one proxy instance runs. Confirm these skills in your own staff before assuming the cost.

Can we run this without outside help?

Some organizations can, especially those that already operate GPU clusters and a PostgreSQL and Redis platform. The question is whether the work fits into your existing on-call and change processes. The hard parts are usually the first production rollout, model version upgrades, capacity planning for peak traffic and evidence collection for audits. Staff time is a recurring cost, and it is easy to undercount if the work is spread across several teams. A fixed-scope engagement can cover the initial build and knowledge transfer, while a retainer covers ongoing upgrades and support. Decide which of those you need before you request a quote.

How do Bedrock Flex or batch pricing change the answer?

They change the comparison more than most teams expect. Flex and batch tiers lower per-token prices for workloads that tolerate delay, so the self-hosted GPU cost must be compared against the discounted rate, not the on-demand one. A self-hosted cluster is paid for whether it is busy or idle, so it favors steady, high-utilization traffic. Latency-sensitive or bursty workloads may stay on Bedrock, and batch jobs may be cheaper to run on a scheduled GPU pool. Split your traffic by pattern before you model the cost, and check current Bedrock tier prices in the AWS pricing pages at the time of the comparison.

What should we ask for in a fixed-scope quote?

Ask for a written scope that lists each deliverable and the environment it runs in: the GPU instances or hardware, the Kubernetes layout, the vLLM and LiteLLM configuration, PostgreSQL and Redis, monitoring through the Prometheus metrics vLLM exposes, secret handling for the LiteLLM master key, and the network isolation and endpoint allowlisting. Also ask for the handover items, the runbooks, the knowledge transfer sessions and the acceptance criteria. Confirm what is excluded, such as model evaluation on your prompts, data migration and compliance evidence. DeployOpen does not publish a price list; a quote follows from the written scope, and its timelines are agreed in the same document.

Which costs recur after the initial build?

Most recurring costs come from capacity and people. GPU or accelerator hours run every hour the cluster is up, whether or not requests arrive, unless you scale nodes down. Storage, networking, PostgreSQL, Redis and monitoring also recur. Staff time for on-call, patching, vLLM and LiteLLM upgrades and model changes is a recurring line too. vLLM ships new minor releases about every two weeks, for example v0.31.0 on October 5, 2026, according to its GitHub releases, so budget for regular upgrade testing. One-time costs include design, implementation, hardening and migration of applications from Bedrock's API to the OpenAI-compatible endpoint.

How DeployOpen scopes the cost work

The cost comparison on this page only holds if the self-hosted side is scoped as carefully as the Bedrock bill. DeployOpen plans and implements self-hosted vLLM and LiteLLM inside your own infrastructure, whether that is your cloud accounts, your data center or an air-gapped network. The work covers the GPU node pools, the vLLM serving configuration, the LiteLLM proxy with its PostgreSQL and Redis dependencies, the reverse proxy that limits which vLLM endpoints users can reach, and the metrics pipeline. Each of those is a line item that a quote should name explicitly.

Most engagements are fixed-scope: the scope, deliverables and timelines are agreed in writing before any work starts. For a cost study, that written scope should list which Bedrock models and pricing modes are being compared, which workloads move, what throughput and latency targets apply, and who will operate the cluster after handover. It should also state which items are one-time (design, deployment, hardening, migration testing) and which recur (GPU capacity, support, upgrades and continuous testing, which DeployOpen offers as retainers).

DeployOpen does not publish a price list, and it does not estimate prices for this page. A quote follows from the written scope. When you request one, ask for each component to be priced separately so you can see what changes if a workload stays on Bedrock, if you size GPUs differently, or if your own team takes over operations. Confirm current Bedrock prices, GPU instance rates and any Provisioned Throughput terms directly with AWS and your account team, since those figures change and are not supplied here.

How DeployOpen delivers a fixed-scope self-hosted inference build

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For Amazon Bedrock cost vs self-hosted vLLM, that means putting your real Bedrock invoices, GPU options and gateway needs into a written scope before any build work starts.

How we scope

Pull your Bedrock invoices by model, token tier and Provisioned Throughput commitment to set the baseline.
Agree in writing which GPU instances, gateway, monitoring and team roles the self-hosted option will include.
Confirm the traffic profile, peak concurrency and latency targets that the vLLM cluster must meet.

How we build

Deploy vLLM behind a reverse proxy that allowlists only the endpoints your users need, not the full server.
Connect a LiteLLM gateway with virtual keys, budgets and rate limits so spend is tracked per team.
Wire Prometheus metrics from vLLM and the gateway into your monitoring stack and test failover paths.

How we hand over

Document the cost model, sizing decisions, upgrade steps and the vLLM and LiteLLM release policy your team follows.
Provide a runbook for scaling, GPU failure, model rollout and rollback, tested with your operators.
Walk through the total cost of ownership worksheet so finance and platform leads can update it themselves.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after handover.
Retainer scope, response expectations and upgrade cadence are agreed in writing, not assumed.
Quarterly review of GPU utilization, gateway spend and vLLM release notes against your agreed scope.