Amazon Bedrock is AWS's managed service for foundation models from Amazon and third-party providers. Pages here cover replacing it with open-weight models self-hosted on vLLM and open-source guardrails.
Comparing Amazon Bedrock with open-weight models served on vLLM on EC2, EKS or your own hardware: model catalog and version control, data retention modes, throughput and cold starts, API differences, and the operating work Bedrock absorbs that a self-hosted cluster hands back to you.
Moving Bedrock workloads to self-hosted open-weight models: mapping Bedrock model IDs to replacements, swapping Bedrock API calls for an OpenAI-compatible LiteLLM endpoint, replacing Guardrails, evaluating outputs on your own prompts and cutting over before Provisioned Throughput terms end.
Cost of ownership by component: on-demand, Flex, Priority and batch token pricing, Provisioned Throughput commitments and Custom Model Import billing against GPU instances, the gateway, monitoring, implementation and the team that runs inference.
Bedrock models move from Active to Legacy to End-of-Life, Legacy periods can be as short as 45 days, and inactive accounts can lose access early. This page covers planning around that lifecycle and when pinning an open-weight model on your own infrastructure removes the forced migrations.
Some Bedrock models require the aws_review mode, which keeps prompts and completions for up to 30 days for possible AWS review. This page explains the retention modes, how to enforce zero retention with SCPs, which models that rules out, and when self-hosting is the only way to keep every model in scope.
For teams already running open-weight models through Bedrock Custom Model Import: per-minute Custom Model Unit billing, scale to zero and cold starts against a vLLM deployment you size and keep warm, plus who controls versions, quantization and upgrades.
Replacing Bedrock Guardrails' content filters, denied topics, word filters and PII masking with self-hosted NeMo Guardrails rails or Guardrails AI validators: which policies map directly, which such as Automated Reasoning checks have no open-source equivalent, and how to test the replacement.
Bedrock is reachable only through AWS endpoints, even over PrivateLink. This page covers running open-weight models on vLLM with NeMo Guardrails in a disconnected network: moving weights and images in, endpoint authentication, monitoring and upgrades without internet access.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.