OpenAI model deprecations: keeping a model version by self-hosting

OpenAI retires generally available models on its own schedule, so every retirement reopens prompt testing. Self-hosting an open-weight model version lets you choose when to change, at the cost of running the stack yourself.

Book a callTalk to us
On this page

Why OpenAI model deprecations force prompt revalidation, and when self-hosting a pinned version fixes it

If your application is tuned to one specific OpenAI model and that model is approaching its retirement date, the retirement date is not yours to choose. OpenAI gives at least six months' notice before shutting down generally available models and less notice for previews, so the deadline is set on OpenAI's schedule and your team has to revalidate prompts, evaluations and output handling inside that window. Start by listing every model identifier in your code, configuration and prompt registry, and note which ones are generally available and which are previews, because the notice you get depends on that status.

The short answer is that a pinned open-weight model served on vLLM behind a gateway such as LiteLLM keeps the model version fixed on your schedule. Applications keep calling one OpenAI-compatible endpoint, and the gateway maps the logical model name to the weights you have chosen. The trade is that the work moves to you: GPU capacity, serving configuration, security patching of the inference stack, upgrades to new vLLM releases, and a regression suite that proves each change is safe. Pinning also does not mean the model matches a given OpenAI model's output quality. Run your own evaluation set against the candidate model before you commit, and confirm the licence terms of each set of weights.

Decide whether self-hosting is worth it by comparing three costs: the revalidation you would do anyway for an OpenAI migration, the recurring work to operate the serving stack, and the infrastructure that runs it. Ask your team who will own on-call response, who approves a version change and how long old weights stay available for rollback. If your workloads change often or depend on newer OpenAI features, a pinned model may become a fixed cost rather than a stability gain, so keep the OpenAI endpoint behind the same gateway as a fallback while you decide.

What a model-version pin requires from your infrastructure

A pinned model version only holds if four parts of your stack hold still: the endpoint your applications call, the weights you keep, the serving engine underneath, and the tests that decide when a change is allowed. Each card below names one of those parts and what to check before you depend on it.

An OpenAI-compatible endpoint

The application contract is the first thing to pin. vLLM's server, started with vllm serve, exposes /v1/chat/completions, /v1/completions, /v1/responses and /v1/embeddings, so existing OpenAI SDK calls can point at your own base URL. Confirm that the request fields your applications use, such as tool calling and structured output, behave the same on the endpoint you deploy before you retire the hosted model.

A versioned, reproducible model artifact

Keep the exact weights, tokenizer and chat template in your own object store under a version tag, with checksums recorded next to the serving configuration. A deprecation notice on a hosted model does not stop a file you already hold from being served, but it only helps if you can rebuild the same deployment later. Ask your team who approves a new artifact and where its license terms are recorded.

A serving engine on a tracked release cadence

vLLM ships new minor releases about every two weeks, with v0.31.0 dated October 5, 2026 in the supplied release history. A pin means you choose one engine version, record it, and schedule a deliberate upgrade rather than taking whatever the container tag resolves to. Plan a monthly review of release notes and a test run on each candidate before it reaches production.

A gateway that keeps old versions reachable

A gateway such as LiteLLM lets you map one logical model name to a specific backend and roll back by changing a route. LiteLLM states it maintains support for the four most recent stable minor versions, so the gateway itself also has a support window you must track. Confirm which gateway version you run, and that your upgrade path stays inside a supported range.

An evaluation set that gates every change

A version pin is only useful if you can prove whether a replacement behaves acceptably. Build a regression set from your real prompts, with expected outputs or scoring rules for the tasks that matter, and run it against the pinned model and every candidate before any switch. Record the results alongside the artifact and engine versions so an auditor or security reviewer can trace each production change.

Access controls on the serving endpoint

A self-hosted endpoint inherits your security obligations. vLLM's --api-key option does not protect every route, including /metrics and /health, so place a reverse proxy in front that allowlists only the endpoints your users need. Confirm this before exposing the model to other teams, because a pinned model reachable by anyone on the network is still an unmanaged service.

Which open-source options hold a model version steady

vLLM is the engine that loads and serves the open-weight model you pin. It is licensed under Apache-2.0 and has no paid edition, so the spend is hardware, the surrounding platform and the people who run it. The server, started with vllm serve, exposes OpenAI-compatible endpoints such as /v1/chat/completions and /v1/embeddings, so applications can keep their client code while the weights stay fixed on your hardware. Pinning is only as strong as your change control, so keep the model identifier and the vLLM image tag in source control and change them through review.

  • Version control: vLLM ships new minor releases about every two weeks, for example v0.31.0 on October 5, 2026, so pin the image tag and upgrade on your own test cycle.
  • Metrics: /metrics exposes Prometheus-format request counts, token throughput, latency histograms and KV cache use for dashboards and alerts.
  • Kubernetes options: native manifests, Helm, the production-stack, KServe and KubeRay are among the documented paths. Choose the one your platform team can operate.
  • Endpoint exposure: --api-key does not cover /metrics, /health, /invocations or /pause, so place a reverse proxy in front that allowlists only user-facing endpoints. See the vLLM security docs.
  • Multi-node traffic: communication between nodes is insecure by default, so isolate those nodes on a private network.

Confirm with your test environment which model revision your vLLM build loads, how your image tag maps to it, and the GPU memory the chosen model needs.

What work moves from OpenAI to your team when you pin a model

When you pin an open-weight model on your own infrastructure, the version stops changing unless your team changes it, and every operational duty that OpenAI carried for the hosted model moves to you. The first new owner is capacity. Someone must size the GPU pool for peak concurrent load, keep spare capacity for failover and decide whether the same accelerators also serve embeddings or evaluation jobs. Serving stack patching is the second duty. vLLM ships new minor releases about every two weeks, so the team needs a rule for which releases it tracks, who tests them and how a rollback works. Pinning a container image or package version keeps the model stable, but it does not keep the serving engine secure, so the team owns the decision to stay behind the release train.

Securing the endpoint is the third duty, and it is easy to underestimate. The vLLM security documentation warns that the --api-key option does not cover every route, including /metrics, /health and /invocations, so you should place a reverse proxy in front that allowlists only the endpoints your users need. For multi-node runs, the same documentation says inter-node traffic is insecure by default, so the nodes must sit on an isolated network. Your security team should own the proxy configuration, the network policy and a periodic check that no unintended route is reachable. Monitoring and on-call follow from these. vLLM exposes Prometheus metrics at /metrics for request counts, latency and KV cache use, and someone must own the alert thresholds and the response when inference slows or fails overnight.

Re-running evaluations before any upgrade is the duty that protects the reason you pinned the model in the first place. Build a regression suite from your production prompts, record baseline outputs and scores, and rerun it against each candidate serving version and model revision before cutover. Confirm with your evaluation owners which metrics count as a pass. One-time costs cover the initial build: architecture and GPU procurement or reservation, gateway and proxy setup, the evaluation suite, security testing and runbooks. Recurring costs cover GPU capacity, operations staff, on-call coverage, patch and upgrade cycles, evaluation reruns and support contracts. Ask any implementation partner for a written split of these items, and check which GPUs, hours and staff are included in the quote.

How to deploy a pinned model version under your change control

This sequence freezes today's OpenAI behavior as a measured baseline, then serves a checksummed open-weight model behind your own gateway so the version you validated is the version you run. Each step ends with a check you can record as change control evidence.

  1. Freeze the golden prompts and evaluation set

    Before touching infrastructure, the application owner and the prompt or ML engineers export the production prompts, system messages, tool schemas and expected outputs currently sent to the OpenAI model, then add representative edge cases such as refusals, structured output and long inputs. Record the exact OpenAI model identifier and the date of capture, and store the set in version control with its own review approval. The step is done when someone who did not write the set can rerun it and get the same baseline scores from the current model.

  2. Select and checksum the open-weight artifact

    The model owner chooses an open-weight model whose license your legal team has approved, then downloads the exact weights and tokenizer files from the publisher and records a SHA-256 checksum for each file. Store the artifacts in your internal object store or model registry under an immutable version tag, and keep the license text beside them. The step is done when a second person can verify the checksums against the stored copy and the artifact is locked against overwrite. Confirm with your legal and procurement teams which license terms apply to your use.

  3. Serve the pinned model on vLLM

    The platform team starts the model with vllm serve on GPU nodes sized for the model and expected concurrency, pointing it at the local artifact path and the pinned tag, not a floating name. The server exposes OpenAI-compatible endpoints such as /v1/chat/completions and /v1/responses, so existing client code needs only a base URL change. On Kubernetes, the vLLM documentation lists native manifests, Helm and the vLLM production-stack as deployment options. The step is done when a smoke test returns a completion from the pinned path and the server logs show the expected model hash or path. Confirm the chosen Kubernetes method, GPU driver stack and node layout against your own environment.

  4. Route traffic through a LiteLLM gateway

    The gateway team deploys a self-hosted LiteLLM proxy backed by PostgreSQL, adds the vLLM server as a model deployment and keeps the OpenAI deployment registered beside it. They issue virtual keys per application or team, set per-key budgets, TPM and RPM limits, and restrict each key to specific model names so a team cannot silently call the new model or the old one. Store LITELLM_MASTER_KEY in a secret manager. The step is done when a test key can reach only its allowed models and a denied request is logged. LiteLLM's production guidance also covers Redis once more than one proxy instance runs.

  5. Allowlist endpoints at a reverse proxy

    The security team places a reverse proxy in front of the vLLM servers so that only the endpoints meant for applications, such as /v1/chat/completions and /v1/embeddings, are reachable from the application network. vLLM's security documentation notes that its --api-key option does not protect every route, including /metrics, /health, /pause and /abort_requests, and recommends this allowlist approach. Ensure the gateway is the only caller of the vLLM service. The step is done when a request to an unlisted path from an application subnet is refused and a scan of the vLLM service from that subnet shows only the allowed routes.

  6. Wire metrics and logs to Prometheus

    The observability team scrapes the vLLM /metrics endpoint from inside the cluster network and collects gateway logs, covering request counts, token throughput, latency histograms and KV cache use. Dashboards and alerts are set for error rate, latency and queue depth against the baseline captured earlier. Because /metrics is excluded from the API key, the scrape path must stay internal. The step is done when the dashboard shows live traffic from the gateway, an alert fires in a controlled test, and the team confirms the metrics endpoint is unreachable from application subnets.

  7. Run parity evaluations against the golden set

    The evaluation owner runs the frozen prompt set against the pinned open-weight model through the gateway and compares outputs with the stored OpenAI baseline. Scoring combines automated checks, such as schema validity and required fields, with human review of a sample for accuracy, tone and policy compliance. Record every difference, decide which ones block cutover, and capture the results with the model checksum, prompt set version and reviewer names. The step is done when the change board has a signed parity report listing accepted and rejected differences. Expect this to be iterative, with prompt or parameter adjustments that must go through the same review.

  8. Cut over with OpenAI as fallback

    The platform team changes the gateway routing for a limited set of applications first, with the OpenAI deployment kept as a fallback in the LiteLLM router for failures and cooldowns. Traffic shifts are approved in stages, and each stage is watched against the dashboards and parity thresholds before the next. The fallback stays configured until the change board accepts the new version as the operating baseline, after which its removal is a separate, recorded change. The step is done when the change record shows every stage, the rollback path has been tested, and the fallback decision is documented. Confirm with your legal team whether continued OpenAI fallback changes your data processing position before you rely on it.

What auditors and change boards ask to see for a pinned model

A pinned open-weight model is only as defensible as the records behind it. Auditors and change boards usually ask the same questions: what exactly is running, who approved each change, who can call it, and what was kept. DeployOpen prepares these artifacts for your environment, but independent auditors and certification bodies are the ones who issue any audit report or certification.

Model artifact inventory

Keep a register that records each served model by its exact name, release version, source repository revision and SHA-256 checksum of the weight files, plus the vLLM image tag used to serve it. Store the register next to the deployment manifests so a reviewer can confirm that the bytes in production match the bytes that were tested. Ask whether your team can reproduce the same serving image months later, since a rebuilt container can change behavior even when the weights are identical.

Change records with evaluation results

Every upgrade, rollback or serving configuration change needs a record that links the proposed change to a fixed evaluation set and its results. Include the prompts, expected behaviors, pass and fail counts, and the reviewer who accepted the change. Because vLLM ships new minor releases frequently, the change board should decide explicitly whether a serving engine update is a change that requires re-evaluation before it reaches production.

Gateway key and budget access evidence

Show who holds each gateway key, which models it can call, and which budgets and rate limits apply to it. In LiteLLM, virtual keys are tied to a PostgreSQL database and can be limited to specific models with per-key, per-user and per-team budgets. Export these settings on a schedule and compare them with your access review tickets, paying attention to keys that were created for testing and never removed.

Logs and metrics retention

Decide how long request logs, token counts and latency metrics are kept, and who can read them. vLLM exposes Prometheus-format metrics at /metrics, which covers request counts, token throughput, latency and KV cache use, but it does not store prompt history for you. Prompt and response logs carry personal and confidential data, so document where they are stored, how access is granted, and how they are deleted when retention ends.

Deprecation calendar tied to OpenAI notices

Track every model you still call through OpenAI, every self-hosted model you have pinned, and the date each OpenAI notice was received. OpenAI gives at least six months' notice for generally available models and less for previews, so the calendar should show the shutdown date, the revalidation deadline and the owner of each item. Review it at each change board so that a pinned model is never the only remaining path when a hosted model retires.

Questions buyers ask before self-hosting a model version

What does it cost to self-host a pinned model version?

The cost has four components: GPU or accelerator capacity, the surrounding infrastructure (gateway, database, monitoring, storage), implementation work, and the engineers who operate and validate the stack. vLLM itself is Apache-2.0 licensed with no paid edition, so the software line is zero, but GPU capacity typically dominates the recurring bill. Most of the cost is one-time for setup and validation, then recurring for hardware or cloud instances, operations and periodic revalidation. Ask any quote to split these components and state which assumptions about utilization and traffic it depends on.

What drives the cost of keeping a model version up?

Four factors move the total most:

  • Hardware choice and how many GPUs the model needs, including redundancy for failover
  • Traffic volume and how steadily the GPUs are used, since idle capacity still costs money
  • How many validation cycles the team runs against its own prompts and evaluation sets
  • Operations coverage, meaning who patches, monitors and is on call for the serving stack

Compare these against your current per-token spend and any enterprise data controls you pay for today.

How long does a self-hosted model deployment take?

No general duration can be stated with confidence, because it depends on the model, hardware availability, the network design and how many applications must be validated. Your written scope from DeployOpen should set out deliverables and timelines before work starts, and it should separate infrastructure setup, gateway configuration, validation and cutover. Ask the provider to name the assumptions behind each phase, including what happens if GPU capacity arrives late.

What must our team provide before a deployment starts?

Expect to supply access to a cloud account, data center or air-gapped environment where GPUs can run, along with network approvals, identity provider details and a secret manager. You also need the prompts and applications that call the model, representative test cases with expected behavior, and named owners who can accept validation results. Without a stable set of prompts and test cases, you cannot tell whether a pinned model still meets the bar, so agree on this input early.

Does pinning an open-weight model keep its output identical?

No. Pinning fixes the model weights and the serving configuration, so the model stops changing on someone else's schedule, but output can still vary with the inference engine version, quantization, sampling settings, batching and hardware. Each vLLM minor release can change behavior, so the pinned stack should be versioned as a whole. Validation against your own prompts and evaluation criteria is still required before and after every change.

Can our own team run this without outside help?

It depends on your skills in GPU operations, Kubernetes, model evaluation and security hardening. A capable platform team can run vLLM with a reverse proxy and a gateway, but the work includes tracking vLLM releases, which ship about every two weeks, and repeating validation for each change. Many teams split the work: internal staff own the day-to-day, while an outside partner handles the initial build, hardening and documentation. Ask for a skills gap review before choosing.

When is staying on OpenAI the better choice?

Staying on the hosted model usually makes sense when you need the newest model capabilities, have low or irregular volume, or lack the GPU and operations capacity to run serving reliably. It can also be right when OpenAI's deprecation schedule gives you enough time to revalidate, and when a gateway lets you move later. Keeping OpenAI behind a self-hosted gateway preserves the option to shift traffic without rewriting applications.

How does LiteLLM licensing affect SSO beyond five users?

LiteLLM is MIT licensed except for content in its enterprise directory, which has its own license. From v1.76.0, SSO is free for up to 5 users, and an Enterprise license is needed beyond that. Enterprise also adds SCIM, JWT authentication, audit logs, team RBAC and secret manager integrations. Confirm current terms in the LiteLLM license file and with the vendor, since your user count and feature needs determine whether the license cost applies.

When pinning a self-hosted model beats staying on the hosted API

Pinning a self-hosted model is worth the effort when a retirement would break workflows you have already validated, when your change control process requires fixed versions, or when data control already obliges you to run inference yourself. In those cases the predictability of the model version is the asset you are buying. A hosted model that is retired on the provider's schedule forces revalidation of prompts, output formats and downstream parsers at a time you did not choose. An open-weight model on your own vLLM server stays at the weights you tested until you decide to change them, and each change becomes a release your change board can review.

The case is weaker when your prompts are simple, your outputs are loosely checked, and the models you use are rarely retired, because revalidation is then a small recurring task. It is also weak when no team can operate GPUs, patch the serving stack and respond to incidents. Pinning shifts the operational work to you: capacity, the serving engine's security posture, monitoring, and planning for the release cadence of vLLM and the model itself. Confirm internally who will hold that on-call duty before you commit. Also confirm that the open-weight model you need meets your quality bar on your own evaluation set, which the hosted model's deprecation notice does not tell you.

Use the sibling pages on OpenAI API cost vs self-hosted vLLM and migrating from the OpenAI API to self-hosted models on vLLM to price the components and plan the cutover. A practical test is to list the models your workflows depend on, record the deprecation dates the provider has published for each, and estimate how often you would need to revalidate. If revalidation is frequent or retirement would stop a regulated process, pinning is likely the better trade. If neither applies, staying on the hosted API with a gateway in front may be the simpler choice.

How to operate the pinned model after cutover

Once the pinned model serves production traffic, this cycle keeps it current on your terms: you track upstream releases, test every candidate change against your own evaluation set, keep the previous artifact ready to restore, and review access and spend on a fixed rhythm. The result is a version that changes only when your team decides it should.

  1. Track vLLM and LiteLLM release notes

    The platform owner subscribes to release notices for vLLM and LiteLLM and reads each one for changes to the OpenAI-compatible endpoints, default sampling behavior, model support, security fixes and database migrations. vLLM ships new minor releases roughly every two weeks, so a standing review slot matters more than ad hoc checks. The step is complete when each release has a triage note that marks it as ignore, schedule for testing or urgent security fix.

  2. Confirm the gateway support window

    Before choosing which LiteLLM version to stay on, the platform owner checks the current enterprise support policy, which states that LiteLLM maintains support for the four most recent stable minor versions for open source and Enterprise deployments. Confirm this window with LiteLLM directly and in writing, because the policy can change and it governs how long you can wait before upgrading. The output is a recorded minimum supported version and a target date that the team agrees on internally.

  3. Run the evaluation set on candidates

    Before any upgrade reaches production, the application owners run the pinned evaluation set against a staging copy that uses the candidate vLLM or LiteLLM version and the same model weights hash. They compare outputs, structured formats, refusal behavior, latency and token counts with the baseline recorded at cutover. The step closes with a signed comparison report that lists every test that changed and whether the change was accepted or blocked.

  4. Stage the candidate with a rollback path

    The platform engineer keeps the prior container image, the vLLM and LiteLLM configuration files, and the exact model weights in a versioned artifact store, so the last known-good release can be restored without a rebuild. Deployment to production uses a new replica set or release alongside the current one, with traffic shifted only after health checks pass on the metrics endpoint. The check is a tested rollback: the team reverts to the prior artifact in a staging environment and confirms the old responses return.

  5. Promote the release and watch metrics

    After a candidate passes evaluation, the platform engineer promotes it in a change window that the application owners have approved. Throughput, latency histograms, KV cache use and error counts from vLLM's Prometheus metrics are compared with the baseline on a dashboard, and the gateway logs are checked for new failures in routing and fallbacks. The step is complete when the team signs off that the metrics match the baseline and the rollback artifact remains in place.

  6. Review budgets and keys quarterly

    Each quarter, the gateway administrator reviews virtual keys, per-user and per-team budgets, TPM and RPM limits, model restrictions and any temporary budget increases, and removes keys tied to people or services that no longer need access. Access to the admin UI and the master key is checked against the current identity provider groups, and the review confirms that the master key still sits in a secret manager. The output is a dated access review with each change recorded and approved.

  7. Document each decision in the change log

    The platform owner records every decision in a single change log: the release reviewed, the evaluation results, whether it was accepted, the artifact retained for rollback, and any budget or key changes made at the quarterly review. The log also notes which upstream security advisories applied and when they were addressed. The step is complete when an auditor or a new engineer could trace from any production model version back to the test evidence and the approval that allowed it.

How DeployOpen scopes a pinned-model deployment

DeployOpen starts every pinned-model deployment with a written scope that names the model, the serving stack and the people who own each decision. We plan the inference stack inside your own cloud account, data center or private network, covering the vLLM server, the gateway in front of it, the metrics pipeline and the environment where the weights are stored. The scope records which open-weight model and exact revision you are pinning, which endpoints applications may call, how the API key and network boundary are enforced, and who approves a change to the pinned version. Deliverables and timelines are agreed in writing before work starts, so you know what is built, what is handed over and what remains with your team.

Scoping also means settling the questions that a hosted deprecation notice normally answers for you. Confirm with your application owners which prompts, evaluation sets and output checks define acceptable behavior, because those become the regression suite you run before any model or vLLM upgrade. Confirm which vLLM release line you will hold, how often you will review new releases, and whether the gateway will keep OpenAI as a fallback. Ask the people who will be on call whether they can operate GPU nodes, and whether that work will be done in-house or under a retainer. Retainers are available for version upgrades, revalidation runs and continuous security testing after the initial deployment is handed over.

A quote from DeployOpen follows from that written scope rather than from a published price list. Ask for the line items to be separated into one-time work (design, build, hardening, documentation and the first validation run) and recurring work (upgrades, revalidation, monitoring and testing), and ask which parts depend on your own GPU capacity and which depend on DeployOpen's effort. DeployOpen prepares systems and evidence for compliance assessments but does not issue audit reports or certifications, so ask your independent auditor early which records they will need from the pinned-model environment.

How DeployOpen runs pinned-model deployments

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For OpenAI model deprecations, we help you decide which open-weight model version to pin, then build and hand over the serving stack that keeps it running on your schedule.

How we scope

  • Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts
  • The written scope lists which OpenAI models and endpoints your applications use today and which open-weight replacements are in scope
  • Scope defines the evaluation prompts, acceptance criteria and sign-off owners needed before any traffic moves
  • Scope states where the serving cluster, gateway and logs will run, including your residency and network constraints

How we build

  • We deploy vLLM behind a gateway that allowlists only the user-facing /v1 endpoints, not the operational routes
  • We pin the exact model weights and engine version, and record checksums so the same version can be reproduced later
  • We run your existing prompts against the pinned model and document output differences before cutover
  • We configure Prometheus metrics, request logging and access controls inside your environment

How we hand over

  • Handover includes runbooks for restarts, scaling, rollback to the previous pinned version and fallback to the hosted API
  • Documentation covers the upgrade procedure, including how a new engine or model release is tested against the pinned baseline
  • We deliver evidence for your internal and third-party assessments, but independent auditors issue any audit reports or certifications
  • Your team receives written records of configuration, access, backup and recovery steps for their own operations

How we support

  • Ongoing support, upgrades and continuous testing are available as a retainer after handover
  • A retainer can cover re-running the regression prompts when vLLM or the pinned model changes
  • Support can include security testing of the endpoint and gateway, scoped separately in writing
  • Ask for the retainer terms, response expectations and included upgrade work in your written quote

Related

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.
Book a callhello@deployopen.com