What a migration off LLM Guard involves
The protectai/llm-guard repository was archived by its owner on 9 July 2026, and its README states that the project and its associated Hugging Face models are no longer under active development or maintained. Any deployment still running LLM Guard needs a written replacement plan, and no new LLM Guard rollouts are recommended. The migration starts with an inventory: list every input scanner and output scanner wired into your application or API server, record the settings each one uses, and note which models it downloads. That inventory is the baseline for everything else.
Next, map each deployed scanner to a maintained option, such as NeMo Guardrails, or to a control implemented elsewhere in your stack. LLM Guard's documentation lists 15 input scanners and 20 output scanners, so confirm which of them your application actually calls. For each mapping, decide whether the replacement covers the same risk, covers part of it, or needs a new control. Write those decisions down, because they become the coverage evidence you show reviewers later.
Testing comes before cutover. Build a test set from prompts and responses your application has really seen, plus adversarial cases for each risk category in scope, and run it through both the old and new guardrail. Compare the blocked, allowed and redacted results, and treat every difference as a gap until someone signs it off. Then cut over in stages with a rollback path, and keep LLM Guard running in parallel until the gap list is closed.
Triggers for starting now include the archive notice itself, an audit finding about unmaintained components, a platform upgrade to Python 3.13 or newer (the last LLM Guard release, 0.3.16, requires a Python version below 3.13), and a security review. A team with prior guardrail experience can run the migration internally if the scanner inventory is small. A partner usually helps when the inventory is large, when several applications share the scanners, or when the evidence has to satisfy an external assessor.
What the migration work includes
The migration produces a set of written deliverables that move each LLM Guard control to a maintained guardrail. The target guardrail is confirmed in the written scope before design begins.
Scanner inventory
A complete list of every LLM Guard scanner your applications call, on both the input and output sides. LLM Guard documents 15 input scanners and 20 output scanners, so the inventory records which ones are in use, along with each scanner's configuration, thresholds and any custom settings.
Scanner mapping
A record that pairs each scanner in use with a maintained equivalent or a documented control. Examples include Prompt Injection, Secrets, Token Limit and Anonymize on the input side, and Deanonymize, Malicious URLs, JSON and Factual Consistency on the output side. Where no equivalent exists, the control is written down and reviewed with your security team.
Rule and policy design
The rules, policies and thresholds the target guardrail will enforce, designed to match the behavior your application had under LLM Guard. The design covers input and output flows, the action taken when a check fails, and how blocked requests and responses are logged for review.
Integration changes
Code and deployment changes for the form LLM Guard runs in today, whether the Python library installed with pip or the API server your applications call. Each call site is listed, changed and tested, including timeouts, error handling and any secrets or model settings the new guardrail requires.
Gap test report
A report that runs the same test inputs against the existing and replacement controls, covering each mapped scanner. Each difference is recorded as accepted, mitigated or blocking the cutover, with the evidence and the named owner who made the decision. Test sets should include prompt injection attempts, leaked credentials and malicious links.
Cutover runbook and decommissioning
A step-by-step runbook for moving traffic to the maintained guardrail, with named owners, go and no-go checks, monitoring during the change and a rollback path back to the existing deployment that has been tested. After traffic has moved, a decommissioning record documents the removal of LLM Guard services, configuration and stored secrets.
What drives cost and duration
Each row names one driver of migration effort, explains why it changes the work, and shows how to keep it manageable.
| Driver | Why it changes effort | How to keep it down |
|---|---|---|
| Scanners enabled and their configuration | Scanners such as Factual Consistency and Ban Topics often need custom policy or tuning, and each one needs its own mapping and tests. | Scope the migration to the scanners in use. Export the active configuration and retire unused scanners from the plan. |
| Applications and LLM endpoints calling the guardrail | Each calling application and model endpoint needs its inputs, outputs and failure behavior checked after cutover. | Inventory every caller from gateway logs and code search. Migrate the highest-traffic paths first. |
| Library or API server deployment | Library use inside application code means changes in each codebase. A separate API server centralizes the change but adds service, scaling and access work. | Confirm which mode each team runs. Plan one shared client wrapper so the replacement is swapped in one place. |
| Traffic volume and latency budget | Higher request rates and tight latency limits raise load testing, sizing and the need for a canary period before full cutover. | Record current latency and scanner timings from production. Use them as the acceptance baseline for the new guardrail. |
| Custom scanners or patches | Custom scanners and patches written against LLM Guard internals must be re-created or redesigned, since no upstream fixes will arrive. | List every fork and override. Decide per item whether it becomes a policy, a maintained scanner or a retired control. |
| Number of environments | Each environment, from development to production, needs its own configuration, model access and test run before cutover. | Use one codified configuration promoted through environments. Avoid hand-editing settings in each one. |
| Evidence requirements from auditors | Assessors may ask for scanner mappings, test results, change records and rollback proof. Each item adds documentation work. | Confirm evidence needs with your auditor before scope is agreed. DeployOpen prepares evidence but does not issue audit reports. |
| Labeled test data available | Labeled prompts and responses show whether the replacement blocks and allows the same traffic. | Reuse the existing corpus for regression. Build labeled samples from logs with privacy review before the build starts. |
Migration phases and what each produces
The sequence moves an archived LLM Guard deployment onto a maintained guardrail one controlled phase at a time, with a written output at every stage.
Inventory deployed scanners, versions and models
DeployOpen's engineers and your platform team list every LLM Guard input and output scanner in use, the application or route each one protects, the installed package and Python versions, and every Hugging Face model artifact the scanners load. The step closes with a signed inventory that your team confirms matches production configuration, so no scanner is missed during mapping.
Map each scanner to a target control
Each in-use scanner is matched to a maintained option, such as a NeMo Guardrails rail or another control you select, and the mapping is recorded in a decision record that states the original behavior, the chosen replacement, and any capability that cannot be matched directly. Your security and compliance owners approve the records. The step is complete when every scanner has an approved decision record or an accepted risk note.
Design guardrail policies and integration points
The team defines the policies that the replacement guardrail will enforce, the thresholds and actions for each rule, where each check runs in the request and response path, and how blocked or modified content is logged and returned to the application. The output is a design document reviewed by the application owners and the security team, including the integration interface the applications will call.
Build and test in a non-production environment
The new guardrail is built in a non-production environment that mirrors the production model endpoints, logging and network paths as closely as your environment allows. Engineers configure the policies, connect the integration, and run the application end to end. The step closes when the build deploys from version-controlled configuration and a smoke test passes against each protected route.
Run gap and regression tests on recorded traffic
Recorded prompts and responses from the existing deployment are replayed through both LLM Guard and the replacement, and the results are compared scanner by scanner. Reviewers examine false positives that would block legitimate use and false negatives that would let unwanted content through, and each finding is either fixed or accepted with a documented reason. The step closes with a test report listing every reviewed difference and its disposition.
Cut over in stages with monitoring and rollback
Traffic moves to the replacement in stages, starting with a limited set of routes or users, while dashboards and alerts track block rates, latency, errors and reviewer escalations against the baseline. A rollback procedure is rehearsed before each stage and is triggered if thresholds set in the design are crossed. The step closes when all traffic runs on the replacement and the rollback path has been tested.
Decommission LLM Guard and retain the evidence pack
LLM Guard services, container images, model caches and secrets are removed from production in a controlled change, and access grants are revoked. The evidence pack is archived with the inventory, decision records, design, test reports, cutover logs and removal confirmations. The step closes when your team signs off on the pack and it is stored where your auditors and compliance owners can retrieve it. Phase durations are not published here and are set in the written scope.
What usually goes wrong and how to prevent it
Most failed guardrail migrations fail quietly, not loudly. The risks below are the ones we see most often when an archived scanner set is replaced, with a prevention step for each.
Scanners dropped without a decision
An LLM Guard input or output scanner may have no direct equivalent in the replacement, and the gap is easy to miss because nothing errors. Build a gap register that lists every scanner in use, its mapped replacement or compensating control, and the residual risk. Have the risk owner sign each gap before cutover.
Thresholds that change block rates
Scanners with similar names can score content differently, so a threshold carried over unchanged may block far more or far fewer prompts than before. Calibrate each threshold against labeled samples from your own traffic and record the block rate before and after. Do not enforce until the difference is understood and accepted.
Latency added to user-facing paths
Each guardrail call sits in the request path, and a new model-based check can add delay that users notice during peak load. Measure p95 and p99 latency with and without the guardrail under realistic concurrency. Decide which checks run synchronously and which can run asynchronously after the response.
Output scanners in the wrong order
Output checks such as redaction, deanonymization, format validation and toxicity filtering interact, and running them in a different sequence can leak data or reject valid answers. Write the intended order as a documented pipeline, then test representative outputs through each ordering change. Keep the pipeline definition under version control.
False positives that break workflows
A stricter replacement can flag legitimate business content, such as account numbers, product codes or internal terms, and stop approved processes. Replay logged traffic in shadow mode, review flagged items with the business owners, and add scoped allow rules with an expiry and a named approver.
Tests built only from synthetic prompts
Test suites written from invented attack strings rarely match how real users and documents reach the application. Build a test set from sanitized production-like traffic, including long documents, multilingual input and malformed files, and add known injection patterns on top. Replay it in shadow mode before any enforcement.
Cutover without a tested rollback
A switch to the new guardrail that cannot be reversed quickly turns any defect into an incident. Keep the archived deployment runnable during the transition, rehearse the rollback in a non-production environment, and define the trigger conditions and the person who authorizes it before cutover begins.
Ongoing costs after the replacement is live
This table separates the one-time migration work from the recurring costs of running a maintained guardrail, with who carries each cost and what moves it up or down.
| Cost component | One-time or recurring | Who carries it | What drives it up or down |
|---|---|---|---|
| Hosting and compute for the guardrail runtime | Recurring | Your cloud or data center team | Request volume, number of active rails or scanners, latency targets, and whether the guardrail runs beside the application or as a separate service |
| Model or provider charges | Recurring, where the design uses them | Your organization, paid to the model host or API provider | Hosted LLM checks bill per call, while self-hosted classifiers move the cost into compute and storage |
| Monitoring and log storage | Recurring | Your platform or operations team | Volume of blocked and flagged events, retention periods set by your policy, and the dashboards and alerts built during cutover |
| Policy tuning and review time | Recurring | Security, product and risk owners in your organization | New attack patterns, false positives reported by users, and how often rails and thresholds are reviewed on a fixed schedule |
| Upgrade testing for the guardrail platform | Recurring | Your engineering team, or DeployOpen under a retainer | Release frequency of the chosen platform, changes to its dependencies, and the size of the regression suite kept from cutover |
| Support retainer or commercial support | Recurring and optional | DeployOpen retainer, or a commercial option from the guardrail maintainers | Scope of incident response, upgrade work and testing. Confirm licensing terms and commercial support options directly with the maintainers |
| Inventory, design, build, testing and cutover | One-time | Your team and DeployOpen, under the agreed written scope | Number of LLM Guard scanners in use, custom rules, and the number of integration points. These do not repeat once the replacement is live |
| Retirement of LLM Guard components | One-time | Your platform team | Removal of pinned Python environments, cached Hugging Face models and API servers after traffic has moved to the replacement |
How the work differs by starting point
When LLM Guard is called from application code, the migration sits inside the codebase and its CI pipeline. Each scanner call needs a mapped replacement, and the test suite has to send the same prompts and responses through both paths.
- Inventory every import and scanner call, including scanners loaded by name from configuration files.
- Check how the application reads scanner results, including risk scores and any sanitized output it uses downstream.
- Confirm which dependencies and model downloads the library triggers at runtime, since LLM Guard models are hosted on Hugging Face.
- Run the old and new guardrails side by side in tests before removing the import.
- Plan the release that removes the dependency so that rollback remains a single deployment.
When LLM Guard runs as a standalone API server, other services call it over HTTP, so the change lands in the request contract and the routing layer. On Kubernetes, the same server runs as workloads with their own probes, secrets and scaling settings, and those need review as well.
- Document every client that calls the server, along with its request format and how it reads responses.
- Keep the existing endpoint live behind a routing layer so callers can switch over one at a time.
- Review image provenance, resource limits, network policy and the pod's access to model storage.
- Confirm that health and readiness probes reflect whether the scanners have finished loading.
- Measure startup time and memory use of the replacement under load, since they may differ from the current server.
The last PyPI release, 0.3.16, was uploaded on 19 May 2025 and requires Python 3.10 or newer but below 3.13. Deployments on that version need their interpreter and dependency pins verified before any migration test runs, because a runtime change can break the old code path on its own.
- Record the exact interpreter version in every environment, including build images and developer machines.
- Freeze the full dependency tree from the lock file, including transitive packages.
- Ask the replacement's maintainers which Python versions it supports, and check whether that range overlaps with 3.10 to 3.12.
- Schedule any Python upgrade separately from the scanner migration so that a failure has one cause.
- Keep the pinned environment reproducible until the cutover is signed off.
Teams that wrote their own scanners or edited the library face a mapping problem that no vendor comparison solves. Each custom scanner has behavior that must be documented and reproduced, or formally retired with the approval of the risk owner.
- Export each custom scanner's logic, thresholds and fallback behavior into a written specification.
- Decide per scanner whether the maintained option covers it, whether it needs a custom rule, or whether it is retired.
- Diff patched library files against the 0.3.16 release so every local change appears in one list.
- Build a regression set from traffic that triggered each custom scanner, with sensitive data removed.
- Record the decision for each scanner so auditors can see why it was kept, replaced or retired.
Where the network blocks outbound downloads, model artifacts and packages must be mirrored in advance. LLM Guard models come from Hugging Face, so the mirror has to cover what the current deployment loads and what the replacement will load.
- Inventory every model file the current deployment loads, with versions and checksums, from its cache directory.
- Mirror the replacement's packages, container images and model assets into the internal registry before the change window.
- Confirm whether the replacement fetches anything on first start, and disable or pre-seed those calls.
- Verify mirrored artifacts against the recorded checksums inside the restricted zone.
- Ask the project which runtime calls reach external services, then block them during testing to prove the offline path works.
What you need to provide
The migration moves fastest when the client supplies a fixed set of inputs before scoping closes. Start with read access to the code and configuration that call LLM Guard, including the scanner classes, their settings and the place where each input and output check runs. Record the exact LLM Guard version in use (the last PyPI release is 0.3.16) and every Hugging Face model identifier the scanners load, with any revision pins. Without those identifiers, the team cannot reproduce current behavior, and the gap analysis will rest on guesswork.
Next, provide evidence of how the guardrail behaves today. Sample traffic is ideal, but a representative set of prompts and responses works too, covering normal use, known attack patterns and the edge cases your product team worries about. Where possible, attach labeled outcomes showing which items were blocked, redacted or allowed and whether those decisions were correct. Confirm who may share production data and how it must be anonymized before it leaves your environment.
Name a policy owner who can accept or reject each gap between the old scanners and the replacement. A gap is any control the new guardrail does not match, such as a scanner with no equivalent or a threshold that behaves differently. The owner should sign off on the mapping, because a technical team should not decide alone which risks are acceptable. Also provide environment access for build and shadow testing, a change window agreed with operations, and a named rollback owner who can reverse the cutover within an agreed time.
Decide three things early because the rest of the plan depends on them. First, the target guardrail, which may be NeMo Guardrails or another maintained option, selected against your scanner inventory. Second, the enforcement thresholds for each block or redact decision, since changing them later invalidates test results. Third, the decommissioning date for the archived deployment. Ask your auditors now which evidence format they expect for the cutover, so that test logs, approvals and rollback records are captured from the first shadow run.
Frequently asked questions
Can our own team migrate from LLM Guard without outside help?
Often yes, if your team has Python and LLM application skills and time to run the work alongside its normal duties. The deciding factors are how many of the 15 input and 20 output scanners you use, whether LLM Guard runs as a library inside your applications or as an API server, and whether anyone on the team has tuned its thresholds. Teams without that context often underestimate the scanner mapping and gap testing, which is where most of the uncertainty sits.
What drives the cost of an LLM Guard migration?
Cost follows scope. The main drivers are the number of scanners in use, how many applications call LLM Guard, whether it runs as a library or an API server, how customized your thresholds and allow lists are, and how much representative test traffic you can provide for gap testing. Compliance evidence and a staged cutover add work when they are required. The migration itself is one-time work. The maintained guardrail you move to carries recurring upgrade and support effort, which can be retained or kept in-house. Ask for a quote that lists each deliverable, the assumptions behind it, and what you provide.
How long does an LLM Guard migration take to complete?
No duration is given without a written scope, because the timeline depends on the same factors that drive cost. A deployment with a small number of scanners, one application and good test data follows a different path than one with many scanners, several applications and compliance evidence to produce. DeployOpen agrees scope, deliverables and timelines in writing before work starts. To plan internally, list your scanners, applications, deployment mode and test data sources, then ask for a fixed-scope proposal.
Is there a like-for-like replacement for every LLM Guard scanner?
No, and you should not assume one. Some scanners may map to a built-in capability of the target guardrail, some may need custom rules or a separate control, and some may have no equivalent you can accept as-is. The first deliverable is a scanner-by-scanner map that records current behavior, the target approach, the gap and the decision for each one. Confirm every target capability against the maintained project's own documentation before committing, since features change between releases.
Should we migrate off LLM Guard now or wait?
Plan the migration now, and keep the current deployment running only as a bridge. LLM Guard was archived on 9 July 2026, and its README states that the project and its Hugging Face models are no longer under active development or maintained, so DeployOpen does not recommend new LLM Guard rollouts. Its last PyPI release, 0.3.16, was uploaded on 19 May 2025 and requires Python 3.10 or newer but below 3.13, which may constrain your runtime upgrades. While you plan, pin versions and track the Python ceiling. Waiting adds risk because the code will no longer receive maintained fixes.
What do we receive at the end of a migration engagement?
Deliverables are listed in the written scope and typically include a scanner mapping, the target guardrail configuration, gap test results, a cutover plan with rollback steps and operating documentation. DeployOpen can also prepare the systems, controls and evidence your team needs for a compliance assessment. DeployOpen does not issue audit reports, attestations or certifications, since independent auditors and certification bodies do that. Ask for the deliverable list and acceptance criteria before signing, so both sides agree on what done means.
How DeployOpen approaches this migration
DeployOpen maps each LLM Guard scanner your applications use to a maintained guardrail, tests for gaps, and cuts over in infrastructure you control. Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
How we scope
- Inventory which LLM Guard scanners your applications call, from the documented 15 input and 20 output scanners.
- Agree in writing which scanner mappings, thresholds and test cases are in scope before work starts.
- Engagements are fixed-scope, so deliverables and timelines are set in writing before build begins.
How we build
- Map each archived scanner to a maintained option such as NeMo Guardrails, recording behavior differences per scanner.
- Build replay test sets from production-like prompts and responses to find detection gaps before cutover.
- Deploy the replacement in infrastructure you control, with rollback steps and a parallel run planned.
How we hand over
- Document the scanner-to-guardrail mapping, known gaps, configuration and the test evidence behind each decision.
- Provide a cutover runbook covering sequencing, monitoring during the switch and rollback criteria.
- Decommission the LLM Guard deployment, including pinned packages, models and secrets it used.
How we support
- Keep the replacement current through a retainer covering updates, testing and guardrail configuration changes.
- Re-run the replay test sets after each guardrail upgrade or model change before accepting new behavior.
- Ask for the support scope, response arrangements and upgrade handling in writing as part of the quote.
