Why teams running LLM Guard need a replacement plan now
If your production prompts and model responses pass through LLM Guard scanners, you are now running a dependency that its owner no longer maintains. The protectai/llm-guard repository was archived on 9 July 2026, and its README states that the project and its associated models on Hugging Face are no longer under active development or maintained. The short answer is that no new LLM Guard rollouts are recommended. Existing deployments should move to NeMo Guardrails or Guardrails AI in a controlled way, with a dated plan, owners and a rollback path, rather than waiting for a security incident or a failed dependency upgrade to force the change.
This is different from a general guardrail build because your scanner inventory already exists in code. The scanners you call, the thresholds you set and the places where you invoke them are written down in your application and gateway configuration, so the first job is to read that code rather than design policy from scratch. Plan for a constraint as well: the last LLM Guard release on PyPI is 0.3.16, uploaded on 19 May 2025, and it requires Python 3.10 or newer but below 3.13. Confirm your runtime version now. If you move to Python 3.13 for security or platform reasons, the archived package will not install on that interpreter without a rebuild, and an unmaintained package will not receive fixes for issues found after its last release.
Your first decision is ownership: who signs off on replacing each scanner, who tests false positives on your own traffic and who approves the cutover date. Ask your implementation partner to confirm the exact LLM Guard version pinned in each service and whether any scanners load models from Hugging Face at startup. Those model files are part of your supply chain and will need the same review as the code, especially if they must be mirrored into an air-gapped network. Write these answers down before any mapping work begins, because they set the scope of the parallel run.
Requirements a replacement must meet before cutover
This table lists the requirements a replacement for LLM Guard scanners must satisfy before production traffic moves over, with what each one means for the build. Read each row as a gate: if a candidate control cannot meet it, the cutover should wait. The main takeaway is that the replacement must match the scanner coverage you already depend on, keep every model and package inside your environment, and produce logs that an assessor can review.
| Requirement | What it asks for | What it means for the build |
|---|---|---|
| Event recording (EU AI Act Article 12(1)) | High-risk AI systems must technically allow automatic recording of events over the system lifetime. | Log every input block, output block and rail decision with a timestamp, policy version and request identifier. Confirm with counsel whether your assistant is high-risk before designing the schema. |
| Log retention (EU AI Act Article 19(1)) | Providers keep automatically generated logs for at least six months unless other law provides otherwise. | Route guard decision logs to retained storage with a six-month minimum, and check whether sector rules require longer. Confirm retention periods with your legal team. |
| Self-hosted detection models | Classifier and safety models used by the guard must run inside your environment. | Inventory every model the current scanners load, including Hugging Face downloads, and pick a replacement that can load the same weights from an internal registry. Plan how updates arrive without online access. |
| Validator and package sourcing | Validator packages and rails configurations must be pulled from a controlled source and pinned. | Mirror Python packages and Guardrails Hub validators into an internal repository. Pin versions in lockfiles so a guard change cannot reach production unreviewed. |
| Python runtime compatibility | NeMo Guardrails supports Python 3.10, 3.11, 3.12 and 3.13. The last LLM Guard release requires Python 3.10 or newer but below 3.13. | Choose the interpreter that every candidate supports, and test the scanner-replacement stack on that exact version. Note that moving to 3.13 excludes the LLM Guard release line. |
| Scanner coverage parity | Each input and output scanner you rely on has a named replacement rail or validator, or a documented decision to drop it. | Map LLM Guard scanners such as Prompt Injection, Anonymize, Secrets, Invisible Text, Sensitive and Malicious URLs to rails or validators. Record any gap as an accepted risk with an owner. |
| Parallel run and comparison testing | Both guards run on the same traffic before cutover, and results are compared for misses and false positives. | Shadow-route a sample of production-like prompts through both stacks, log disagreements, and review them with the policy owner. Define exit criteria in writing before the run starts. |
| Policy versioning and rollback | Rail configurations and validator settings change under version control with a tested way back. | Store Colang or config files and validator settings in git, tag each release, and keep the LLM Guard configuration runnable until the parallel run closes. |
Which LLM Guard scanners you are likely running
Input scanners run on the user prompt before it reaches a model. LLM Guard documents 15 of them, including Anonymize, Ban Topics, Invisible Text, Prompt Injection, Secrets, Token Limit and Toxicity. Teams usually adopt a few of these first, so the goal is to find which ones are instantiated in code and which are only present in a copied example. Search the codebase for the scanner class names, then check each call site to record the scanner, its configuration and the application that owns it.
- Anonymize and Secrets: Record which entity types are detected and whether the scanner replaces values or only flags them, because replacement changes what the model receives.
- Prompt Injection and Invisible Text: Record the threshold and whether a match blocks the request or only logs it.
- Ban Topics: List the topic labels and any custom wording, since these are usually written by the product team and rarely documented.
- Token Limit: Record the limit and where it is set, since it often sits in application config rather than the scanner call.
- Toxicity: Record the threshold and whether the same threshold applies to all applications.
For each scanner, note the owning application, the model it guards and whether a block produces a user-visible refusal. Confirm these details against your own repository and gateway logs, not against the scanner documentation alone.
Output scanners run on model responses before they reach the user or downstream systems. LLM Guard documents 20 of them, including Deanonymize, Factual Consistency, JSON, Malicious URLs, No Refusal, Sensitive and Toxicity. Output scanners are often configured more loosely than input scanners and are easier to forget, so they need the same inventory discipline. Pay particular attention to any scanner that changes the response, such as Deanonymize, which restores values that were masked on input.
- Deanonymize: Confirm it is paired with an input Anonymize scanner and that the mapping store is protected and retained according to policy.
- JSON and Factual Consistency: Record whether they block, retry or only annotate, and which downstream parser depends on the output format.
- Malicious URLs: Record the URL policy and whether it applies to links in generated answers only or also to retrieved content.
- No Refusal and Sensitive: Record the phrases or categories used and which applications rely on them.
- Toxicity: Record the threshold and whether output is ever shown to end users without this check.
Record each output scanner next to the input scanner that pairs with it, because a replacement that covers one direction but not the other leaves a gap. Confirm the active configuration in the deployed service, since defaults in a library version can differ from what was configured in production.
A scanner name tells you little on its own. The behavior that matters is the threshold, the action taken on a match and the application that calls it. Two applications that both use Toxicity can have different thresholds and different responses to a block, and a migration that copies only the scanner name will change behavior silently. Build a register with one row per scanner, per application and per environment, so differences are visible before cutover.
- Threshold values: Export the values from the deployed configuration or from the code that constructs each scanner, and keep them with the commit or release they came from.
- Action on match: Record whether the request is blocked, the output is rewritten, the event is logged or the result is only scored.
- Call path: Identify whether the scanner runs in the application, in a shared API server or at the AI gateway.
- Fail behavior: Record what happens if the scanner errors or times out, since many teams fail open without deciding to.
- Logging: Record which fields are written to logs for each decision, because this determines what evidence exists for audits.
Illustrative example: a single Toxicity scanner might block at one threshold in a customer chat and only log in an internal tool. Your register should show both rows. Confirm each value with the team that owns the application, not only with the platform team.
Once the register exists, each row needs a target. The archived project has no maintenance, so new work should move to maintained guardrails. NVIDIA NeMo Guardrails provides input, dialog, retrieval, execution and output rails configured in a config file with optional Python actions and Colang files. Guardrails AI provides validators from Guardrails Hub that can be chained into input and output guards. Map each scanner to the rail or validator that performs the same job, and mark the rows where no direct equivalent exists.
- Direct matches: Some scanners, such as regex or toxicity checks, usually have a validator or classifier equivalent to confirm in the target.
- Partial matches: Some scanners, such as Anonymize with a specific reversible mapping, may need custom code or a different architecture.
- No match: Scanners with no equivalent should be listed as open decisions, with a documented owner.
- Configuration translation: Thresholds do not carry over automatically, so each target needs its own tested value.
- Versioning: Record the target library version, since NeMo Guardrails v0.24.1 and Guardrails AI v0.11.0 are both current releases that change over time.
Confirm every mapping by running the same test prompts through both systems. Do not assume equivalence from scanner names.
Mapping LLM Guard scanners to NeMo Guardrails and Guardrails AI
LLM Guard's input and output scanners do not map one to one onto a replacement. NeMo Guardrails and Guardrails AI cover different parts of the job, so most migrations use one framework for conversational rails and the other for validation, or a single framework with custom code for the gaps. The cards below describe what each target offers and what to confirm before cutover.
NeMo Guardrails as the rail framework
NeMo Guardrails is Apache-2.0 licensed and organizes controls into input, dialog, output, retrieval and execution rails, configured in config.yml with Colang files and optional Python actions. It is the closer match when your LLM Guard use centers on prompt injection, jailbreak attempts, topic restrictions and output moderation in a conversational flow. Check whether the built-in self-check and jailbreak detection models meet your accuracy needs on your own traffic, and confirm which models must be downloaded for an air-gapped network.
Guardrails AI validators from Guardrails Hub
Guardrails AI builds guards from validators published on Guardrails Hub, such as regex matching or toxicity detection, installed separately and chained together. This maps well to LLM Guard's pattern-based output scanners, including checks for sensitive content, malicious URLs and JSON structure. Confirm that each Hub validator you need exists, is maintained, and behaves as LLM Guard's scanner did on your data. Where no validator matches, plan a custom validator and test it.
Structured output with Pydantic models
Guardrails AI can generate output that matches Pydantic models using function calling or prompt optimization. LLM Guard's JSON output scanner checks that a response is valid structure, so this is a direct replacement for that behavior and goes further by shaping the output. Decide whether your applications need the model to produce a schema or only need the output checked. Test schema failures, since retries and fallbacks change latency and what users see.
Custom actions for scanners without a match
Some LLM Guard scanners, such as Factual Consistency or Invisible Text, may need custom validators or NeMo Guardrails actions rather than a stock rail. Treat each custom check as new code with its own tests, owner and version. Write the expected behavior from the LLM Guard scanner's documented output first, then compare results on a labeled sample. Confirm this mapping during testing, not during planning, because the gap only appears against real prompts.
Parallel running before cutover
Run LLM Guard and the replacement side by side on the same traffic, with the replacement in log-only mode, before any block decision moves. Compare which requests each stack flags, and review disagreements manually to find missed attacks and new false positives. Record both decisions so audit logs show what changed and when. Set the cutover criteria in writing before the parallel run starts, including what counts as acceptable disagreement.
Inventory before mapping
Start by listing every LLM Guard input and output scanner in use, the thresholds and configuration applied to each, and the applications that call it. LLM Guard documents 15 input scanners and 20 output scanners, so most deployments use a subset. Record which scanner blocks, redacts or only logs, because replacement behavior must match each mode. Also record whether models hosted on Hugging Face are pulled at runtime, since the archive means those assets will not receive updates.
How the parallel run and cutover are sequenced
The sequence moves production guardrail checks from LLM Guard to a maintained stack without a gap in coverage. Each phase produces a named artifact, and no traffic moves to the new stack until the discrepancy evidence supports it.
Inventory every LLM Guard scanner in use
Your engineers list each input and output scanner that the application calls, with its configuration, thresholds and the model it loads. Trace the call path from the application through any wrapper code, middleware or gateway hook to find scanners that run only on certain routes or tenants. The output is a scanner inventory that records each scanner's purpose, its position (input or output), its owner and the route it protects. The step is done when the application owners confirm the list matches what production actually executes, not only what the code base suggests.
Map scanners to rails and validators
The security and platform teams map each inventoried scanner to a NeMo Guardrails rail or a Guardrails Hub validator, or mark it as a gap. Record the input or output stage, the expected action (block, redact or flag) and the response your users see. The output is a mapping workbook and a gap list, where each gap has an owner and a documented decision: accept the risk, build a custom validator or add a different control. The step is done when every scanner has a mapped target or a signed-off gap decision.
Build rails and validators in a test environment
Engineers implement the mapped rails in NeMo Guardrails configuration files, or install and chain validators in Guardrails AI, and build any custom logic the gap list requires. Where existing Colang 1.0 configurations are involved, run the migration tooling and review the converted output manually, since NVIDIA states that converted configurations still need review. The output is a versioned guardrail configuration with a test suite that covers each mapped scanner's known good and known bad inputs. The step is done when the suite passes in a non-production environment with the same model versions as production.
Run both stacks in shadow mode
Send copies of live traffic through both LLM Guard and the new guardrails, while only LLM Guard's decisions reach users. Log every case where the two stacks disagree, including the input, the output, each verdict and the latency of each check. Review discrepancies in batches to classify them as missed detections, new false positives or differences in wording. The output is a discrepancy log with a documented disposition for each category. Confirm with your privacy and records owners that logging the shadow traffic is permitted, because the logs may contain personal data or sensitive prompts.
Tune thresholds on sample data
Using the discrepancy log and a labeled sample of production-like prompts and responses, adjust thresholds, topic definitions and redaction rules until the new stack meets the detection and false positive targets your risk owners set. Keep each configuration change in version control with a reason attached. The output is a tuned configuration, a set of test results per scanner category and a sign-off record from the risk owner. The step is done when the tuned stack's results are reproducible on a fresh sample, not only on the sample used for tuning.
Cut over in stages behind the AI gateway
Route a small, defined share of traffic through the new guardrails at the AI gateway, then widen the share in steps, watching blocked-request rates, latency and user complaints at each stage. Keep LLM Guard running in the path so that switching back is a configuration change at the gateway, not a redeployment. The output is a cutover runbook with stage gates, the rollback trigger for each gate and the on-call contact for each stage. The step is done when all traffic runs on the new stack for an agreed observation period and no rollback trigger has fired.
Decommission LLM Guard packages and models
After the observation period closes, remove the LLM Guard packages, pinned model files and API server deployments from the production images, build pipelines and container registries. Delete or archive the scanner configuration and document where the retained discrepancy logs and the mapping workbook are stored for audit purposes. The output is a decommission record that lists each removed package, model and endpoint, and confirms that no application still references them. Run a search across repositories and deployment manifests to verify the removal, because scanners often remain in forgotten services and test images.
How to run LLM Guard and the replacement side by side
Run the replacement in shadow mode before anything is switched off. Both stacks score the same prompts and model responses, but only the existing LLM Guard decision is enforced. The candidate rails or validators log their verdicts without blocking, and each request gets a record of both decisions. Disagreements, where one stack blocks and the other passes, are the work queue. Review them in a fixed cadence with a named owner from the security team and one from the application team, and record for each disagreement whether the old scanner was wrong, the new rail was wrong, or the policy needs to change. Log both verdicts in the same store as your AI gateway logs so reviewers see them side by side, and confirm with your logging team how long those records are kept.
Plan the NeMo Guardrails migration as a configuration review, not a file conversion. If the replacement uses Colang, note that NVIDIA documents a Colang 1.0 to 2.x migration using the nemoguardrails convert tool and says converted configurations still need manual review. Treat each converted flow as unverified until a reviewer has read it against the original intent, tested it with the disagreement set, and confirmed that flow ordering and the default language version in your config.yml match what you deployed. The Colang 2 migration guide is the starting point for that review. Keep the old scanner running on a Python version it supports: the last LLM Guard release on PyPI, 0.3.16, requires Python 3.10 or newer but below 3.13, so pin the shadow service to that range in a container image rather than letting it follow the platform default.
Define parity in writing before cutover. Parity should be a set of agreed thresholds, not a feeling that the new stack looks similar. Agree on the scanner-to-rail mapping for every scanner you rely on, the test corpus per risk (including your own prompts and responses, since vendor benchmarks may not reflect your traffic), the maximum acceptable rate of disagreements that are missed attacks or leaked data, and the rate of new false positives your application owners will tolerate. Confirm with the vendors and the maintainers of any Guardrails Hub validators which language and model versions your pinned release depends on. Cutover should happen only when every high-severity category meets its threshold for a full review cycle, and the signed decision, the corpus version and the disagreement log are retained as evidence of the change.
Evidence to keep from the migration
This table lists the records an auditor or internal risk team will ask for when LLM Guard scanners move to NeMo Guardrails rails or Guardrails AI validators. Each row names the artifact and the component that produces it, so the owning team can plan where each record is generated and stored. Treat the set as one trail: the inventory explains the mapping, the shadow-run logs prove the new controls behave like the old ones, and the retention settings show the logs still exist when someone asks.
| Evidence item | What it shows | Produced by |
|---|---|---|
| Scanner inventory | Every LLM Guard input and output scanner in use, with the prompts, models and applications that call it | Your scanner config files and application code, reviewed by the platform team |
| Mapping decisions | Which NeMo Guardrails rail or Guardrails Hub validator replaces each scanner, and why any scanner was dropped | Migration decision record kept with the change ticket |
| Shadow-run discrepancy logs | Requests where LLM Guard and the replacement disagreed, with the verdict from each side | Parallel run of both guards behind the AI gateway, compared by a script or reviewer |
| Threshold change records | Each score cutoff set or adjusted, the reason, the reviewer and the date applied | Versioned guard configuration and change approvals |
| Configuration versions for rails and validators | The exact config.yml, Colang files, Python actions and validator versions live at each cutover point | Source control tags or releases for the guardrail configuration |
| Blocked-request logs | What each guard blocked or redacted, which rule fired and which application made the call | Guard server or gateway logging, stored in your own log platform |
| Article 12 and 19 log retention settings | Confirmation that event logs are recorded and kept for the period your legal team sets, with at least six months required by Article 19(1) where it applies | Logging pipeline and storage policy, confirmed with counsel for each system in scope |
What teams ask before replacing LLM Guard
How much effort does replacing LLM Guard actually take?
The effort depends mostly on how many scanners your applications call and how much custom tuning sits behind them. LLM Guard documents 15 input scanners and 20 output scanners, so the first step is counting which ones are in use, not the total available. Each scanner in use needs a matching rail in NeMo Guardrails, a validator from Guardrails Hub, or a custom check, plus tests against your own traffic. Teams with a handful of scanners and default thresholds have a much smaller job than teams that tuned thresholds, wrote custom scanners or depend on several output checks at once. Ask any partner to size the work from a written inventory of your scanners, call sites and thresholds before agreeing a fixed scope.
Can our own team migrate from LLM Guard without outside help?
A team with Python experience and access to the application code can often do the mapping itself, but the hard part is validation, not coding. The work includes building a test set from real blocked and allowed requests, measuring how each replacement behaves on that set, and agreeing who signs off on changes to policy. Teams that already run a self-hosted AI gateway, such as LiteLLM or Apache APISIX, have an easier path because the guard calls already sit in one place. Teams without an owner for AI policy, or without time to review false positives, usually benefit from a partner for the parallel run and the cutover. DeployOpen plans, implements and documents this kind of work under a written, fixed scope.
How long should the parallel run last before cutover?
The parallel run should last until the replacement has been checked against enough traffic to cover the request types your users actually send. Length is set by traffic volume, the variety of use cases behind each application, and how often the policy changes during the run. Low-volume internal tools may need fewer weeks of comparison than customer-facing chat with many topics, but both need a period that includes peaks and unusual inputs. Keep LLM Guard running in logging-only mode during the overlap, record every disagreement between the two systems, and set exit criteria in writing before the run starts. Confirm the duration with your partner against your own traffic, since no published figure applies to every environment.
Should we move to NeMo Guardrails or Guardrails AI?
Choose NeMo Guardrails when you need dialog control, retrieval rails for RAG chunks or execution rails for tool calls in one configuration model. Choose Guardrails AI when your main need is input and output validation built from reusable validators, or structured output that matches Pydantic models. NeMo Guardrails is Apache-2.0 licensed, configures rails in the Colang language, and v0.24.1 was published on September 16, 2026. Guardrails AI is also Apache-2.0 licensed, with v0.11.0 published on August 14, 2026, and its guards are built from validators on Guardrails Hub. Many LLM Guard users mainly ran prompt injection, toxicity and secrets checks, which both tools can cover with different effort. Pilot both against your own scanner list before deciding.
What do we need from our internal engineers and risk owners?
You need engineers who know where guard calls sit in your code, a person who owns guardrail policy, and reviewers who can judge blocked and allowed requests. Engineers map each scanner to its replacement, change application code, and maintain the runtime that serves the guards. Policy owners decide which topics, personal data types and output rules are blocked, and they approve threshold changes during the parallel run. Security or compliance staff define the evidence they need, such as logs of blocked requests. Plan for these people to be available during the overlap period, because that is when most disagreements need a decision. Confirm named owners in writing before work begins.
Does migrating from LLM Guard change our compliance evidence?
It can, because the guard decisions you record today may not match what a replacement records. Under Article 12(1) of the EU AI Act, high-risk AI systems must technically allow automatic event logging over their lifetime, and Article 19(1) requires providers to keep those logs for at least six months unless other law provides otherwise. Your replacement should log blocked requests, the rule that fired and the version of the policy in force, so that evidence remains continuous across cutover. Keep the parallel-run comparisons as records of the change. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors and certification bodies issue any audit report or attestation.
What happens to our existing guard configuration during the move?
Your existing thresholds, allowlists and custom scanner settings become the starting inventory, and each one needs an explicit decision about whether it carries over, changes or retires. Export the settings from every application that calls LLM Guard, including the API server deployments if you use them. Store the new configuration in version control so that each policy change has an author, a reason and a rollback path. Running both systems in parallel lets you compare decisions on the same inputs before any setting is switched off. Confirm with your partner how configuration files will be versioned and reviewed before the first cutover.
Questions to ask an implementation partner
Start with the inventory method. Ask the partner how they will list every LLM Guard scanner your applications call, including the ones wired in through configuration files or gateway plugins that nobody on the current team wrote. A useful answer names the source of truth (code search, gateway configuration, request logs over a representative period) and says who signs off the final list. Ask how they will record scanner settings such as thresholds and anonymization entities, because those values move to a new platform with the rule and not with the scanner name.
Next, ask how Colang conversion will be handled if NeMo Guardrails is the target. NVIDIA's migration guide describes a nemoguardrails convert path from Colang 1.0 to 2.x and states that converted configurations still need manual review, so ask who performs that review, what test cases prove each converted flow behaves as the original did, and whether the partner will keep Colang 1.0 running where a rail has not yet been migrated. For Guardrails Hub validators, ask how each validator is paired with a rule and what happens to scanners with no close equivalent, which should be listed as open decisions rather than silently dropped.
Then ask about discrepancy review and the parallel run. Ask how the team will compare LLM Guard verdicts with the new rails on the same traffic, who decides whether a difference is a regression or an improvement, and how those decisions are logged for later audit. Ask where detection models and validator packages are stored, and whether the partner will mirror model files and packages into your own registry so that a future rebuild does not depend on a public archive. Finally, ask how false positives will be measured on your own data, not only on vendor benchmarks, and how the old scanner will stay runnable after archive, for example by pinning the last PyPI release, which is 0.3.16 uploaded on 19 May 2025, and by documenting the Python version it needs. Ask what the exit date for that pinned environment is and who owns it.
Integration points around the replacement
Replacing LLM Guard touches four components that sit on different trust boundaries: the gateway that enforces policy, the guardrails server that evaluates content, the observability store that keeps decision records, and the model endpoint that generates output. Each needs a clear owner and a defined place in the network before scanners are moved.
AI gateway as enforcement point
The gateway is where application traffic meets your guard policy, so it is the first boundary to design. Route every model call through it so that input rails run before a prompt reaches a model and output rails run before a response reaches a user or a tool. Check that the gateway can call an external guard service, that it fails closed when the guard is unreachable, and that it passes a request identifier downstream so blocked requests can be traced. Apache APISIX and LiteLLM both appear in the gateway topic, so confirm which one your team already operates before choosing the hook.
Guardrails server inside the boundary
The guardrails server runs the rails that replaced LLM Guard's scanners. NeMo Guardrails can run as a Python library or as a server that loads configurations from a folder and exposes an HTTP API, and NVIDIA also offers a production container deployed with Helm on Kubernetes. Guardrails AI can run as a Flask REST service started with guardrails start. Place this service in the same network zone as the gateway, not in a vendor cloud, so prompts are not sent outside your boundary during evaluation. Confirm which detection models it downloads and whether they can be mirrored for air-gapped use.
Observability for decision logs
Each block, redaction and pass decision is evidence, and the replacement is only as good as the record it leaves. Send guard verdicts, rule names and policy versions to a self-hosted observability store such as Langfuse or Phoenix, keeping prompts and outputs in your own storage. Treat these records as sensitive, since they may contain the very data the guard was meant to protect, so restrict access and set retention deliberately. For EU AI Act scope, Article 12 concerns automatic event recording and Article 19 sets a minimum keep period of six months for providers of high-risk systems, so confirm with counsel whether your system falls in scope.
Model endpoint behind the guards
The model endpoint is the last boundary and the one the guards exist to protect. vLLM serves an OpenAI-compatible endpoint for shared GPU capacity, and Ollama suits smaller or local deployments. Only the gateway should hold credentials for this endpoint, so applications cannot bypass the rails by calling the model directly. Verify that network policy blocks direct access from application subnets, and that changing the model version does not silently change which guard thresholds were tested. Record the model identifier alongside each guard decision so an audit can reconstruct what was evaluated.
How DeployOpen scopes and runs a guardrail migration
DeployOpen approaches the move off LLM Guard as a fixed-scope migration: we inventory the scanners your applications call, map each one to a maintained rail or validator, and run old and new controls side by side until the evidence supports cutover. The work happens inside your own infrastructure, and the written scope sets what changes and what stays the same.
How we scope
- We inventory every LLM Guard input and output scanner that your applications, gateway and pipelines call today.
- We map each scanner to a NeMo Guardrails rail or a Guardrails Hub validator, and record any scanner with no direct equivalent.
- The written scope names the models, environments and applications covered, with deliverables and timelines agreed before work starts.
How we build
- We configure the replacement rails or validators in your environment and keep policies in version control with review steps.
- We run LLM Guard and the new guardrails in parallel, comparing blocked and allowed decisions on representative traffic from your own use cases.
- We test each replacement control against documented attack and leakage cases, and log the results as cutover evidence.
How we hand over
- We deliver the scanner inventory, mapping table, configuration files, test results and a documented cutover and rollback plan.
- We train your engineers to change policies, add validators and review blocked-request logs without depending on archived code.
- We document where LLM Guard detections had no replacement and what compensating control or accepted risk your team needs to approve.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after the migration is complete.
- We can track releases of NeMo Guardrails and Guardrails AI and test upgrades in staging before they reach production.
- We confirm with your team which support boundaries, response expectations and reporting cadence apply under the retainer before it starts.
