Which open-source LLM guardrails fit your situation
If you are putting an LLM application in front of users and need runtime input and output checks, choose NeMo Guardrails when you want a maintained, dialog-aware rail system configured in Colang, and choose Guardrails AI when your priority is validator-based guards and structured output that matches Pydantic models. Keep LLM Guard only for supporting existing deployments, because its repository was archived on July 9, 2026 and it receives no security fixes. Both NeMo Guardrails (Apache-2.0, v0.24.1 on September 16, 2026) and Guardrails AI (Apache-2.0, v0.11.0 on August 14, 2026) are actively released, so the choice comes down to how your application is built and who will maintain it.
The decision depends more on architecture and team skills than on a feature checklist. NeMo Guardrails suits teams that want to define conversational boundaries, topic control and rails for retrieval chunks and tool execution in one config.yml with Colang files, and that can invest in learning Colang. Its Colang 1.0 default and the 2.x migration path mean you should plan a manual review of any converted configuration. Guardrails AI suits teams whose engineers already work in Python, want validators chained from Guardrails Hub, or need structured data from the model. Its standalone Flask service, started with guardrails start, lets teams in other languages call it through the OpenAI SDK.
Before you commit, confirm three things in your own environment. First, test latency and model dependencies, since some rails call classifier or LLM models and you need to know which ones run on your hardware. Second, check which Python versions your platform supports, because NeMo Guardrails lists Python 3.10 through 3.13. Third, decide who owns the configuration after launch, including reviewing rails when a model or policy changes. Those answers, more than any single feature, will tell you which tool your team can run inside its own infrastructure.
Which guardrail fits each scenario
This table maps common team situations to the guardrail option that fits best and the condition that would change the choice. Read each row against your own runtime architecture, staffing and existing deployments before shortlisting. The main takeaway is that NeMo Guardrails and Guardrails AI are both maintained options, while LLM Guard is archived and should only appear in existing deployments.
| Scenario | Best fit | Why |
|---|---|---|
| Dialog flow control and topic rails | NeMo Guardrails | Its Colang language models dialog flows, input, output and retrieval rails in one configuration. This fits if conversation steering matters more than schema validation. If your flows are simple, a validator library may be enough. |
| Validators chained onto structured JSON output | Guardrails AI | Validators from Guardrails Hub chain onto output guards, and it generates output matching Pydantic models. This changes if your main need is dialog control, or if validator quality for your domain is not yet proven. |
| Existing LLM Guard deployment | Keep running, then plan migration | LLM Guard is archived and receives no security fixes, so existing use should be isolated and monitored. Plan a move to NeMo Guardrails or Guardrails AI, and confirm which scanners you depend on before choosing the target. |
| Kubernetes-hosted production service | NeMo Guardrails | NVIDIA offers a production microservice on the same configuration model, deployed on Kubernetes with Helm, alongside the open-source server and Dockerfile. Confirm the microservice licensing terms before relying on it, since they differ from the Apache-2.0 library. |
| No ML engineers available | Guardrails AI | Its validators are installed and chained as configured components, and it can run as a Flask REST service. This fits when rules are mostly regex or simple checks. If you need custom classifiers or Colang flows, budget for specialist skills first. |
| Teams that need built-in detection models | NeMo Guardrails | The library includes self-check moderation, hallucination detection, jailbreak and injection detection and topic safety models. Confirm model size, hardware and offline model availability in your air-gapped or restricted environment. |
| Teams already standardizing on LangChain | NeMo Guardrails | LangChain integration is opt-in through an environment variable. This fits if you want that integration. If you do not use LangChain, the opt-in setting is irrelevant and does not change the choice. |
Three open-source LLM guardrails and their licenses
Three runtime guardrail options appear on most shortlists, and their maintenance positions differ. Two are actively released under permissive licenses, and one is archived and no longer receives security fixes. License and maintenance status should narrow the field before you compare approaches, latency or model dependencies.
NeMo Guardrails
NeMo Guardrails is NVIDIA's Apache-2.0 toolkit for runtime rails, configured with a config.yml file, optional Python actions and Colang files. It covers input, dialog, retrieval, execution and output rails, and its server exposes a /v1/chat/completions endpoint that applications can call in place of a direct model call. The repository is actively released, with v0.24.1 published on September 16, 2026. Check the Colang 1.0 to 2.x migration, since NVIDIA says converted configurations still need manual review.
Guardrails AI
Guardrails AI is an Apache-2.0 library that builds input and output guards from validators, such as regex matching or toxicity detection, and chains them together. It also generates structured output that matches Pydantic models and can run as a Flask REST service started with guardrails start. Validators are installed separately from Guardrails Hub, so each one needs its own license and maintenance review. The project released v0.11.0 on August 14, 2026. Decide early whether to keep the anonymous metrics reporting that guardrails configure asks about.
LLM Guard
LLM Guard is Protect AI's MIT-licensed scanner library, with 15 documented input scanners and 20 output scanners, usable as a pip library or an API server. The repository was archived on July 9, 2026, and the README says it is no longer under active development or maintained. That means no security fixes for new findings. Teams already running it need a migration plan toward a maintained option, and new rollouts are not recommended. The last PyPI release is 0.3.16, uploaded on May 19, 2025.
Licenses and third-party terms
Apache-2.0 and MIT are both OSI-approved open-source licenses, but they cover the published code only. NVIDIA's NeMo Guardrails microservice is a separate production container deployed on Kubernetes with Helm, so its terms are a separate check. Guardrails Hub validators are published individually and carry their own terms. Ask each vendor for written terms before deployment, and confirm the license of any detection models your configuration pulls in, particularly for air-gapped networks.
How the three options compare on capability and operation
This table compares NeMo Guardrails, LLM Guard and Guardrails AI on license, maintenance, approach, rail types, configuration, Python support, deployment modes, structured output and model dependencies. Read it as a checklist: LLM Guard's archive status matters more than any feature row, and latency under your own load must be measured in your environment.
| Capability | NeMo Guardrails | LLM Guard | Guardrails AI |
|---|---|---|---|
| License | Apache-2.0 | MIT | Apache-2.0 |
| Maintenance status | Actively developed; v0.24.1 released September 16, 2026 | Archived July 9, 2026; no longer maintained and no security fixes | Actively developed; v0.11.0 released August 14, 2026 |
| Approach | Programmable rails that intercept inputs, dialog, retrieval chunks, tool calls and outputs | Library of input and output scanners applied to prompts and responses | Input and output guards built from validators, plus structured output generation |
| Rail or guard types | Input, dialog, retrieval, execution and output rails | 15 input scanners and 20 output scanners documented | Validators from Guardrails Hub, chained into input and output guards |
| Configuration language or format | config.yml with optional Python actions and Colang 1.0 or 2.0 (1.0 is the default) | Python scanner objects in code; API server configuration per deployment | Python guard definitions; validators installed separately with pip |
| Python version support | Python 3.10, 3.11, 3.12 and 3.13 | Last PyPI release 0.3.16 requires Python 3.10 or newer but below 3.13 | Confirm the current supported range in the project README before pinning |
| Deployment modes | Python API, guardrails server loading a config folder, Dockerfile for a server image, and NVIDIA's Kubernetes microservice with Helm | Python library installed with pip, or an API server | Python library, or a standalone Flask REST service started with guardrails start |
| Structured output | Not a stated core function in the supplied facts; confirm before relying on it | Not a stated core function in the supplied facts; confirm before relying on it | Generates output matching Pydantic models using function calling or prompt optimization |
| Model dependencies | Optional self-check, fact-checking, hallucination, content safety and topic safety models, plus third-party integrations; LangChain integration is opt-in | Includes models hosted on Hugging Face, which the project states are no longer maintained | Works with proprietary and open-source models; validator choice determines model needs |
| Latency under load | Measure in your environment with your rails and models enabled | Measure in your environment, including model-based scanners | Measure in your environment, including validator chains and any LLM calls |
How open-source LLM guardrails differ in latency and model dependencies
Every guardrail check costs time, and some checks need a second model to run. The gap between tools is less about whether they block bad inputs and more about how many model calls each request triggers. NeMo Guardrails ships self-check input and output moderation, fact-checking and hallucination detection, plus NVIDIA content safety and topic safety models, so a rail that uses an LLM or classifier adds a call on the request path. Guardrails AI validators vary widely: a regex validator is a local string check, while a toxicity or hallucination validator may load a model. LLM Guard scanners such as Prompt Injection also differ in what they load, but its maintenance status means you should not plan new rollouts around it.
Four factors drive latency in any chain: the number of rails or validators you enable, the number of model calls each one makes, where classifiers are hosted (in-process on the same hardware, a sidecar, or a remote endpoint across a network boundary), and whether output is streamed to the user or held until every output rail has finished. A chain of five rails that each call a remote model can cost far more per response than one local regex check, and streaming changes when the user first sees text but not whether a rail can still block it. Ask each vendor or your own engineers which rails call a model, which models they load, and whether those models run on CPU or GPU.
Benchmark each chain on your own hardware and with traffic shaped like production, including long prompts, retrieval chunks and tool calls. Measure p50 and p95 latency and error rates with each rail added, one at a time, so you can see which rail costs the most. Record where model weights are stored, who downloads them, and whether your network policy allows those downloads, because air-gapped or restricted environments may need the models mirrored in advance. Confirm the current model list and licenses for each classifier with its publisher, since these change between releases.
What running each option inside your infrastructure involves
NeMo Guardrails can be embedded in an application through its Python API or run as a guardrails server that loads a configuration folder and exposes an HTTP API. Your team owns that folder: the config.yml file, the Colang files that define input, dialog, retrieval, execution and output rails, and any optional Python actions that call your internal systems. Colang 1.0 is the default, and Colang 2.0 is also supported, so pick one version and record it in your standards, since NVIDIA documents a migration path that still needs manual review. For a container, the project provides a Dockerfile to build a server image. For multi-replica production on Kubernetes, NVIDIA offers a separate NeMo Guardrails microservice, a production container image built on the same configuration model and deployed with Helm, which shifts chart values, image sourcing and upgrades into your platform backlog.
- Configuration ownership: Your team maintains
config.yml, Colang files and Python actions under version control, with review for every rail change. - Model and classifier downloads: Confirm which self-check, content safety, topic safety or jailbreak models your rails call, where their weights come from, and whether they can be mirrored into an air-gapped registry.
- Runtime choice: Decide between the Python API inside an existing service, the guardrails server image, or the Helm-deployed microservice, because each carries different upgrade and scaling duties.
- LangChain opt-in: LangChain integration is off until you set
NEMOGUARDRAILS_LLM_FRAMEWORK=langchain, so document that setting in deployment manifests.
Before an air-gapped install, ask NVIDIA and your own security team which packages, container layers and model artifacts must be pulled from outside your network and how each will be verified.
Guardrails AI installs with pip, and its validators are not bundled with the core package. Each validator is published on Guardrails Hub and installed separately, then chained into an input or output guard. Your team owns the guard definitions, the validator versions you pin, any custom validators you write, and the decision to contribute them back to the Hub. For a shared service, guardrails start runs a standalone Flask REST service that clients can call through the OpenAI SDK, so the service becomes an internal component with its own authentication, logging and patching. The guardrails configure command asks whether to enable anonymous metrics reporting, and that choice should be made and recorded deliberately during setup, not accepted by default in a controlled environment.
- Validator inventory: List each Hub validator, its version and its source, since each one is installed separately and may pull its own dependencies.
- Service ownership: Your platform team runs the Flask service, including process management, TLS termination and access control in front of it.
- Custom validators: Your application team owns any validator written in-house, along with its tests and documentation.
- Telemetry decision: Record the answer given to the anonymous metrics prompt and confirm that the setting matches your data egress policy.
For an air-gapped install, confirm with Guardrails AI how Hub validators are fetched and whether they can be mirrored to an internal package index, and test that each validator runs without outbound network calls.
LLM Guard can be used as a Python library installed with pip or deployed as an API server. Both paths carry the same core risk: the project's repository was archived by its owner on July 9, 2026, and its README states that the project and its Hugging Face models are no longer under active development or maintained. The last PyPI release, 0.3.16, was uploaded on May 19, 2025, and requires Python 3.10 or newer but below 3.13. Your team would own the scanner selection, the Python environment pinned to that version range, the model artifacts the scanners load, and every security fix, because no upstream fixes will be issued. DeployOpen's position is to support existing LLM Guard deployments and help plan a move to maintained guardrails such as NeMo Guardrails, not to recommend new rollouts.
- Unpatched dependencies: Every transitive package and model file remains yours to track, since the archive means no security fixes will arrive upstream.
- Python version lock: Keep the runtime within the supported range, and plan for the constraint on upgrades to newer interpreters.
- Scanner inventory: Document which of the input and output scanners are enabled, because each one changes what the guard can detect.
- Migration planning: Map each scanner to its replacement in your target platform before the existing deployment is retired.
For an air-gapped environment, confirm that every model the enabled scanners need is already staged internally, and record the archive date in your risk register so the exception is reviewed.
How to choose and pilot a guardrail in CI and staging
This sequence turns a shortlist of runtime guardrails into a go or no-go decision backed by a fixed test set, measured latency and false positive rates, and a named owner for updates. It keeps the choice tied to your own threats and infrastructure rather than to vendor claims.
Define the threats and required responses
The security lead and the product owner list the threats the application must handle, such as prompt injection, data leakage, toxic output or off-topic requests, and decide the response for each: block the request, rewrite the content, or log it for review. Write each threat as a row with its response and the team that receives alerts. The step is done when the table is signed off by both the security and product owners, because that table becomes the acceptance criteria for every later step.
Shortlist options that match your stack
The platform engineering lead checks which language, runtime and deployment model your team already operates. NeMo Guardrails configures rails in Colang with a config.yml file and can run as a server image on Kubernetes through its Helm-deployed microservice, while Guardrails AI installs validators from Guardrails Hub with pip and can run as a Flask REST service. Confirm Python version support against your base images, since NeMo Guardrails supports Python 3.10 through 3.13. LLM Guard is archived, so it should not enter a new shortlist. The step closes with a short list of two or three candidates and a note on the Python version each one needs.
Build a fixed test set of attacks and benign prompts
The security engineers assemble one versioned test set containing known attack prompts, such as injection and jailbreak attempts, alongside ordinary prompts that real users send, including edge cases that should pass. Source attack examples from your own threat table and from scanners such as garak or promptfoo, and store the set in the repository so it runs in CI. The step is complete when the set has a recorded expected outcome for every prompt and the same set can be replayed against each candidate without changes.
Pilot each candidate in staging and measure results
The platform engineers deploy each shortlisted guardrail in a staging environment inside your own infrastructure, connect it to the same model and application path used in production, and run the fixed test set. They record the detection rate per threat, the false positive rate on benign prompts, and the added latency per request at realistic concurrency. Check that the model dependencies each guardrail needs, such as classifier models or external services, are available in your network. The step closes with a results table per candidate that compares all three measures on the same run.
Check Colang migration effort before committing
If the team plans to move between Colang versions, the engineers run the nemoguardrails convert tool on a copy of the configuration and review the output by hand. NVIDIA's migration guide says converted configurations still need manual review, so the team should estimate the review work from the size of its Colang files and the number of flows it uses. Record which rails needed rewriting and which tests changed. The step is done when the team has a written estimate of the review effort, based on its own files, that it can compare with the other candidates.
Decide go or no-go on defined criteria
The security lead, product owner and platform lead review the results against the criteria set in step one. A go decision requires that every threat in the table meets its agreed response, that false positives on benign prompts stay within a limit the product owner accepts, that latency fits the application's response requirements, and that the license and maintenance status allow continued use. Ask the vendor or project for the current release history and confirm the license text before signing off. A no-go is recorded with the reason so the next candidate starts from the same test set.
Assign ownership for rule and version updates
Before production, the team names an owner for the guardrail configuration, for each validator or model it depends on, and for upgrades to the library. The owner reviews release notes, reruns the fixed test set after each update, and approves changes in CI before they reach production. Document who receives alerts when a rail blocks traffic and who can change a response rule. The step is complete when the ownership record is in the runbook and a test run on the latest candidate version has passed in staging.
What do teams ask before running LLM guardrails themselves
What drives the cost and effort of self-hosting NeMo Guardrails, Guardrails AI or LLM Guard?
The main cost is engineering time, and the license is not the driver because all three libraries are open source. NeMo Guardrails and Guardrails AI are Apache-2.0 licensed, and LLM Guard is MIT licensed. Effort rises with the number of applications covered, the rails or validators you need, the models or external services the rails call, and the hosting you choose for the server. Recurring work includes upgrades, regression tests for conversation flows, and monitoring of false positives. Ask any provider for a written scope that lists who builds the rails, who hosts the models, who runs upgrades, and who owns the evidence. The quote should separate one-time build work from ongoing support.
Can a team run NeMo Guardrails or Guardrails AI without outside help?
Yes, a team with Python experience and an engineer who can own the configuration can run either library in-house, and both are Apache-2.0 licensed with public repositories. NeMo Guardrails asks you to write a config.yml file, optional Python actions and Colang files, and Colang 1.0 is the default. Guardrails AI asks you to install validators from Guardrails Hub and chain them into guards. The harder parts are judging whether a rail blocks the right traffic, keeping conversation flows correct across upgrades, and running the server in your own cluster. Outside help is most useful for that testing and hardening work, not for the first line of code.
How long does a guardrails pilot take, and what sets the timeline?
No fixed duration applies, because the timeline depends on scope. The number of applications, the number of rail types in use, the languages and channels your users write in, and whether you run retrieval or tool-execution rails all change the work. Integration with your existing CI pipeline and with identity and logging systems adds time. A pilot covering one application with a few rails is a different job from a rollout across many services. Ask for a timeline in the written scope, with each milestone tied to a deliverable, and ask what the client must provide and when.
What does the work need from our team: Python, Colang, validators, model hosting and CI access?
The work needs several skills, and you should confirm each one before the engagement starts:
- Python skills to write actions, custom validators and the glue code that calls the guardrail server.
- Colang authoring for NeMo Guardrails, including a plan for the Colang 1.0 to 2.x migration, which NVIDIA says still needs manual review after conversion.
- Validator selection and tuning for Guardrails AI, using Guardrails Hub entries or custom validators.
- Model hosting for any self-check, fact-checking or moderation models you enable, along with the GPU or CPU capacity they need.
- CI access so that rail tests run on every change to prompts, flows and models.
Ask the provider which of these they will staff and which your team must own.
Is LLM Guard still safe to use after its archive?
LLM Guard should not be used for new rollouts, and existing deployments need a plan. The protectai/llm-guard repository was archived by its owner on 9 July 2026, and its README says the project and its Hugging Face models are no longer under active development or maintained. An archived project receives no security fixes, so any new prompt injection or model issue will stay open unless your team patches it. The last PyPI release, 0.3.16, was uploaded on 19 May 2025. Teams already running it should inventory where it is deployed, pin versions, and plan a move to a maintained option such as NeMo Guardrails or Guardrails AI. Confirm that migration scope in writing before work starts.
Do guardrails stop prompt injection on their own?
No single guardrail stops prompt injection on its own, so treat rails as one layer among several. NeMo Guardrails includes jailbreak and injection detection and Guardrails AI offers validators you can chain, but detectors can miss novel phrasing, and they can also block legitimate requests. Limit what tools and data the model can reach, keep privileged actions behind separate checks, and test the application with adversarial inputs before release. Testing tools such as garak, PyRIT and promptfoo are a separate layer that checks whether your controls hold. Ask any provider how they will measure detection and false positive rates on your traffic.
What should we ask for in a written scope and quote?
Ask for a written scope that names each application and rail or validator in scope, the models and external services involved, and the hosting target in your own cloud, data center or air-gapped network. The quote should list deliverables, acceptance tests, the CI integration, documentation, and the handover of configuration and runbooks. Ask which items are one-time and which recur, such as upgrades, support and continuous testing, and what triggers a change order. Confirm who is responsible for model licenses, hardware and vendor access. DeployOpen plans and implements work in this way and provides fixed-scope engagements with a quote that follows from the written scope. Ask for that written scope before you compare options.
How DeployOpen scopes guardrail selection and rollout
DeployOpen plans, implements, integrates, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For runtime guardrails, that means starting from your application's traffic, threat model and deployment constraints, then building the rails you choose and handing over a configuration your team can run and change.
How we scope
- Confirm which LLM application, models and data flows the guardrails will sit in front of.
- Record which threats are in scope, such as prompt injection, data leakage and unsafe output.
- Agree in writing which guardrail libraries are evaluated, and which licenses and maintenance status apply.
- Fix deliverables and timelines in the written scope before any work starts.
How we build
- Deploy the chosen guardrail framework inside your environment, with no dependency on a vendor-hosted service.
- Write input, output and tool rails for your application, and check which models each rail calls.
- Measure added latency and resource use on your own hardware before setting production targets.
- Test guardrail behavior against attack sets, including a fresh round of checks before go-live.
How we hand over
- Document the configuration files, custom actions, model dependencies and known gaps in each rail.
- Provide runbooks for updating rules, rotating models and upgrading the framework version.
- Walk your engineering and security teams through the rails and the test evidence behind them.
- Keep a written record of decisions so auditors and reviewers can trace why each rail exists.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Retainer scope, response expectations and upgrade frequency are set in writing, not assumed.
- Re-test guardrails when the underlying framework, model or application behavior changes.
- Ask any provider, including DeployOpen, to confirm current support windows for the chosen release.
