How to use this LLM guardrails requirements checklist
A team about to put a language model in front of staff or customers needs agreed requirements before it compares NeMo Guardrails, Guardrails AI or controls built into an AI gateway. Without them, the shortlist tends to be decided by whichever demo looks best, and the team discovers later that nobody decided how far a blocked request may travel, which risks are in scope, or who approves a policy change. This checklist is meant to be worked through before any tool is shortlisted, so the choice of library or gateway follows from the requirements rather than shaping them.
The checklist is organized by functional, security, integration, operational and compliance requirements. Each item states what must be decided and carries a verification step, such as a test prompt set run against the candidate configuration, a log sample reviewed by the compliance lead, or a latency measurement taken on the target hardware. Security, platform and business owners should sign off on the functional and security sections together, because a rule that blocks too much is a business decision and a rule that blocks too little is a security decision. Record each sign-off with a date and a version of the policy it approved.
DeployOpen plans, implements and tests self-hosted guardrail controls and prepares the evidence that a compliance assessment will examine. DeployOpen does not issue audit reports, attestations or certifications. Those come from independent auditors and certification bodies, so the checklist should name which evidence the client needs to present and to whom. Any implementation partner, including DeployOpen, should be asked to confirm in writing which items it will deliver and which remain with the client.
Which functional requirements a guardrail must meet
This table lists the functional requirements to agree before choosing guardrails, what each one asks for, why it matters for the build and how to verify it. Use the verification column to turn each requirement into a test your team can run on its own data before any platform is selected. The main takeaway is that most guardrail failures come from untested input and output paths, not from missing features.
| Requirement | What it means for the build | Why it matters and how to verify |
|---|---|---|
| Prompt injection and jailbreak coverage | Input rails that screen user prompts before inference, plus checks on content retrieved from documents or tools | Injection attempts often arrive through pasted text or indexed files. Verify with a labeled set of attack prompts drawn from your own use cases, and record the miss rate per attack family. |
| Data leakage prevention | Input rails that detect identifiers and secrets, and output rails that block sensitive values before they reach the user | Leakage can happen in either direction. Verify by seeding test records into a sandbox knowledge base and checking whether they appear in responses and logs. |
| Unsafe or off-policy output | Output rails with moderation or content safety models, configured for your approved topics and tone | Policy definitions differ by organization. Verify with a labeled set of acceptable and unacceptable answers written by your business owners, not by the vendor. |
| Off-topic use | Dialog rails that define allowed subjects and the refusal behavior for everything else | Scope drift is common in general-purpose assistants. Verify by testing out-of-scope requests across each intended user group and checking refusal wording. |
| Input versus output checks | Separate rail sets for prompts and responses, with a defined order of execution and a clear fallback when a check fails | Checks on one side do not cover the other. Verify that each risk is tested at the point where it enters or leaves the system. |
| Retrieval rails on RAG chunks | Checks applied to retrieved text before it reaches the model, if the assistant uses a knowledge base | Only needed where documents feed the model. Verify with planted instructions inside test documents and confirm they are not followed. |
| Execution rails for tools and actions | Checks on tool calls and custom actions, with allowlists and approval steps for risky operations | Needed only where the model can call systems. Confirm which actions are reachable and that each has a scoped credential and a logged trace. |
| Structured output validation | Schema checks on responses that feed other systems, such as JSON fields or Pydantic-style models | Malformed output can break downstream code. Verify by sending malformed and edge-case responses and confirming the validator rejects or repairs them. |
| Latency budget per request | A measured time allowance for each guard, agreed with the application owner before design | Every added check adds response time. Verify by measuring guard latency under expected concurrency on the target hardware, not only in a lab. |
Setting a latency budget per request before choosing guards
Set the latency budget for each request path before you compare guardrail frameworks, because every check adds time in front of the model call (input rails, classifier calls, schema validation of inputs) and behind it (output rails, fact or hallucination checks, redaction, structured-output validation). A guard that is acceptable on a batch summarization job can make an interactive chat feel broken. Write the budget as a number of milliseconds added on top of the model's own time to first token and total generation time, agreed with the product owner and the platform team, and record which guard checks are allowed to consume it.
Set the budget per route, not once for the whole application. A chat turn, a retrieval call that screens RAG chunks before they reach the prompt, and a tool call that an agent wants to execute each have different tolerances and different risks. Retrieval rails run on many chunks per query, so their cost scales with context size. Execution rails run on each tool invocation, where a slow check can stall an agent loop. Ask the team to list every guard per route, mark which ones run synchronously and which can run in parallel or asynchronously, and decide which failures must block the request versus which can log and continue. The answer changes how much budget each route can spend.
Measure the budget on the hardware you will run, not on a laptop or a vendor demo. Run representative prompts, including adversarial and long-context ones, through the full path at expected concurrency, and record p50, p95 and tail latency per guard. NeMo Guardrails and Guardrails AI run checks differently: NeMo Guardrails executes configured rails in its own flow around the model call, while Guardrails AI applies validators that you chain into a guard, so the same policy can cost different amounts of time in each. Self-hosted classifier models need their own capacity planning: they consume GPU or CPU, have their own queueing behavior under load, and must be sized so that guard latency does not grow when traffic peaks. Confirm with your engineering team how many replicas of each classifier you need to hold the budget at peak load.
Security requirements for guardrail components
Guardrails are themselves software that sees prompts, policies and credentials, so they carry their own security requirements. The cards below set out what to keep inside the trust boundary, how to protect gateway secrets, and how to prove that policy changes and guard decisions can be trusted.
Classifier models and validators stay inside
Self-check moderation, jailbreak detection and content safety models, along with Guardrails Hub validator packages, should be fetched, stored and executed inside your own network. Confirm that no guard call leaves your boundary, including model downloads on first start and telemetry. The Guardrails AI quickstart asks whether to enable anonymous metrics reporting during configuration, so this answer belongs in your install record. Verify with network egress tests while guards run.
Gateway master keys and secrets
An AI gateway such as LiteLLM holds provider API keys, virtual keys and a master key that grants administrative control. Keep the master key in a secret manager rather than in configuration files, rotate it on a defined schedule, and restrict who can view virtual keys. Confirm which secret manager integrations your edition supports and whether they are included in your license tier. Verify by searching repositories, container images and logs for leaked keys.
Access control over policy files
Rail definitions, Colang flows, config.yml and custom Python actions decide what the assistant blocks, so write access to them is a security control. Limit edit rights to a named group, require peer review before merge, and separate the people who author policies from those who deploy them. Confirm that the gateway's admin roles can be scoped by team. Verify by attempting a policy change from an account outside the group and confirming it is refused.
Tamper evidence for policy changes
A silent edit to a rail can disable prompt injection checks without anyone noticing. Store policy files in version control with signed commits or protected branches, and record the deployed version identifier alongside every logged decision so you can tell which rules were active at a given time. Ask your implementation partner how configuration hashes are checked at startup. Verify by changing a rule in a test environment and confirming the change appears in both history and the decision log.
Data minimization in guard logs
Logs of blocked requests are useful for tuning, but they can copy the sensitive content the guard was meant to stop. Decide which fields to store, such as rail name, decision, score and policy version, and which to hash, mask or drop, such as raw prompts and personal identifiers. Set a retention period with your compliance team before rollout. Verify by sampling stored decision records and confirming they contain no unredacted personal data.
Adversarial testing of guard effectiveness
A guard that has never been tested against hostile input gives false assurance. Build a regression set of prompt injection, jailbreak and data extraction attempts drawn from your own use cases, run it against each policy version, and track both missed attacks and false positives on legitimate requests. Repeat the run after every model or policy change. Ask partners whether their testing covers both input and output rails, and what results they will hand over.
Integration requirements with the AI gateway and applications
Guardrails only protect anything when they sit on the request path between applications and models, so the gateway, identity provider, logging stack, model endpoints and incident process must agree on how requests are checked and recorded. Confirm each interface below with the vendor or in your own environment before choosing a guardrail framework.
Gateway hook for input and output checks
Decide where the guard runs relative to the gateway: as a pre-call and post-call hook inside the gateway, or as a separate guard service the gateway calls over HTTP. LiteLLM and Apache APISIX both sit in this position, but the hook mechanism, request body access and failure behavior differ by product and version. Confirm with the vendor whether a guard can block, rewrite or flag a response, and whether streaming responses can be checked before the first token reaches the user.
Identity mapping for per-user policy
Policies such as topic limits or data classes often differ by department, role or team, so the guard must receive a verified identity, not just an API key. Confirm how the gateway passes user and team identifiers to the guard, whether it accepts identity provider tokens, and which SSO options are included at your license tier. In LiteLLM, SSO with Okta, Microsoft Entra ID and generic OIDC is listed, but team-level guardrails and JWT authentication are described under the Enterprise tier, so confirm that scope.
Blocked-request records for observability
Every blocked, redacted or flagged request needs a record that links the policy version, the rail or validator that fired, the user and the gateway request identifier. Confirm where these events are written, whether they reach the observability stack without the raw prompt, and how long they are kept. Records that contain the rejected prompt become sensitive data in their own right, so decide on masking before the logs are shipped.
Model endpoint and provider connectivity
The guard needs a path to the classifier models and the protected model endpoints, and both paths must stay inside the approved network. Confirm which model endpoints the guard calls, whether self-hosted classifier models are reachable from the guard service, and what happens when a classifier endpoint times out. Fail-open and fail-closed behavior should be a written decision per use case, not a default inherited from the framework.
Ticketing and incident handling
A blocked prompt is often the first sign of an attack or a misconfigured application, so alerts from guard decisions need an owner and a route into the incident process. Confirm which alert thresholds trigger a ticket, who can change a policy in an emergency, and how the change is recorded. Check that the ticketing system receives event identifiers rather than raw prompt text, and that the on-call process covers gateway and guard outages.
Application contract for guard responses
Applications must handle a guard decision in a defined way, such as a refusal message, a redacted response or a retry with a narrower prompt. Agree on a response schema that includes a decision code and a request identifier, and confirm that the application can show the user a safe message without exposing the rule that fired. Test this contract with the application teams before the guard is placed in front of production traffic.
Policy configuration and versioning requirements
Guard policies are code, so they need the same controls as application code. In NeMo Guardrails, behavior lives in a config.yml file, optional Python actions and Colang files. In Guardrails AI, behavior lives in the chosen validators and the chain that combines them. Topic lists and regex patterns look like configuration, but a one-character change can open or close a path for prompt injection or data leakage. Agree that every change goes through version control, peer review, a staged rollout (test, then a limited pilot, then production) and a rollback procedure that restores the previous policy version within a time the business accepts. Confirm the exact rollback mechanism with the vendor and with your own deployment tooling, since neither library defines an enterprise change process for you.
Settle who approves changes before the first release. A workable model separates the people who write rails (security engineering or the platform team), the owner of the business scope (the product or process owner who decides which topics are in or out of bounds), and a reviewer from risk or compliance for changes that affect data protection or regulated outputs. Require a test suite to run on every proposed change. It should include a fixed set of benign and adversarial prompts, expected block and allow outcomes, and output checks on known sensitive values. Treat a planned migration as a change that needs this suite. NVIDIA documents a Colang 1.0 to 2.x migration using the nemoguardrails convert tool and states that converted configurations still need manual review, so a migrated policy should not reach production until its test results match the old behavior or every difference is signed off.
Every logged decision should carry the identifier of the policy version that produced it. Without this link, an incident review cannot show which rule blocked or allowed a response on a given day. Record a policy version or commit hash, the rail or validator that fired, and the decision in each log entry, and keep those records in the same retention store as the rest of the gateway logs. Ask the implementation partner to show how the policy identifier is written at inference time, how it is retained alongside the LiteLLM or other gateway logs, and how a reviewer can retrieve the exact configuration that was live at a given moment. Confirm these fields against your own log schema before you approve the design.
Operational requirements and how to verify each
This table lists the operational requirements a guardrail layer must meet after go-live, with the evidence to ask for against each one. Read each row as a question to put to a vendor or implementation partner, and treat a missing artifact as a gap to close before rollout. The main takeaway is that logging, update paths and failure handling need written answers, because they decide whether the guard layer can be defended to an assessor.
| Requirement | What to verify | Evidence to ask for |
|---|---|---|
| Logging of blocked and allowed requests | Each decision records the rail or validator that fired, the policy version, the input or output hash, the user or key identity, and the outcome | Sample log records from a test run, log schema, and a description of which fields contain raw prompt text |
| Retention for applicable rules | Retention periods are set per data class and match the duties that apply to your system, including EU AI Act Article 12 event recording and Article 19 retention of at least six months where it applies | Written retention schedule, legal review sign-off, and proof that deletion jobs run and are logged |
| Alerting on guard behavior | Alerts fire on spikes in blocks, sudden drops in block rates, validator errors, and timeouts that cause requests to bypass checks | Alert rules as configuration, a test showing a triggered alert reaching an on-call channel, and the runbook it links to |
| Model and validator updates in restricted networks | Classifier model files and validator packages are imported through an approved transfer path, hashed, and tested before promotion | Import procedure, checksum records, a dated change log of model and validator versions, and a rollback test |
| False positive review | Blocked samples are reviewed on a schedule by a named team, with thresholds adjusted through versioned policy changes | Review cadence, labeled sample sets from your own data, and the record of threshold changes with approver names |
| Uptime and scaling | The guard service has defined failure behavior (fail closed or fail open per rail), health checks, and horizontal scaling under peak load | Architecture diagram, load test results against your latency budget, and the documented fail-closed or fail-open decision for each rail |
| Incident response for a guard failure | A guard outage or bypass triggers containment, such as halting the affected endpoint, with notification steps and post-incident review | Written incident playbook, tabletop exercise record, and evidence that the gateway blocks traffic when guards are unavailable |
| Policy versioning and rollback | Every policy change is version controlled, tested against a regression set, and can be reverted to a prior version | Git history or equivalent, regression test results, and a timed rollback demonstration in a non-production environment |
Questions buyers ask before choosing LLM guardrails
What does the requirements work for LLM guardrails cost and what drives it?
The requirements phase costs most where the organization has to make decisions it cannot delegate: which risks matter, which data the guards must protect, and what a blocked request should look like to users. Cost comes from four components: policy engineering (writing and tuning Colang flows or validators), testing against the organization's own prompts and documents, compute for classifier models that inspect every request, and gateway work to attach the guards to the AI gateway and logging. Policy engineering and testing are mostly one-time at the start but recur with every policy change. Classifier compute and gateway capacity recur with traffic. Ask any quote to split these four lines, to state who does each one, and to say which items assume the organization supplies test data and a gateway owner.
How long does a guardrails requirements phase usually take?
The phase is shorter when risk owners, use cases and test material already exist, and longer when the organization has to agree on them first. Work that usually takes the most time is agreeing on acceptable false positive and false negative trade-offs with the business, deciding where classifier models will run, and collecting representative prompts and documents. No timeline is set here, so ask a partner to state the expected duration in the written scope and list which inputs from your side are assumed to be ready when the clock starts.
What must our internal team provide for guardrails to work?
Three roles are needed before a partner or library can produce useful controls. Risk owners decide which outcomes are unacceptable, such as disclosure of personal data or out-of-scope advice, and sign off on policy. Someone must provide labeled test data: real or representative prompts, documents and responses, marked as allowed or blocked, so each rail can be measured. A gateway owner must control how applications reach model providers, including keys, routing and where blocked-request logs are written. Confirm that these people have time allocated and that the labeled data can be used in a non-production environment under your data handling rules.
Can we build LLM guardrails ourselves with NeMo Guardrails or Guardrails AI?
Yes, a team with Python and security engineering skills can build a first set of controls with either library, but the effort is in policy and testing rather than installation. NeMo Guardrails (v0.24.1, released September 16, 2026) is Apache-2.0 licensed and configures input, dialog, retrieval, execution and output rails with a config.yml file and Colang files. Guardrails AI (v0.11.0, released August 14, 2026) is also Apache-2.0 licensed and builds guards from validators installed from Guardrails Hub. Self-build is realistic when you have an owner for policy, a test set and capacity to run red-team checks after each change. Without those, a partner can take on the requirements and testing work under a written scope.
How should we choose between NeMo Guardrails and Guardrails AI at the requirements stage?
Decide based on the control model you need, not on which library is more popular. NeMo Guardrails fits when you need conversational flow control, dialog rails and rails over retrieved chunks, with policies written in Colang; note that Colang 1.0 is the default and that NVIDIA's migration guide says converted configurations still need manual review. Guardrails AI fits when your main need is per-message validation of inputs and outputs, such as regex or toxicity checks, or enforcing structured output that matches a Pydantic model. Record the requirement in your checklist first: which rails you need, where the classifier models run, how policy versions are tracked, and what log each blocked request produces. Then test both libraries against that list on your own data before choosing.
Do LLM guardrails satisfy an audit on their own?
No. Guardrails are one control inside a larger set of evidence, and they do not make a system compliant by being installed. They produce records, such as policy versions, test results and logs of blocked requests, that an assessor can review. For high-risk AI systems under the EU AI Act, Article 12(1) requires technical support for automatic event logging over the system's lifetime, and Article 19(1) requires providers to keep those logs for at least six months unless other law provides otherwise. Those duties sit with the provider and need governance, retention and access processes around the guard logs. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors and certification bodies issue any audit report or certification.
What should we ask an implementation partner before signing a guardrails scope?
Ask for a written scope that names each risk the guards must address, the rails or validators planned for each one, and the test method that proves each guard works on your data. Ask where classifier models will run and who keeps them current, how policy changes are versioned, reviewed and rolled back, and what each blocked-request log contains and where it is stored. Ask which items depend on your team's inputs, what happens to false positives that block legitimate work, and how the gateway integration is tested. Confirm the deliverables, the timeline and what ongoing testing is offered as a retainer, and ask for a quote that follows from that scope.
Turning the checklist into a signed requirements baseline
The checklist only becomes useful when it is converted into a dated, owned baseline that a steering group signs before any library, gateway or partner is scored. Give every requirement a single named owner who answers for it: the security lead for the risk list and detection thresholds, the platform or AI engineering lead for the gateway integration and latency budget, the compliance lead for logging and retention, and the product owner for the policy catalog. Record the approval date and the version of the document, and keep a change log so that a later question about why a rail was accepted can be answered from the record.
Split every item into must-have or nice-to-have before scoring starts. A must-have is a requirement that, if unmet, stops the deployment: for example, detection models that run inside your boundary, blocked requests logged with enough detail to reconstruct the decision, and a policy change path that can be reviewed and rolled back. Nice-to-have items, such as a particular dashboard or a convenience for policy authoring, should be weighted lower so that a polished demo cannot outscore a tool that meets the controls. Ask each candidate to show how it meets each must-have, not to state that it does.
For each item, define the test evidence that closes it. A latency requirement closes when a load test on your own prompts and models shows the guard chain stays within the agreed budget at peak concurrency. A logging requirement closes when a sample of blocked requests can be traced to the policy version that produced them. A prompt injection requirement closes when a documented adversarial test set has been run and the results are retained. Use the baseline to score candidate NeMo Guardrails, Guardrails AI and gateway options, including the NeMo Guardrails repository and its releases, and to score partners on their ability to produce that evidence. Finally, schedule a review of the baseline whenever the chosen library or gateway publishes a new release, since a new version can change rail behavior, supported Python versions or configuration formats. Confirm the current release history of each component yourself before the review.
What to do in the first two weeks after the checklist is agreed
This sequence turns the agreed requirements into named owners, a labeled test set, a shortlist, a proof of concept result and a written scope. Each step closes with a document the next step depends on, so the platform decision rests on evidence from your own prompts and documents.
Confirm risk owners and sign-off authority
The security lead, the product owner for the assistant, and the compliance or privacy contact each confirm, in writing, which risks from the checklist they own and who approves changes to input and output policies. Someone in the business should own the acceptable-use scope, since that scope determines which topics the rails must refuse. The step is done when the checklist carries a named owner against every risk and a sign-off record exists for the latency budget and logging retention you agreed.
Collect representative prompts and documents
The product team and the security team pull a sample of real user prompts, the documents the assistant will retrieve from, and a set of known attack prompts such as jailbreak and injection attempts. Handle this material under your existing data handling rules, since prompts often contain personal data. The output is an inventory with source, owner and sensitivity label for each collection, stored inside your own environment. The step is done when the inventory covers every use case named in the scope.
Build the labeled evaluation set
Reviewers from the business and security teams label each sampled prompt and output as allowed, blocked, or needs redaction, and record the reason. Include adversarial examples and borderline cases that a false positive would affect. Keep a separate holdout set that the team does not use while tuning policies. The output is a versioned labeled set with a labeling guide and a reviewer agreement check. The step is done when two reviewers have labeled a shared sample and the disagreements have been resolved in writing.
Shortlist platforms against the baseline
The platform engineering lead compares candidate frameworks, such as NeMo Guardrails and Guardrails AI, and the gateway layer, such as LiteLLM or Apache APISIX, against the agreed baseline. Check each candidate for licensing, supported deployment model, where classifier models run, how policies are versioned, and whether blocked requests can be logged to your own storage. The output is a one-page comparison table with a pass or fail note against each checklist item. Confirm current release details and license terms with each project before you rely on them.
Run a proof of concept against the labeled set
The engineering team deploys the shortlisted options in a non-production environment that matches the target network, then runs the labeled set through each one. Measure detection rate, false positives on the allowed examples, added latency per request against the agreed budget, and whether every blocked request produces a complete log record. Run the adversarial set again after each policy change. The output is a results report with raw counts, the policy versions tested, and the failures that remain open. The step is done when the results are reviewed by the risk owners.
Request a written scope from the implementation partner
Share the checklist, the labeled set summary, the proof of concept report and the chosen architecture with the implementation partner, and ask for a fixed scope in writing. The scope should name the deliverables (policy configuration, gateway integration, logging, handover documentation), the responsibilities for model hosting and updates, the acceptance tests, and the exclusions. Ask how testing of the guards will continue after go-live. The output is a written scope and a quote that follows from it. The step is done when both parties have signed the scope and acceptance criteria are linked to the labeled set.
How DeployOpen scopes and delivers guardrail requirements and implementation
DeployOpen turns the requirements in this checklist into a written scope, then plans, implements, tests and documents the guardrails inside the client's own cloud accounts, data centers or air-gapped networks. Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
How we scope
- We confirm which risks the guardrails must cover, such as prompt injection, data leakage and unsafe output.
- We record which checks run on input, which run on output, and where each check sits in the AI gateway.
- We agree the latency budget per request and the self-hosted classifier models the guard path may call.
How we build
- We configure the chosen guardrail framework, such as NeMo Guardrails or Guardrails AI, with policies kept in version control.
- We wire guard decisions into the AI gateway so blocked requests are logged with the reason and policy version.
- We test each guard against attack cases and sample benign traffic to measure false positives on the client's own data.
How we hand over
- We deliver the policy files, configuration, deployment scripts and runbooks needed to operate the guardrails in-house.
- We document the test results, guard decisions and log retention settings that support later compliance reviews.
- We walk operations and security staff through policy change, rollback and how to add or retire a validator.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- We can rerun red-team tests and regression checks when models, policies or gateway versions change.
- We confirm with the client which logs, reviews and external audit steps remain their responsibility.
