NeMo Guardrails implementation plan and cost

What an implementation plan covers for NeMo Guardrails

The implementation plan for NeMo Guardrails decides what the guardrails will check, where they run and how they are maintained, before any rail is written. NeMo Guardrails is an Apache 2.0 toolkit that wraps an LLM application in rails. It supports five rail types: input rails, dialog rails, retrieval rails for RAG chunks, execution rails for custom actions and tools, and output rails. The plan chooses which of these apply to your application, then records how each one is configured in config.yml, written in Colang and implemented in Python actions, and whether it runs in your application process or as a separate service.

Teams usually start this plan for one of three reasons. A pilot LLM feature is moving toward production and needs controls it does not yet have. A compliance or customer security review asks for documented input and output controls. Or the model provider is changing, which alters how the existing rails behave and what they cost per request. Each trigger leads to a different rail set, so the first deliverable is a written list of the risks each rail addresses and the application paths it must cover.

Teams with strong Python and LLM application skills often build the first rails in-house, since the configuration model is code and files their developers already manage. Partners are more often brought in to scope the rail set, design the test corpus of prompts and expected outcomes, and set up production operation. Confirm with your own engineers which of these tasks they can own over the next year, because the test corpus and ongoing operation are where gaps tend to appear.

The plan is a scoping exercise. It ends with a written scope that lists rails, environments, test cases, handover items and the model and safety-model calls each rail makes. Cost follows from that scope, so ask any partner to quote against it and to separate one-time build work from recurring operation and upgrade work. Do not treat the plan as a price estimate until the scope is agreed.

What the implementation work includes

Each card names one workstream and what you receive at its end. Together they cover scoping, configuration, model wiring, deployment, testing and handover.

Rail selection workshop

We map input, dialog, retrieval, execution and output rails to the risks your application actually carries, such as prompt injection, off-topic use, leaked retrieval chunks, unsafe tool calls and unsafe responses. You receive a rail map that records which rails apply, which are deferred, and why.

Config, Colang and action design

We design the config.yml file, the Colang files and any Python custom actions that implement the rail map. Decide early whether Colang 1.0 or 2.0 fits your team, since Colang 1.0 is the default and migration later needs manual review. You receive versioned configuration files and an action catalog.

Model and safety-model wiring

We connect the rails to your LLM and to any self-check, content safety, topic safety or jailbreak detection models you select. Each added model call adds latency and cost per request, so confirm the call count per rail with your team before approving the design. You receive tested provider settings and a per-rail call inventory.

Library or guardrails server deployment

We deploy NeMo Guardrails either as a Python library inside your application or as a guardrails server in your infrastructure, using the configuration folder layout and Docker image where needed. The choice affects hosting, scaling and on-call work, so record it in the decision record. You receive a running deployment and its deployment notes.

Regression test suite

We build a suite of allowed and blocked prompts and expected responses drawn from your use cases and known attack patterns, and run it on every configuration change. Rail behavior can shift between 0.x releases, so this suite is the basis for upgrade decisions. You receive the test set, the runner and a pass and fail record.

Handover and decision record

We hand over configuration, runbooks for rail changes and incident response, and a decision record covering rail scope, model choices, library or server choice and known gaps. Your team owns these documents after handover, so confirm the named owners before the handover date. You receive the full handover package.

What drives cost and duration

This table lists the scoping factors that change the effort and recurring runtime cost of a NeMo Guardrails rollout, why each one matters, and how to keep it manageable.

Driver	Why it matters	How to keep it down
Number and type of rails enabled	Each extra input or output check adds model calls on every request and another set of test cases to write and maintain.	Start with the rails that address your confirmed risks, then add others only when a test shows a gap.
Safety-model calls per request	Self-check moderation, fact-checking, hallucination detection and content or topic safety models each add latency and compute, and they run on the request path.	Enable only the checks you need, measure added latency per check in staging, and confirm where each model runs and who pays for its compute.
RAG retrieval rails and chunks checked	Every retrieved chunk that a retrieval rail inspects adds work, so cost rises with chunk count and document size.	Limit the number of chunks checked per query and agree the chunk count with the team that owns retrieval.
Custom execution actions and tools	Actions that call internal systems need credentials, error handling, timeouts and tests against systems you may not control, which often takes more effort than the rails themselves.	Inventory every action and tool first, mark which ones touch sensitive systems, and reuse existing service accounts and API clients where policy allows.
Colang version and existing Colang 1.0 configurations	Colang 1.0 is the default, and existing Colang 1.0 configurations need conversion with the nemoguardrails convert tool plus manual review of what the tool misses.	Count existing Colang 1.0 files and flows before scoping, and decide early whether new work is written in Colang 2.x.
Number of policies and languages	Each policy and each supported language adds rules, example dialogs and test cases, and the depth of testing is set by how many policies must be verified.	Write policies in priority order, cover the highest-risk policies in full depth first, and confirm with stakeholders which languages must be in scope.
Latency budget and traffic volume	Tighter latency targets and higher request volumes make each model or safety-model call more expensive, and they may require caching, sizing or a separate serving tier.	Set the latency budget with the application owner before choosing rails, and load test representative traffic to size the deployment.
Deployment target and environments	Running as a library inside an existing application differs in effort from a separate server or a Kubernetes deployment with Helm, and each extra environment adds configuration and promotion work.	Decide between the Apache 2.0 library and the NeMo Guardrails microservice early, and reuse one configuration set across environments with only endpoint and credential changes.
Logging and review requirements	Logging blocked requests and reviewing them for false positives needs storage, access control, retention rules and a review workflow owned by someone.	Define which events are logged, who reviews them and how often, and reuse your existing logging platform where possible.
Compliance evidence needs	An assessor may ask for documented policies, test results, change records and proof that rails were tested after each upgrade, and collecting this evidence is additional work.	Agree the evidence list before build starts. DeployOpen prepares controls and evidence, while independent auditors and certification bodies issue any assessment outcome.

Project phases and what each produces

The sequence moves from mapping the application's risks to a monitored production guardrail layer, with a written output at every phase.

Map the application and threats
We review the LLM application's users, data sources, tools and outputs with your team, then map the threats each entry and exit point faces, such as off-topic use, prompt injection and disclosure of sensitive data. The step closes with a threat map that your security and product owners sign off on.
Select rails and set budgets
For each threat we choose which of the five rail types apply (input, dialog, retrieval, execution and output) and whether each rail needs an extra model call. We then set a written latency and model-call budget per request. The output is a rail decision record listing each rail, its purpose, its cost driver and the owner who accepted it.
Design configuration and Colang flows
We write the config.yml file, the Colang flows that define allowed and blocked dialog, and any Python actions that call your internal systems. Colang version is confirmed at this stage, since the 1.0 default and 2.x differ in syntax. The step is done when the configuration loads in a development environment and every rail in the decision record maps to a flow or action.
Wire models and safety checks
We connect the main LLM and any moderation, fact-checking, hallucination or jailbreak detection models you approve, and record which calls run on each request. Hosting, credentials and data residency for each model are confirmed against your infrastructure. The output is a model wiring document with a measured call count per request path.
Build the test corpus and baseline
We build a test corpus of benign, borderline and adversarial prompts tied to the threat map, then record how the current rails respond. The baseline captures blocks, passes, false positives and latency per request. The step closes with a regression suite that can rerun after every change or upgrade.
Choose deployment and roll out to staging
We choose between the Python library inside your application and a guardrails server or the NVIDIA microservice, based on your runtime, scaling and support needs. The chosen mode is deployed to staging with logging and access controls, and the regression suite is run against it. The step is complete when staging results match the baseline within the tolerances agreed in the scope.
Cut over to production and monitor
We move traffic to production in the agreed sequence, with a rollback path, and set monitoring for blocked requests, latency, model-call volume and error rates. Alert thresholds are agreed with your operations team. The output is a production readiness record showing the cutover checks passed and the monitoring confirmed.
Hand over the runbook
We deliver the runbook covering configuration changes, the upgrade and regression process, incident steps for a blocked or failing rail, and ownership of each model credential. Your team walks through the runbook and runs one rollback drill. The step closes when your named owners accept the handover in writing.

What usually goes wrong and how to prevent it

Most NeMo Guardrails problems appear after go-live rather than during the build. The cards below name the failure modes we see most often in planning and the check that prevents each one.

Latency budget discovered too late

Each enabled rail can add one or more model calls before the application answers, and those calls stack. Measure end-to-end latency with the full rail set on a production-like load before committing to a design, and set a per-rail latency budget that the team signs off on.

Safety model call cost unmeasured

Self-check moderation, fact-checking, hallucination detection and the NVIDIA content and topic safety models each add calls with their own cost and latency. Count the extra calls per request, multiply by expected traffic, and confirm the cost with the model provider or the hosting team before enabling them.

Action paths left without execution rails

Teams often protect only user input and model output, then let tools that change records, send messages or move money run unchecked. Map every custom action and tool, and add execution rails and dialog rails that validate parameters and confirm high-impact steps.

Test corpus too small to tune blocking

A handful of hand-written prompts will show rails working in a demo and then block legitimate requests or miss attacks in production. Build a test set from real, anonymized traffic and adversarial probes, track false positives and false negatives separately, and retune thresholds against that set.

Colang 1.0 copied into 2.x unreviewed

The nemoguardrails convert tool produces a starting point, and NVIDIA's migration guide says converted configurations still need manual review. Read every converted flow against its original behavior, rerun the test set, and treat the migrated configuration as new code.

LangChain path left at default

LangChain integration is opt-in through the NEMOGUARDRAILS_LLM_FRAMEWORK=langchain environment variable. Set this variable deliberately in each environment and record the choice in the deployment configuration so that a container or host image cannot change behavior silently.

Version pinned without reading release notes

Pinning a version protects against surprise upgrades, but the pin only helps if someone has read what changed in that release. Check the GitHub release notes for the pinned version, rerun the regression set, and record the decision before the pin is adopted or moved.

No owner for blocked-request review

Blocked requests pile up in logs without anyone deciding whether the block was correct, so false positives stay in place and users route around the assistant. Name an owner, set a review rhythm, and define how confirmed errors feed back into rail configuration and tests.

Recurring costs after go-live

This table lists what keeps costing money or effort once NeMo Guardrails is live, who typically pays for each item, what drives it, and what to ask for in a quote.

Component	Who pays	What drives it	What to ask for in a quote
LLM and safety-model inference per request	Your organization, through a model provider invoice or your own accelerator capacity	Number of rails that call a model on each request, such as self-check moderation, fact-checking or hallucination detection, plus the main application model call	Per-request call count by rail type, which models are used, whether they are self-hosted or third party, and how the estimate changes at peak traffic
Compute for hosting safety models	Your organization, through your own infrastructure or cloud accounts	Which NVIDIA content safety, topic safety or jailbreak detection models you run locally, their memory and GPU needs, and replica count for availability	Sizing assumptions for each model, the hardware or instance types proposed, and whether capacity is priced separately from the guardrails server
Server or Kubernetes hosting for the guardrails server	Your organization, through cloud or data center bills and your platform team	Whether the library runs inside an existing service or as a standalone guardrails server, the number of configurations loaded, and redundancy requirements	Hosting assumptions, environments covered (development, test, production), logging and storage included, and who maintains the base images
Microservice option: Kubernetes and Helm operating effort	Your platform team, plus NVIDIA licensing if you choose the microservice (terms to be confirmed with NVIDIA)	Chart upgrades, cluster changes, secrets rotation, image updates and on-call coverage for a production container image	Confirmed licensing terms and support tier from NVIDIA, the Helm chart version you will track, and the operating tasks the partner will take on versus your team
Test corpus maintenance as policies change	Your compliance or product owners for policy decisions, and the engineering team or DeployOpen for test updates	Changes to blocked topics, new jailbreak patterns, Colang flows and the number of test cases that must pass before each release	How the regression corpus is versioned, who approves new cases, expected test runtime per change, and whether maintenance is inside a retainer
Upgrades across releases	Your engineering team, or a partner under a support retainer	Behavior changes between 0.x releases, Colang 1.0 or 2.x configuration changes, Python version support (3.10 to 3.13) and dependency updates	Upgrade scope per release, regression testing before each upgrade, rollback approach and whether the work is fixed-scope or retainer-based
Monitoring and review of blocked requests	Your security or operations team, with tooling costs for logs and dashboards	Volume of blocked and flagged requests, retention period, review workflow and the need to tune rails when false positives appear	Logging format and storage assumptions, review cadence, alert thresholds, and who owns tuning decisions after go-live
Internal skills	Your organization, through staff time or training	Colang and Python action maintenance, Kubernetes skills for the microservice, and the number of people who can review rail behavior	Named roles that must be covered, a handover plan from the implementation, and training or documentation delivered as part of the work

How the work changes by starting point

A new application has the most room to choose rails before any configuration exists. The work starts with a list of behaviors the product must block or steer, then maps each one to an input, dialog, retrieval, execution or output rail.

Scope: define rails for each user journey, and record which extra model calls each rail adds to every request.
Effort drivers: the length of the policy list, the number of custom actions, and how many rails call a model or safety model.
Readiness: confirm the Python runtime against the supported versions (3.10 to 3.13) and choose Colang 2.0 or 1.0 on purpose.
Decision: whether the rails run as a library inside the application or as a separate guardrails server.

What you need to provide before scoping

A NeMo Guardrails scope is fixed only after the client confirms the inputs below. Each one changes which rails are needed, how many model calls run per request, and how much testing is required. Start with a written description of the LLM application and its users, including whether users are staff or the public, which languages they write in, and which topics the application must refuse or redirect. Rail design follows directly from this, and an application serving customers needs different input and output rails than an internal assistant.

Next, supply the model providers and safety models your organization has approved, along with the approved endpoints for self-check moderation, fact-checking and hallucination detection. Each rail that calls a model adds latency and cost per request, so the approved list determines which rails are practical. Provide sample prompts and responses, including known abuse cases such as jailbreak attempts, prompt injection and data extraction attempts drawn from your incident history. These samples become the first test cases and show whether the planned rails catch the behavior you care about.

Confirm your data classification rules and logging rules. Guardrail logs can contain user prompts and model outputs, so the classification of that content decides what may be stored, for how long, and where. Provide your latency targets as measured end to end, since they constrain how many checks can run in sequence. State the hosting environment, the Python version (NeMo Guardrails supports Python 3.10 to 3.13) and the container platform if one is used. Give DeployOpen access to the repository and a test environment that mirrors production closely enough to measure latency.

Finally, name the owners. Policy decisions about what the application may say need a named business owner who can approve changes to rails. An operations owner must review blocked requests on a set schedule and decide whether a block was correct. Ask for these names in writing, because unclear ownership is one of the most common reasons rail behavior drifts after launch and the quote must reflect who signs off.

Frequently asked questions

What does a NeMo Guardrails implementation cost, and what drives the price?

The cost is set by the written scope, and DeployOpen does not publish a price list. The main drivers are the number of rails in use (input, dialog, retrieval, execution and output), the number of policies and Colang flows to design, the number of environments, and how large the test corpus must be. Ask for the quote to split one-time work (design, configuration, custom actions, testing, handover) from recurring work (upgrades, regression runs and support). Also ask whether the quote includes the model and safety-model calls at runtime, since those are billed by your own providers or infrastructure, not by the engagement.

How long does a NeMo Guardrails rollout take?

Duration depends on the same inputs that drive cost: how many rails you apply, how many policies need decisions, and how many environments (for example development, staging and production) must be configured. Most engagements are fixed-scope, so the timeline is agreed in writing before work starts and tied to named deliverables. Policy decisions made by your business and risk owners often set the pace more than engineering does, so confirm early who signs off on each policy and how quickly those reviews can happen.

Can our own team implement NeMo Guardrails without a partner?

Yes, if your team already has solid Python skills and experience building LLM applications, the framework itself is approachable. The hard parts are usually not the installation or the config.yml file. They are building a test corpus of benign and adversarial prompts that reflects your users, and making the policy decisions about what to block, route or allow. Teams that lack those two things, or lack time to test rails before each release, tend to benefit from a partner for those specific tasks.

Should we use the NeMo Guardrails library or the microservice?

The decision depends on where you host, who operates it and what support you need. The Apache 2.0 library runs inside your application and adds no separate service to operate, but your team owns scaling, observability and upgrades. The NeMo Guardrails microservice is a production container image built on the same configuration model and deployed on Kubernetes with Helm, which suits shared guardrails for several applications. Confirm the hosting target, your on-call model and whether you need vendor support before choosing.

Do safety models add cost to each request?

Yes. Self-check moderation, fact-checking, hallucination detection, content safety and topic safety models, and jailbreak or injection detection each can add model calls for every request they inspect. Those calls add latency and consume compute or API budget, so the cost scales with traffic and with the number of rails that call a model. Ask for the plan to list which rails call which model, how many calls one user request triggers at peak, and where those models run.

Do Colang 1.0 configurations need to be rewritten?

Colang 1.0 is the default, and Colang 2.0 is also supported, so a rewrite is required only if you move to 2.x. NVIDIA provides the nemoguardrails convert tool for the migration, but it states that converted configurations still need manual review. The full migration is covered in a separate topic, but the implementation plan should budget for that review and for regression testing of converted flows. If you already run on Colang 1.0 and plan to stay there, the plan should say so and note the upgrade risk.

What must we provide, and what do we receive at handover?

You provide the target infrastructure and access, the LLM and safety-model endpoints and credentials, the policy owners who decide what each rail should do, sample prompts and conversation logs for the test corpus, and a named engineer for reviews. At handover you receive the configuration files (config.yml, Colang files and any Python actions), deployment artifacts for the library or server image, the test corpus and results, and runbook documentation for operations. Confirm the exact list of deliverables in the written scope.

How DeployOpen scopes, delivers and hands over NeMo Guardrails work

DeployOpen plans and implements NeMo Guardrails inside infrastructure you control, starting from a written scope that names the rails, model calls and integrations involved. The work is delivered as configuration, tested rails and documentation your team can run after handover.

How we scope

The written scope lists which of the five rail types apply, such as input, dialog, retrieval, execution and output rails.
We map every model and safety-model call each rail makes, since these drive latency and per-request cost.
The scope states whether you run the open-source library or the NVIDIA microservice, with the reasons recorded.

How we build

We write config.yml, Colang files and Python actions against the Colang version you choose, with the default being 1.0.
Each rail is tested against representative prompts, blocked-response cases and edge cases before it is accepted.
Deployment follows your chosen path, either the Python API inside your application or the guardrails server in a container you run.

How we hand over

You receive configuration in version control, the build and deployment scripts, and the test suite used for acceptance.
Documentation covers each rail's purpose, its model dependencies, known limits and the settings your team may change.
Handover includes a walkthrough with your engineers so they can run tests and change configuration without us.

How we support

Fixed-scope engagements are agreed in writing before work starts, with deliverables and timelines stated in the scope.
Ongoing support, upgrades and continuous testing are available as a retainer, which suits teams that change rails often.
Before any NeMo Guardrails version upgrade, we can rerun the regression tests against your rails and report what changed.