Which NeMo Guardrails deployment fits your team
The library API server fits teams that need direct integration, a proof of concept, or a way to develop and evaluate guardrail configurations. The NeMo Guardrails microservice is the production-grade, high availability option, and it is supported under the NVIDIA AI Enterprise plan. Choose the library when your team owns the application, can tolerate limited production reliability, and wants to iterate on Colang and config.yml before committing to a platform. Choose the microservice when uptime requirements, vendor support and a managed configuration path matter more than keeping the stack small.
The library API server manages guardrails through the config folder structure, so changes are made to files in version control and the server loads them from a folder. Its Docker deployment serves OpenAI-compatible providers, including self-hosted vLLM, TGI and llama.cpp endpoints, which suits teams running models inside their own network. NVIDIA's comparison table notes that it has no high availability out of the box, so if you adopt it for production, plan your own redundancy, health checks and failover, and confirm with NVIDIA which support channels cover it. The microservice instead manages configurations through platform API endpoints and is deployed on Kubernetes with Helm.
The microservice is not a standalone solution. It is built on the library and requires the NeMo microservices platform, which means other NeMo microservices must run alongside it. Before you choose it, ask NVIDIA which components, cluster resources and NVIDIA AI Enterprise entitlements your environment would need, and who in your organization will operate the platform dependency. Teams that want to avoid that dependency can keep the library and build their own production controls around it.
How the library and microservice compare on the factors that matter
This table sets NVIDIA's own comparison of the NeMo Guardrails library API server against the NeMo Guardrails microservice across configuration, reliability, support and platform dependency. Read each row as a question about your deployment: where the docs state a point, the cell says so, and where they do not, the cell tells you what to confirm with NVIDIA before you commit.
| Factor | Library API server | NeMo Guardrails microservice |
|---|---|---|
| Configuration management | Guardrails are managed through the config folder structure, loaded by the server from a folder. | Configurations are managed through platform API endpoints in the NeMo microservices platform. |
| Production reliability | NVIDIA's table lists limited production reliability for the library API server. | NVIDIA positions the microservice as the production-grade option. Confirm the specific reliability targets with NVIDIA. |
| High availability | No high availability out of the box, according to NVIDIA's table. You must design redundancy yourself. | Described as the high availability option. Ask NVIDIA how replicas, failover and upgrades are configured. |
| Support model | Supported by the open-source community. Support levels are not covered by a vendor agreement. | Supported under the NVIDIA AI Enterprise plan. Ask NVIDIA which tiers, response terms and version windows apply. |
| Platform dependency | Runs on its own, as a Docker container or through the API server, without the wider NeMo platform. | Requires the NeMo microservices platform and other NeMo microservices. NVIDIA says it is not a standalone solution. |
| Build basis | The base library itself, which you can call from Python or serve over HTTP. | Built on top of the library, using the same configuration model and deployed on Kubernetes with Helm. |
| Recommended use | NVIDIA recommends it for direct integration, proofs of concept, development and evaluating configurations. | Suited to production deployments that already run, or plan to run, the NeMo platform under NVIDIA AI Enterprise. |
When the library API server is enough and when it is not
NVIDIA recommends the library API server for direct integration, so an existing service can call guardrails over HTTP or import the Python API without adopting the rest of the NeMo platform. The configuration sits in a config folder that your team versions alongside the application, which keeps policy changes reviewable in the same repository.
- Configuration ownership: Rails, Colang files and optional Python actions live in the config folder, so your team owns their review and release process.
- Model endpoints: The Docker image supports OpenAI-compatible providers, including self-hosted vLLM, TGI and llama.cpp endpoints, so confirm which model your rails call and where it runs.
- Availability: The library API server has no high availability out of the box, so the service that calls it needs its own failover plan.
- Monitoring: Build request logging and alerting around the guardrails server, because the library does not supply an operations dashboard.
- Upgrades: Pin the release you tested, such as the Docker image version you validated, and test configuration changes against that version before rollout.
Confirm with NVIDIA's deployment documentation how your chosen image handles restarts and scaling before you place it in the request path.
For proofs of concept, development and evaluating configurations, NVIDIA points to the library API server as the right fit. A team can write rails in Colang, run them against sample prompts and iterate quickly without provisioning a Kubernetes cluster or a platform license.
- Fast iteration: Edit the config folder and restart the server to test dialog, input and output rails against realistic attack inputs.
- Colang version choice: Colang 1.0 is the default, while Colang 2.0 is also supported, so settle the version before writing policies you intend to keep.
- Migration review: If you convert between Colang versions, the nemoguardrails convert tool helps, but NVIDIA says converted configurations still need manual review.
- Evaluation scope: Measure whether rails block what you need and whether they interfere with legitimate requests, since the library includes self-check moderation, fact-checking and jailbreak detection that each need tuning.
- Handoff plan: Record which configuration and image version produced your results so the production team can reproduce them.
Before a proof of concept becomes a service, confirm which rails depend on external models or services in your environment.
For production workloads with uptime requirements, NVIDIA positions the NeMo Guardrails microservice as the production-grade, high availability option, available with the NVIDIA AI Enterprise plan. If you choose the library API server here, your team must build the reliability layer that the microservice provides, and it must be documented as your own architecture.
- High availability: Design redundant instances, health checks and failover, since the library has no high availability out of the box.
- Configuration management: Use your own change control, testing and rollback for the config folder, because the microservice instead manages configurations through platform API endpoints.
- Observability: Define metrics, logs and alerts for rail decisions, latency and model errors, and route them to the team that owns incident response.
- Support path: The library is supported by the open-source community, so confirm who responds to a production incident and under what terms.
- Security review: Include the guardrails service in your threat model and access controls, since it sits in the request path for model traffic.
Ask NVIDIA and your own architecture reviewers to confirm the availability, support and upgrade model you will operate before committing to the library.
What drives the cost of each deployment
The library and the microservice put their costs in different places. One carries a licensing and platform commitment, the other carries the engineering work to run, migrate and test what you deploy. Read each card as a line item to confirm in a written quote.
NVIDIA AI Enterprise subscription
The microservice is supported under the NVIDIA AI Enterprise plan, so this is a recurring commercial cost paid to NVIDIA that the library does not carry. The library is Apache-2.0 licensed and supported by the open-source community, so it has no equivalent license line. Ask NVIDIA or your reseller which NVIDIA AI Enterprise edition, term and support tier cover the microservice, whether that subscription is per GPU or per node, and what renewal terms apply.
NeMo microservices platform dependency
The microservice needs the NeMo microservices platform and other NeMo microservices, so it is not a standalone product. Your cost includes running those dependencies, their upgrade cycles and their support coverage, all of which recur. Ask whether the platform components are included in your subscription or priced separately, and which of them you must run in your own cluster.
Kubernetes and Helm operating skills
The microservice is deployed on Kubernetes with Helm, so your platform team pays in staff time to build, upgrade and monitor that stack. The library can run as a Docker container, which lowers the skill bar for a single service. Estimate who on your team owns cluster upgrades, secrets and scaling, and whether those skills already exist or must be hired or trained.
Self-hosted model endpoints
Both deployments call a model for self-check moderation, fact-checking and similar rails, so the hosting cost of that model is separate from NVIDIA's license. The Docker image's default framework serves OpenAI-compatible providers, including self-hosted vLLM, TGI and llama.cpp endpoints. GPU or CPU capacity for those endpoints recurs every month. Confirm which rails in your configuration call a model and how much capacity your traffic needs.
Colang 1.0 to 2.x migration effort
Colang 1.0 is the default, and NVIDIA's migration guide describes a conversion tool, but converted configurations still need manual review. That review is a one-time engineering cost whose size depends on how many flows, actions and rails you have. Teams starting fresh on Colang 2.0 avoid this step, though they still pay for authoring. Ask any partner to scope the review as a fixed line item with named deliverables.
Testing and regression effort
Every change to a rail, action or model endpoint can shift what gets blocked, so testing is a cost that recurs with each release. Plan for evaluation datasets, regression runs and sign-off by security or compliance owners. This effort is roughly the same for both deployments, but the microservice adds platform upgrades to the regression cycle. Ask for a written test plan and who maintains it after handover.
How each deployment is built and what runs where
The library, its server and the production microservice share one configuration model, but they run in very different places. The cards below separate what you install, what you build, and what the production option expects around it, so you can map each piece to your own infrastructure before comparing support and availability.
The library Python package
The nemoguardrails package is installed from PyPI and called directly from your application through its Python API. Guardrails run in the same process as your LLM calls, so the application team owns scaling, restarts and logging. Supported Python versions are 3.10 through 3.13, so check that your runtime image matches before you standardize on it. Confirm which framework your build uses before you commit to a Python version.
The server built from the Dockerfile
The library can also run as a guardrails server that loads configurations from a folder and exposes an HTTP API. The repository provides a Dockerfile to build a server image, so you build and host that image yourselves. NVIDIA positions this path for direct integration, proofs of concept and evaluating configurations. Your team owns the config folder, its version control and the server's availability.
The Docker image and its default framework
The published Docker image uses a lightweight httpx-based default framework rather than the LangChain path, which is opt-in through the NEMOGUARDRAILS_LLM_FRAMEWORK environment variable. It serves OpenAI-compatible providers, including self-hosted vLLM, TGI and llama.cpp endpoints. Check that your model endpoints expose the OpenAI-compatible interface and that your application does not depend on LangChain-only features.
The config.yml and Colang files
Each guardrails configuration is a config.yml file, optional Python actions and Colang files that define input, dialog, retrieval, execution and output rails. Colang 1.0 is the default and Colang 2.0 is also supported. Moving between them uses the nemoguardrails convert tool, and NVIDIA says converted configurations still need manual review. Budget review time for every migrated rail.
The production microservice container image
The NeMo Guardrails microservice is a production container image built on the same configuration model and deployed on Kubernetes with Helm. Its configurations are managed through platform API endpoints rather than a local config folder. NVIDIA describes it as the production-grade, high availability option available with the NVIDIA AI Enterprise plan. Ask NVIDIA which Helm chart version, Kubernetes baseline and storage components your environment would need.
The NeMo microservices platform dependency
NVIDIA states the microservice is not a standalone solution because it needs other NeMo microservices. The microservice is built on top of the library, so it inherits the library's rail behavior, but it cannot run on its own in the way the library image does. Before you choose it, map which other NeMo services your platform team would have to host, patch and support, and confirm that list with NVIDIA in writing.
Upgrades, monitoring and skills each option demands
This table compares what each NeMo Guardrails option asks of your team after go-live: upgrades, availability, monitoring, configuration backup and skills. Read the rows as questions to take to NVIDIA or your NVIDIA AI Enterprise contact, since the sources cover architecture and platform dependency but not every operating procedure. The main takeaway is that the library shifts availability and operations work to your team, while the microservice moves that work onto a Kubernetes and Helm platform you must already run.
| Area | Library API server | NeMo Guardrails microservice |
|---|---|---|
| Upgrade path | You pull new library and image versions yourself, such as v0.24.1 released September 16, 2026, and retest your Colang and config.yml policies before rollout. | Upgrades arrive through the NeMo microservices platform and its Helm deployment. Confirm the upgrade cadence, rollback method and support windows with NVIDIA. |
| Colang version changes | Colang 1.0 is the default. Moving to 2.x uses the nemoguardrails convert tool, and NVIDIA says converted configurations still need manual review. | Configurations use the same model as the library, but confirm how Colang version changes are handled on the platform before you plan a migration. |
| Availability design | NVIDIA lists limited production reliability and no high availability out of the box. Redundancy, failover and load balancing are your design work. | NVIDIA describes the microservice as the production-grade, high availability option. Confirm the exact redundancy model and failure behavior for your deployment. |
| Monitoring responsibility | You own health checks, logs, latency tracking and alerting for the server and the Docker container, using your own tooling. | Monitoring depends on the platform's observability stack. Confirm which metrics and logs are exposed and whether you must integrate them with your SIEM. |
| Configuration backup | Your config folder, Python actions and Colang files live in your source control. Backup is your responsibility, and version control is the main safeguard. | Configurations are managed through platform API endpoints. Ask NVIDIA how they are stored, exported and restored, and whether backup procedures are documented. |
| Platform dependency | Runs as a Docker container or API server without the wider NeMo platform, which suits direct integration and proofs of concept. | Requires the NeMo microservices platform, and NVIDIA says it is not a standalone solution because it needs other NeMo microservices. Plan for that dependency in your run cost. |
| Infrastructure skills | Container and API server operations, Python packaging, and Colang policy authoring. Teams with a small platform footprint can usually operate it. | Kubernetes, Helm and the NeMo platform's component model, in addition to Colang policy work. Confirm your team has or can hire these skills. |
| Support terms | Community supported, per NVIDIA's comparison table. Plan internal ownership of fixes and incident response. | Supported under the NVIDIA AI Enterprise plan. Ask NVIDIA for the support scope, response commitments and SLA terms in writing, since the sources do not state them. |
How to decide between the library and the microservice
This sequence turns the library-versus-microservice question into a short list of checks, each with an owner and a finished output, so your team can decide with its own requirements in hand rather than a generic recommendation.
Confirm whether production uptime requirements exist
Start with the service owner and the business owner of the LLM application. Establish whether the guardrailed endpoint must meet a defined availability target, whether an outage would stop a customer-facing or regulated process, and whether anyone has written a recovery expectation. NVIDIA's comparison table describes the library API server as having limited production reliability and no high availability out of the box, so a written requirement for high availability points toward the microservice. The step is done when the requirement is written down with an owner's name on it, or when the owner confirms that the workload is a proof of concept or internal tool where that requirement does not apply.
Check NVIDIA AI Enterprise entitlement
Ask procurement and your NVIDIA account contact whether your organization holds an NVIDIA AI Enterprise plan, or whether one is in scope for this budget cycle. NVIDIA positions the microservice as the production-grade option available with that plan, and the comparison table lists its support under the same plan. Ask for the entitlement terms in writing, including what support covers and who signs. The step is done when you have a yes, a no, or a dated quote, and the answer is recorded alongside the uptime requirement from the previous step. If no entitlement exists, the library path remains the documented option for your team to run and support itself or through a partner.
Assess Kubernetes and Helm skills
Ask the platform engineering lead who will operate the service day to day. NVIDIA documents the microservice as a production container image deployed on Kubernetes with Helm, so the team needs to own cluster upgrades, secrets, ingress, scaling and rollback for that workload. The library can run as a Docker container or through its API server, which requires less cluster knowledge. Name the operator, confirm they have run Helm releases in a comparable environment, and check whether your cluster policies allow the chart and images your security team will need to review. The step is done when the operator and cluster owner have agreed in writing who gets paged for a failed rollout.
Confirm the need for other NeMo microservices
Ask the architecture owner whether your guardrail design depends on other components of the NeMo microservices platform. NVIDIA's table states that the microservice requires the NeMo microservices platform, and NVIDIA notes it is not a standalone solution because it needs other NeMo microservices. If your design only needs the guardrails layer in front of a model endpoint you already run, the platform dependency is a cost you carry without a matching benefit. If you already plan to use other NeMo services, the dependency may be intended. The step is done when the architecture owner signs a short statement naming the NeMo services in scope, or confirms that none are required.
Pick a model endpoint for the guardrails
Confirm which model endpoint the rails will call for self-check moderation, fact-checking or other model-backed checks, and whether it is self-hosted or external. NVIDIA's Docker deployment guide describes an httpx-based default framework that serves OpenAI-compatible providers, including self-hosted vLLM, TGI and llama.cpp endpoints. Ask the ML platform owner for the endpoint address, the authentication method, and the latency and capacity they can commit to. Also ask the data owner whether prompts may leave your environment. The step is done when the endpoint is named in the configuration plan and the data owner has approved the data flow for both the library and the microservice options.
Plan the Colang version and migration review
Ask the guardrails policy owner which Colang version your existing configurations use. The library supports Colang 1.0 and 2.0, with 1.0 as the default, and NVIDIA documents a migration path using the nemoguardrails convert tool. NVIDIA also says converted configurations still need manual review. Inventory the config.yml files, Python actions and Colang files, then decide who reviews the converted output and which test cases must pass before a change goes live. The step is done when the policy owner has a file inventory, a named reviewer, and a set of regression prompts that the chosen deployment must answer the same way after migration.
Questions buyers ask before choosing a NeMo Guardrails deployment
Can the NeMo Guardrails library run in production?
It can run in production, but NVIDIA's own comparison table describes the library API server as having limited production reliability, with no high availability out of the box. NVIDIA recommends the library for direct integration, proofs of concept, development and evaluating configurations. If you run it as a production service, your team owns redundancy, failover, scaling and monitoring around the server, and that work recurs with every upgrade. The Docker image uses a lightweight httpx-based default framework and serves OpenAI-compatible providers, including self-hosted vLLM, TGI and llama.cpp endpoints, so the runtime can stay inside your network. Confirm the availability targets your service needs, then test failover under your own load before you commit. The NVIDIA deployment options page is the best starting point for that comparison.
What does NVIDIA AI Enterprise add to the NeMo Guardrails microservice?
It adds vendor-backed support. Per NVIDIA's comparison table, the microservice is supported under the NVIDIA AI Enterprise plan, while the library API server is supported by the open-source community. NVIDIA also describes the microservice as the production-grade, high availability option. The research behind this page does not include the plan's price or its support terms, so ask NVIDIA for the current terms, what support covers in a Kubernetes and Helm deployment, and whether it extends to the other NeMo microservices your configuration depends on. The library is Apache 2.0 licensed, so the code carries no license fee of its own. Your running costs come from infrastructure, the people who operate the service, and any support you buy.
Does the NeMo Guardrails microservice work without the NeMo platform?
No. Per NVIDIA's comparison table, the microservice requires the NeMo microservices platform and is built on top of the library. NVIDIA states that the microservice is not a standalone solution because it depends on other NeMo microservices. In practice, you adopt a platform rather than a single container, so the platform's dependencies, upgrade cadence and cluster requirements become part of your architecture. The microservice is deployed on Kubernetes with Helm, which means your cluster team must own that environment. If you only need a guardrails server, the library API server is the option NVIDIA describes for that use. Ask NVIDIA which NeMo services your use case would pull in and what the full platform footprint is before you plan capacity.
How much effort does a Colang 1.0 to 2.x migration take?
The effort depends on how much Colang your configuration contains, and no standard duration applies. Colang 1.0 is the default, and Colang 2.0 is also supported. NVIDIA documents a migration path using the nemoguardrails convert tool, and it states that converted configurations still need manual review. Plan for that review, not only for the conversion. The main effort drivers are the number of flows and rails, the custom Python actions they call, how many behaviors depend on Colang 1.0 semantics, and how complete your test coverage is for each rail. Teams with thorough test suites can confirm parity faster. Ask any implementation partner for a written scope that lists each flow to review, the test cases used to prove the behavior matches, and who signs off on any change in behavior. Read the NVIDIA migration guide before you size the work.
Can our own team run the NeMo Guardrails library without outside help?
Yes, if your team has Python and container operations skills, because the library is designed to run directly. The package supports Python 3.10, 3.11, 3.12 and 3.13. Configurations live in a folder with a config.yml file, optional Python actions and Colang files. Your team would own the high availability design that NVIDIA's table says the library lacks, along with logging, secrets handling, upgrades to new releases such as v0.24.1, regression testing of rails after each upgrade, and incident response. Those tasks recur for as long as the service runs. Before you decide, check whether the people who would hold that work have time for it on a steady schedule. Outside help can take on part of that load, and the right split depends on your staffing, change process and how often your policies change.
What does outside help cover for a NeMo Guardrails deployment?
DeployOpen plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own infrastructure, including your cloud accounts, data centers or air-gapped networks. For this topic, that can include deploying the library or the microservice in your environment, connecting the guardrails to your model endpoints, hardening the server, and writing the runbooks your operators need. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. Ongoing support, upgrades and continuous testing are available as retainers. DeployOpen does not publish a price list, so a quote follows from the written scope. Ask that the scope name the platform option, the high availability design, the migration review, the test cases and who maintains the rails after handover. DeployOpen prepares systems and evidence for compliance assessments, but independent auditors and certification bodies issue any audit reports or certifications.
What drives the total cost of each NeMo Guardrails option?
Cost splits into one-time work and recurring work, and each option shifts the balance. The library's Apache 2.0 license covers the code, so its costs are the infrastructure to host it, the engineering to build high availability, the migration and testing effort, and the staff who maintain policies. The microservice adds the NVIDIA AI Enterprise plan, the Kubernetes and platform footprint it needs, and the platform upkeep. One-time costs include setup, the Colang migration review and the first test suite. Recurring costs include upgrades, rail maintenance, infrastructure, plan fees and any retainer. Costs rise with the number of rails, custom actions, model endpoints and environments you run. When you request a quote, ask for each component to be priced separately, including who performs the migration review, who runs upgrades and whether the plan terms are included or billed by NVIDIA.
Official NeMo Guardrails deployment sources to read next
Start with NVIDIA's NeMo Guardrails deployment options page, because it sets the boundary between the library, which can run as a Docker container or through its API server, and the NeMo Guardrails microservice, which NVIDIA describes as the production-grade, high availability option available with the NVIDIA AI Enterprise plan. Read it with a specific question in mind: which of your workloads need a config folder you version yourself, and which need configuration managed through platform API endpoints. Then read the microservice production page, which covers the Kubernetes and Helm deployment and the dependency on other NeMo microservices.
Next, read the Docker deployment guide if you plan to run the library server in a container. It documents the lightweight httpx-based default framework and how OpenAI-compatible providers such as self-hosted vLLM, TGI and llama.cpp endpoints are wired in. Test those provider settings against your own model endpoints before you rely on them, since the guide describes the default and your gateway, TLS and authentication setup will differ.
Use the NeMo Guardrails documentation for the current configuration model, and the GitHub releases page to track the library version you intend to pin. The latest researched release on that page is v0.24.1, published on September 16, 2026, so check whether a newer release has appeared before you finalize a version. Confirm the current NVIDIA AI Enterprise terms, including what support covers and what it costs, directly with NVIDIA or your NVIDIA account contact, because licensing terms change and this page does not state them.
How DeployOpen works on this
DeployOpen plans, implements, customizes, integrates, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, and NeMo Guardrails work follows the same path. We start by deciding which deployment shape fits your environment, then agree the work in writing before any build begins.
How we scope
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
- The written scope names the deployment shape, either the library API server or the microservice, and the reasons for that choice.
- Scope lists the rail types to configure, such as input, dialog, retrieval, execution and output rails, along with the model endpoints they call.
How we build
- We build the Colang and config.yml policy set, plus any Python actions, inside your own cloud accounts, data centers or air-gapped networks.
- Configurations are tested against your attack cases, including injection and jailbreak prompts, before they reach a shared environment.
- Where you run Colang 1.0 configurations on Colang 2.x, we include the migration and the manual review that NVIDIA says converted files still need.
How we hand over
- Handover includes configuration documentation, deployment steps and the test results that show which policies were checked.
- We document the Docker or Kubernetes deployment so your own staff can rebuild and redeploy the service.
- Your team receives a walkthrough of how to update policies and confirm the version of NeMo Guardrails running in each environment.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, separate from the initial fixed-scope work.
- Upgrade work tracks new NeMo Guardrails releases and re-runs the policy tests after each change.
- Ask for the support terms, response expectations and upgrade scope in writing, since these are set per engagement and are not published.
