Open-source alternative to Portkey Guardrails

Open-source alternative to Portkey Guardrails: the short answer

If your guardrails are tied to Portkey plan tiers, or your partner checks send prompts and responses to outside providers, the maintained open-source options are NeMo Guardrails and Guardrails AI, and both can run inside your own infrastructure beside a self-hosted gateway. LLM Guard is archived and should only be used to support existing deployments, not for new rollouts. Portkey's Basic, Partner and Pro guardrail tiers map to different check sets by plan, and Portkey's open-source gateway does not carry that feature set with it: the MIT-licensed gateway routes and retries requests, but the guardrail checks and their hosted logging are where the plan differences sit.

Replacing guardrails is therefore a separate decision from replacing the gateway. Start by inventorying which checks your team actually enables today, including regex and JSON schema rules, prompt-injection and gibberish checks, and any partner policies that pass content to a provider such as Aporia, SydeLabs or Pillar Security. Each check then needs a home: a deterministic rule can run in your gateway path, while model-based scanning needs compute you control. Confirm with your security and data protection teams whether any current check sends content outside your network, because that determines whether the replacement must run fully inside your boundary.

Plan for the operating work as well as the software choice. NeMo Guardrails uses a config.yml file and Colang files, and its Colang 1.0 to 2.x migration needs manual review according to NVIDIA's guide. Guardrails AI builds guards from validators installed from Guardrails Hub and can run as a standalone service. Ask each vendor or project to confirm current versions, supported Python releases and licensing for the components you intend to deploy, and test each rule against your own prompts before cutting over traffic.

Which open-source option covers each Portkey guardrail check

This table maps each Portkey guardrail check and action to NeMo Guardrails, Guardrails AI and LLM Guard, and notes where custom code or a model download is likely. Treat each cell as a starting point and confirm validator names in each project's current docs before you commit. The main takeaway is that the gateway handles deny, fallback and retry, while the guardrail library supplies the check logic, and LLM Guard is archived.

Portkey check or action	NeMo Guardrails	Guardrails AI	LLM Guard
Regex match	Possible through custom Python actions or Colang flows; confirm current built-in options	Regex validator from Guardrails Hub, installed separately	Confirm whether a regex or banned-pattern scanner is in your pinned version, else write one
JSON schema validation	Output rails and structured output configuration; confirm schema checks in your version	Structured output matched to Pydantic models; validate with a custom or Hub validator	JSON output scanner on the output side
Code detection	Custom action required; confirm whether a built-in check exists	Confirm a Hub validator; otherwise a custom validator	Confirm; not listed among documented scanners
Gibberish detection (LLM-based)	Custom action or self-check prompt; confirm a dedicated rail	Confirm a Hub validator for gibberish or nonsense text	Confirm; not listed among documented scanners
Prompt injection	Jailbreak and injection detection included in the library; models may need download	Hub validators from the community; confirm current injection coverage	Prompt Injection input scanner, but the project is archived and unmaintained
PII redaction	Confirm PII handling through third-party integrations or custom actions	Confirm a Hub PII validator; custom code is likely for redaction	Anonymize on input and Deanonymize on output; no new maintenance
Partner policies (Aporia, SydeLabs, Pillar)	No hosted equivalent; replace with your own rails or a vendor you contract directly	No hosted equivalent; build or install validators yourself	No hosted equivalent; the archived project offers no vendor policy feed
Deny, log, fallback and retry	Rails can block or refuse responses; logging and fallback sit in your gateway or app	Validator failure actions are set in code; fallback and retry sit in your gateway	Scanners return results only; your gateway or app must deny, log and retry
Project status (latest release)	Actively released; v0.24.1 on September 16, 2026	Actively released; v0.11.0 on August 14, 2026	Archived July 9, 2026; last PyPI release 0.3.16 (May 19, 2025)

Which guardrail framework fits which team

Portkey's guardrails are tied to plan tiers, so the replacement is a separate check layer that your team operates beside the gateway. The four options below differ mainly in how much configuration they need, how much maintenance they carry, and whether they run as a library, a server or a container.

NeMo Guardrails

Best fit for teams that want a maintained, policy-style framework. It is Apache-2.0 licensed and configures input, dialog, retrieval, execution and output rails in Colang, with Colang 1.0 as the default and 2.0 also supported. The learning curve is the highest of the four because Colang is its own language, and NVIDIA's migration tool still requires manual review of converted files. It can run as a Python library or as a server image, and NVIDIA offers a Helm-deployed microservice on the same configuration model. Check the Colang version your team standardizes on before writing rails.

Guardrails AI

Best fit for teams that want validators they can chain and extend. It is Apache-2.0 licensed, and its guards are built from validators on Guardrails Hub, such as regex matching or toxicity detection, installed separately and chained together. Its Pydantic-based structured output suits applications that need typed responses. It can run as a Flask REST service started with guardrails start, which lets gateway code call it through the OpenAI SDK. Validator quality varies, so review each hub validator's maintenance status before you rely on it.

LLM Guard

Not recommended for new rollouts. LLM Guard is MIT licensed and offers 15 input and 20 output scanners, but its owner archived the repository on 9 July 2026 and its README says it is no longer maintained. The last PyPI release is 0.3.16, uploaded on 19 May 2025, and it requires Python 3.10 or newer below 3.13. Keep existing deployments running only with a security review of their dependencies, and plan a move to a maintained framework.

Custom validators

Best fit for narrow policies that no framework covers. Your team writes regex, schema or business-rule checks in its own code, and places them in the gateway request path or in a Guardrails AI or NeMo Guardrails action. This avoids a new framework but moves all test, versioning and false-positive tuning work to your team. Budget for a test set of known good and bad prompts, and confirm who owns the rules after the first developer leaves.

Where each check runs beside a self-hosted gateway

Input checks run before the gateway forwards a request to a model, and output checks run on the response before it reaches the application. In a self-hosted setup, the gateway (LiteLLM or the MIT-licensed Portkey gateway) is the only component that calls the model, so the guardrail layer sits either inside the gateway's request hooks or as a separate service the gateway calls over HTTP. Regex and schema checks are deterministic and can run in process with negligible overhead. Scanners that load models, such as prompt injection or PII detectors, add latency and memory and are usually better run as a separate service so they can be scaled and restarted without touching the gateway. The NeMo Guardrails library runs in process from Python, and it can also run as a guardrails server that exposes an HTTP API. Confirm which mode your gateway supports before you design the call path, and measure added latency on your own prompt sizes.

Decide the failure mode for each check before go-live. Fail closed blocks the request when the guard is unreachable or times out, which protects data but makes the guard a dependency for every call. Fail open lets traffic through, which keeps the application available but means an outage becomes an unguarded path. A common pattern is to fail closed for PII and secrets checks and to fail open only for low-risk checks, with the choice written into the runbook. If a check calls an LLM, such as a self-check or a judge model, that call must stay inside your boundary, so point it at a local model served on vLLM or Ollama or at an endpoint you control, and confirm where NeMo Guardrails or Guardrails AI send their model calls in your configuration. Also confirm whether the NeMo Guardrails LangChain integration is in use: it is opt-in and is enabled only when NEMOGUARDRAILS_LLM_FRAMEWORK=langchain is set.

Map each Portkey guardrail outcome to a gateway setting before you cut over. Deny is a guard result that returns an error to the caller, and the gateway should return the same error shape your applications already handle. Log means the guard result is written to your logging backend with the request identifier, so you can audit blocked and flagged traffic. Fallback and retry belong to the gateway's routing configuration (for example, fallback models in LiteLLM), so a guard that fails on one model can route to another, but only if that second model is also inside the boundary and passes the same checks. Write these decisions as a table of check, trigger, action and owner, and test each row with known bad inputs in a staging environment before production traffic moves.

How to move guardrail rules off Portkey without a gap in coverage

This sequence maps every Portkey guardrail to an open-source check, proves that denials and fallbacks still behave the same, and only then removes the Portkey rule and its partner keys. The result is a documented, tested cutover in which no request reaches a model unchecked.

Inventory every guardrail and its action
Export the guardrail attachments for each virtual key, config and workspace in your Portkey account, and record the check name, the input or output hook it runs on, its threshold or pattern, and the action taken on failure. The action matters as much as the check: a deny, a log-only entry, a fallback to another model and a retry each need a different replacement. Confirm the full list with the account owner, because guardrails attached at one level can be inherited by traffic you did not expect. The step is done when every attachment has one row in a shared register.
Classify each check by where it runs
Tag every row as deterministic (regex match, JSON schema validation, code detection), LLM-based (gibberish or prompt injection classifiers) or partner (checks delivered by a third-party provider through its API key). Deterministic checks can run in your own process with no external call. LLM-based and partner checks depend on models or services that you must host, license or replace, so they carry the largest coverage risk. The step is done when each row has a class and a named open-source candidate, with the partner rows flagged for a separate decision.
Reproduce deterministic checks first
Rebuild regex and schema checks as validators or rules in the open-source framework you choose, and keep the patterns identical to the Portkey definitions so behavior can be compared directly. Guardrails AI provides regex and schema validators from Guardrails Hub, and NeMo Guardrails can run equivalent logic in custom actions. Check each rebuilt rule against the same sample inputs on both systems. The step is done when every deterministic row has a matching open-source rule with recorded test results.
Test deny and fallback paths against a replay set
Build a replay set from logged prompts and responses that cover known blocked content, borderline cases and normal traffic, anonymizing personal data before it leaves the source environment. Run each item through the Portkey guardrail and the replacement, and record whether each one denies, passes, falls back or retries. Illustratively, a request that Portkey denies after an output check should produce the same denial and the same fallback model in the replacement. The step is done when disagreements are listed and resolved, not averaged away.
Rewrite Colang or validator configs and review output
Translate the remaining LLM-based and policy rules into NeMo Guardrails Colang configurations or Guardrails AI guards, using the NVIDIA migration guide if you adopt Colang 2.x. NVIDIA states that converted configurations still need manual review, so have a security engineer read every converted flow against the original intent. Confirm which safety models your chosen framework needs and whether they run inside your network. The step is done when each rewritten configuration has a reviewer's sign-off and passes the replay set.
Run both paths in shadow mode
Send live traffic through Portkey as the enforcing path while the open-source checks run in parallel and only log their results. Compare verdicts per check and per application, and investigate every case where the two disagree before any traffic moves. Watch latency and failure rates for the new checks, because they add time on the request path. Confirm with the team that owns each application how much disagreement is acceptable. The step is done when a defined period of agreement has been recorded in your own logs.
Cut over one application at a time
Move enforcement to the open-source guardrails for one application or virtual key, keeping the Portkey rule in logging mode as a fallback for that application. Confirm that denials reach the user and the audit log in the same form as before, and that fallbacks still route to an approved model. Repeat for the next application only after the previous cutover has run cleanly. The step is done when every application enforces through the self-hosted checks and the register shows the new owner of each rule.
Retire partner keys and remove Portkey rules
Once every application runs on the self-hosted checks, delete the Portkey guardrail attachments and revoke the API keys for Aporia, SydeLabs, Pillar Security or any other partner provider you enabled. Confirm with each provider that no residual data processing or billing continues on your account, and keep the closed register as evidence for your change record. Ask your compliance lead which records they need before deletion. The step is done when no Portkey guardrail or partner key remains active and the cutover record is filed.

Keeping Portkey routing while the guardrails move

When the gateway stays on Portkey's hosted service, Portkey runs the checks. Requests and responses pass through Portkey's gateway, where Portkey Guardrails can deny a request, log the result, or fall back to another model or retry. Your applications keep the same endpoint and the same configs, so the change is in the policies, not the calling code.

Who runs the checks: Portkey's hosted gateway evaluates the guardrails attached to each config, subject to the plan tier you hold.
Tier limits: The Developer plan gets basic guardrails only, the Production plan adds Partner and Pro guardrails, and Enterprise adds custom guardrails.
Partner calls: Partner policies from providers such as Aporia, SydeLabs and Pillar Security run as calls to those providers, using API keys you add, so prompt content leaves Portkey for a third party.
Log trail: Guardrail verdicts are written to Portkey's log store, so your audit evidence depends on Portkey's retention and export settings.

Before you decide how much to move, confirm which guardrails sit in which tier, which partner calls receive prompt content, and how long Portkey keeps verdicts.

What buyers ask before replacing Portkey guardrails

What drives the cost and effort of replacing Portkey guardrails?

The main drivers are the number of rules you enforce today, whether any of them need custom validators, how strict your PII policy is, how much latency the request path can absorb, and how much labeled evaluation data you have to test against. A short list of regex and JSON schema checks is a small job. A set of PII redaction rules across several languages, plus prompt-injection screening on every request, adds model selection, benchmarking and tuning work. Ask any vendor or consultant to inventory your current Portkey guardrail configuration first, since that inventory sets most of the scope.

Which parts of a guardrail migration are one-time and which recur?

One-time work includes mapping each Portkey check to an open-source equivalent, writing custom validators, wiring the checks beside the gateway, and building a test set of allowed and blocked prompts and responses. Recurring work includes tuning rules as false positives appear, hosting and upgrading any classifier models, upgrading the guardrail framework when new releases ship, and on-call response when a check fails or slows traffic. Confirm with any provider which of these are covered by a fixed-scope delivery and which fall under a retainer.

Can an internal team replace Portkey guardrails without outside help?

Often yes, if the team already runs Python services and has someone who can own security policy. The work needs Python for the guard code and integration, familiarity with NeMo Guardrails' configuration if you choose it, since Colang and config.yml are its core formats, and a security reviewer who can judge false positives and blocked-content policy. Teams that lack Colang experience or have never tested prompt-injection defenses usually spend more time learning than building. A useful test is whether someone can write and run a regression set of prompts before the cutover.

How long does a guardrail migration take and what sets the duration?

The duration depends on the size of the rule set, the number of custom validators, how many applications route through the gateway, and how much review the security team requires before cutover. Parallel running, where both Portkey and the new checks inspect traffic for a period, adds time but lowers risk. Ask for a timeline tied to written milestones in your scope document rather than a generic estimate, and confirm which milestones depend on your own approvals.

Is LLM Guard still an acceptable open-source choice?

For new work, no. The protectai/llm-guard repository was archived by its owner on 9 July 2026, and its README says the project and its Hugging Face models are no longer actively developed or maintained. Its last PyPI release, 0.3.16, was uploaded on 19 May 2025. Existing deployments can keep running while you plan a move, but they should be treated as a migration task. Maintained options such as NeMo Guardrails, which is Apache-2.0 licensed and was released as v0.24.1 on 16 September 2026, are the usual next step.

What should I ask for in a fixed-scope quote for guardrail work?

Ask for a written scope that lists each Portkey guardrail being replaced, the open-source checks that replace it, the custom validators included, the applications and routes covered, and the test sets used to prove behavior. The quote should also name deliverables such as configuration files, runbooks and evidence of testing, and state what is excluded, such as model hosting, changes to applications or rule tuning after handover. Ask how acceptance is measured and whether ongoing tuning, upgrades and testing are offered as a separate retainer.

Will guardrails change how the gateway behaves for end users?

Yes, usually in small but measurable ways. Input checks can block or rewrite requests, output checks can deny or redact responses, and each added check adds latency to the request path. False positives will show up in support tickets, so the test set and a clear fallback behavior matter. Decide in advance whether a failed check denies, logs, retries or falls back to another model, because Portkey Guardrails can be configured for each of those actions and your replacement should have an equivalent decision.

What changes for security, compliance and application teams

Guardrail policy moves from a vendor console into files your teams version. In Portkey, guardrails are switched on and tuned through the hosted interface, and the tier a team is on decides which checks are available. With the open-source approach, the check list, thresholds, deny or fallback actions and the mapping of checks to routes live in configuration files, such as the config.yml and Colang files used by NeMo Guardrails, and any custom validators live in Python code. Security engineers should treat these as controlled artifacts: they go through pull requests, have named reviewers, and are deployed through the same pipeline as the gateway. The change also means the team now owns the rollout order, so a new check should be shipped in a log-only or shadow mode before it can deny live traffic.

Application teams and security teams share false positives, but the decision sits with the owner of each rule. A blocked prompt or response is usually a product problem for the application team and a risk decision for security. Agree in writing who may add an allowlist entry, who approves a threshold change, and how long an exception stays open. Teams should expect some legitimate traffic to be flagged by regex, schema or model-based checks, and should measure that with a test set drawn from their own prompts rather than relying on any vendor's published figures. DeployOpen can help define that test set and the review workflow as part of a fixed-scope engagement, but the acceptance thresholds should come from your own risk owners.

Audit evidence comes from the configuration history and the denial logs, and both need to be kept. Version control provides the record of which checks were active at a given date, who approved each change and what the diff was. Denial and fallback events should be written to a log store your organization controls, with fields for the check name, the route, the action taken and the request identifier, so an auditor can trace a blocked request back to the rule that stopped it. Confirm retention periods with your compliance function and check whether any prompt content in logs counts as personal data under your policies. DeployOpen prepares these controls and evidence for assessment, but it does not issue audit reports, attestations or certifications. Those come from your independent auditor or certification body.

How DeployOpen scopes and delivers guardrail replacement

DeployOpen plans, implements, customizes, integrates, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure. For Portkey guardrail replacement, that means mapping each existing Portkey guardrail to a check that runs beside your own gateway, and confirming where every input and output is scanned before any traffic moves.

How we scope

Inventory each Portkey guardrail in use, including regex, JSON schema, partner and LLM-based checks, and the action taken on failure
Record which checks run before the model call, after it, or both, and where each runs
Agree deliverables, timelines and exclusions in writing before work starts, as engagements are fixed-scope

How we build

Implement deterministic checks such as regex and schema validation, and confirm which PII or prompt-injection scanners fit your data and maintenance requirements
Wire guard calls into the self-hosted gateway so deny, log, fallback and retry behavior matches the original policy
Test each check against sample traffic, including known false positives, and run security testing on the guarded endpoints

How we hand over

Document each rule, its owner, its runtime location and the data it sees, so reviewers can trace every control
Provide configuration files, runbooks and deployment steps in your repositories and infrastructure
Walk your platform and security teams through the setup and record evidence for your compliance assessment, which auditors then review

How we support

Ongoing support, upgrades and continuous testing are available as retainers
Track upstream releases of each guardrail library and review them before applying updates
Re-test guardrail rules after model, prompt or gateway changes, and confirm the maintenance status of any dependency before adopting it

Open-source alternative to Portkey Guardrails