garak REST generator for custom LLM application testing

What the garak REST generator needs from your LLM application

If your team runs a custom LLM application or a gateway behind an HTTP API and wants garak to probe that system rather than a public model, the REST generator is the component you need. It connects garak to any REST endpoint that returns plaintext or JSON, and you describe that endpoint in a short YAML file. The harder part is not the configuration syntax. It is making sure the request garak sends and the response it reads match what your application actually does in production.

Several features of a typical application can quietly change what garak tests. The request body often needs a specific structure, such as a message array, a user identifier, a model name or a conversation ID, and a mismatch can produce errors that look like model refusals. Session cookies, bearer tokens and custom authentication headers must be present on every call and must be refreshed if they expire during a long run. Streaming responses, where the reply arrives as server-sent events or chunked output, need a parser that reconstructs the full answer before a detector scores it. Confirm with the application owner which of these apply before writing the YAML, and capture one successful request and response with a trusted tool so you know the exact shape.

Guardrails and rate limits deserve equal attention because they change the signal itself. A safety filter in front of the model may block a probe and return a canned message, so garak may record a pass that reflects the filter rather than the model. Conversely, a filter may rewrite or truncate output so a real weakness is hidden. Rate limits and quotas can return HTTP 429 responses or timeouts across the 10 generations garak makes per prompt by default, which can distort failure rates and load production capacity. Ask the team who owns the gateway, what limits apply to the scan identity and whether a non-production environment with the same controls is available.

Teams usually need the REST generator instead of a native generator for a hosted model when the thing under test is not the bare model. Examples include an internal assistant that adds retrieval, tools, system prompts or policy checks, or a gateway that routes between several models. Testing the raw model API in those cases would measure a different system from the one users reach. If you are only evaluating a model provider directly, a native generator for that provider is simpler to set up and maintain. The decision is about what you want the results to describe, and that should be written down before the first scan.

Before starting, agree on three points with the application owner: the endpoint and request schema, the authentication method and who issues and rotates the credential, and the target environment for scans. Keep the YAML file and the endpoint version under change control, because a later change to the application can invalidate earlier results. The garak user guide at docs.garak.ai describes the generator configuration and should be checked against the release you install, since the current version is stated in the project's release notes.

Which garak pieces handle a custom endpoint

garak is assembled from a small set of plugin types, and the REST generator is the piece that connects those types to your own application. Knowing which plugin does what tells you where a failed result came from and where to look when a scan behaves unexpectedly.

Probes send the attack prompts

Probes are the plugins that hold the test prompts for weaknesses such as prompt injection, data leakage, jailbreaks and hallucination. For a custom application, the probe decides what is asked, not how the request reaches your endpoint. Before a scan, review which probe families apply to your use case so you do not spend calls on categories your application never exposes.

Detectors judge each response

Detectors inspect the text returned for each attempt and flag output that matches a failure pattern. Each probe runs with its recommended detectors under the default probewise harness. When a result looks wrong, check whether the detector misread a refusal or a harmless answer as a failure, since your endpoint's phrasing may differ from the models garak was tuned around.

Evaluators summarize pass and fail

Evaluators turn detector output into the per-attempt and per-probe results that appear in the report. By default garak makes 10 generations per prompt and marks a probe as FAIL when any attempt produces undesirable output, with a failure rate attached. Confirm that this threshold suits your risk tolerance before comparing results across endpoints or releases.

The REST generator carries each prompt

The REST generator is the plugin that sends each probe prompt to your endpoint over HTTP and returns the text the endpoint produces. It is configured by a short YAML file that describes the URL, method, headers, request body and how to read the response. Most failed scans trace back to this file, so validate it with one manual request before the first full run.

The harness connects probes to the generator

The probewise harness runs each selected probe through the generator and passes the responses to that probe's detectors. It controls the order and scope of the run, so it is the component to inspect when the wrong probes execute or when a probe reports no attempts. Record the harness settings with each run so later scans remain comparable.

JSONL report and garak.log

Each run writes a JSONL report with every attempt, its evaluation and the response your endpoint returned, and garak.log keeps debugging detail that continues across runs. Review the report to confirm the prompts reached the intended application, then use the log to diagnose timeouts, parsing errors and authentication failures. Both files can contain sensitive or harmful text, so store them accordingly.

Choosing between the REST generator and a native generator

This table compares the REST generator with native generators for hosted model APIs, LiteLLM routes and direct model server calls. Read each row as a trade-off: the REST generator tests the application path you ship, while native generators test a model or gateway interface with fewer moving parts. The main takeaway is to start at the layer that matches the question your scan must answer, and to confirm what each option covers in your environment before you run it.

Decision factor	REST generator (application or gateway)	Native generator (model API, LiteLLM or model server)
When it fits	Your question is whether the deployed application, its system prompt, retrieval layer and guardrails resist probes	Your question is whether a specific model or route is vulnerable, independent of application logic
Stack in scope	Application logic, prompt templates, tools, filters and authentication as the client sees them	Model behavior and any gateway transforms for the native route, but not the application code around it
Setup dependence	Depends on the application's API contract, request schema and response shape, all described in YAML	Depends on the provider or server interface, which garak supports natively with fewer configuration fields
Effort to maintain	Must be updated when the application API changes, including new fields, auth flows or streaming behavior	Changes mainly when the provider or model name changes, so upkeep is usually lighter
Attribution of findings	A failure may come from the model, the prompt, a filter or the parser, so triage needs application logs	A failure points more directly at the model or route under test, which simplifies root cause analysis
Production load and side effects	Each probe runs through application infrastructure, which may trigger rate limits, logging, billing or downstream tool calls	Load falls on the provider or model server, so confirm quotas and whether tool use is enabled
Recommended first endpoint	Start at the gateway if one exists, to test the policy layer every application shares	Start at the model when you need a baseline, then move to the application to test what users actually reach
What to confirm before running	Whether the test environment mirrors production guardrails, and who approves scan traffic against the endpoint	Which model version and route the provider serves, and that results are recorded against that version

Configuring the garak REST generator in YAML, step by step

This sequence takes a team from an empty Python environment to a garak REST generator that sends prompts to its own application, reads the reply and records each attempt. Each step ends with a check you can verify before moving on. Treat the first pass as a debugging run that proves the wiring works, and do not read its output as a security result.

Install garak from PyPI in isolation
The engineer creates a virtual environment on a Python 3.11 or newer interpreter, activates it, and installs garak with pip from PyPI. Keep the scan host separate from the application's own runtime so dependencies and API keys stay contained. Record the installed version, since the release you pin determines which YAML keys and probes you can rely on. Done when garak runs from the command line and prints its help text without import errors.
Capture one valid request and response
A developer who knows the application, working with a non-production test account, sends one ordinary chat message through the normal interface and records the exact HTTP request. Use browser developer tools or a proxy you control, and note the method, full URL, headers and body shape. Redact the token in any shared notes. Then save the raw response body, including the nesting that holds the model's text. Done when you can replay the request with a single command and receive the same response structure.
Write the YAML endpoint file
The engineer translates the captured request into a YAML endpoint file that names the URL, the HTTP method, the headers, a request body template with a placeholder for the prompt, and the path to the reply text in the JSON response. Reference the credential through an environment variable so the file itself holds no secret. Confirm exact key names and template syntax in the user guide at docs.garak.ai for the release you pin, because these details change between versions. Done when the file parses and the placeholder and reply path match the captured sample.
Confirm one prompt extracts the reply
Send a single benign prompt through the generator from the command line, pointing at the YAML file. Compare the text garak receives with the reply text you located in the captured response. Check that the generator returns only the model's words, without surrounding JSON or an error message. Inspect garak.log for request and parsing messages. Done when the extracted text matches the reply field exactly.
Run a small probe set first
Select a few probes from different categories and limit the run to them, so you can read every attempt and fix wiring problems quickly. Garak makes 10 generations per prompt by default, so even a small set sends many requests to your endpoint. Confirm with the application owner that this load is acceptable on a test environment. Review the JSONL report and garak.log after the run. Done when every attempt in the report shows a sensible prompt and a reply that was extracted correctly.
Schedule the full run after review
Once the small set produces clean, readable attempts, widen the probe selection toward the full run. Keep the same garak release pinned so results compare with later runs. Record the endpoint file, garak version and probe list in your run notes. Done when the full run completes and writes its JSONL report, and the team has reviewed each probe marked FAIL before drawing conclusions about the application.

How gateway, direct application and air-gapped targets differ

An AI gateway sits in front of several models and usually presents one API to callers. When garak targets the gateway, the scan exercises the gateway's routing, policy and filtering together with whichever model answers each request. That makes the result a statement about the whole path, not about a single model, and it means the model that produced a given output may change between runs if the gateway routes by load, cost or tenant.

What to check before configuring the generator:

Authentication: confirm whether the gateway expects a bearer token, an API key header or a tenant-scoped credential, and whether that credential has permission to call every model the gateway exposes.
Scan host placement: the scan host must reach the gateway's public or internal listener on the port it serves. Confirm any allowlisting or private DNS names with the gateway team.
Generator config: set the request headers for the credential, and map the response field that holds the completion text. Gateways often wrap model output in an envelope, so the JSON path may differ from the model's native format.
Model selection: if the gateway accepts a model parameter, decide whether the scan pins one model or lets routing choose, and record which.
What a result shows: a FAIL shows that the gateway path produced undesirable output for that probe at that time. It does not show which model failed, nor that other routes are equally exposed.

Record the gateway version and routing configuration alongside each JSONL report so a later run can be compared fairly.

What drives the effort of a custom REST target

This table lists the factors that change how long a garak REST target takes to configure and keep working, and how to reduce each one. Read each row as a lever: the left column names the driver, the right columns explain why it moves effort and what to do about it. The main takeaway is that a small, stable, well-documented endpoint with one YAML file per environment costs far less to maintain than an endpoint that changes often or needs custom handling.

Driver	Why it changes effort	How to keep it down
Number of endpoints and request shapes	Each distinct route or request body needs its own mapping in the YAML and its own test of parsing, so effort grows with every shape.	Expose one test endpoint that accepts a single documented request format, and keep other routes out of the scan path.
Authentication scheme	A static key is a header entry. Session tokens need a login step, signed requests need a signing routine, and mutual TLS needs certificates and trust configuration on the scan host.	Use a static key or a test-only token issued for scanning, and confirm with your security team which scheme the scan host is allowed to use.
Response format	Plaintext is read directly. JSON with nested fields needs an accurate path to the text, and streamed output needs chunk handling that the generator may not cover, so confirm support before scoping.	Return plaintext or a flat JSON object with the generated text in one known field, and avoid streaming for the scan endpoint.
Conversation or session state	If each answer depends on earlier turns, the scan must send and track session identifiers, and results become harder to reproduce between runs.	Use a stateless test mode where each prompt is independent, and document any state the endpoint requires.
Number of environments	Each environment (development, staging, production) needs its own base URL, credentials and tuned settings, multiplying the work to set up and verify.	Keep one reusable YAML per environment that differs only in URL and key reference, and share the rest of the configuration.
Guardrail layers in front of the model	Input and output filters, moderation services and gateways can block, rewrite or truncate prompts, which changes what garak sees and can require tuning of detectors and parsing.	Document each layer in front of the model, decide with the owning team whether the scan tests the model, the guardrails or both, and record the chosen path.
Change frequency of the API contract	Every change to field names, paths or auth flow can break the YAML and force a rerun of verification, so effort recurs for as long as the contract moves.	Agree a fixed contract version for scanning, pin the garak version used for comparisons, and set a review step before any contract change reaches the scan configuration.

Checking garak REST generator results before you trust them

Before you read a garak result as a finding about your application, confirm that garak reached the application the way you intended. A clean or failing report means little if the REST generator sent the wrong request, received an error page, or parsed an empty string as a model reply. Start with a known-good prompt, one you know your application answers correctly, and check that the text garak stores in the report matches what the endpoint returns when you call it directly with the same headers. Then deliberately break the authentication header, using an expired or invalid key in a test environment. The generator should surface a clear error, such as a rejected status or a failed request, rather than recording a silent empty reply. If it records an empty reply quietly, every probe result downstream is unreliable.

Next, verify response parsing against several real replies, not one. Collect a normal answer, a refusal, a long multi-part answer, a reply containing markdown or code, and at least one error body returned by the gateway, such as a rate-limit or content-filter message. Run each through the parser configuration in your YAML file and compare the extracted text with the raw body. Error bodies deserve particular attention, because a parser that pulls a field such as the message text from a JSON error can make a gateway refusal look like a model answer. Confirm with the owner of the application which JSON path or plaintext form carries the model output, and record that decision alongside the YAML so a later change to the response schema is caught.

Read the default settings carefully. garak makes 10 generations per prompt unless you change that setting, and it marks a probe as FAIL with a failure rate when any attempt produces undesirable output. That means a single bad generation out of ten can fail a probe, and a failure rate describes how often the model produced output the detector flagged across attempts, not how severe the weakness is. Interpret the number against what the target actually does. A stochastic application may pass one run and fail the next, so a single run is not a baseline. Repeat runs on an unchanged target, compare the spread in failure rates, and only then decide whether a change in your application or model has moved the result. Record the garak version, probe set and generation count with each run so comparisons stay valid.

Finally, open the JSONL report and check for empty or truncated responses before you count passes. The report records every attempt and its evaluation, so filter for attempts whose output is blank, very short, or cut off at a token or timeout limit. An empty or truncated response is not a pass: the detector had nothing to flag, so the probe looks safe while the weakness was never exercised. Count these cases per probe and ask whether the target timed out, returned an error body that the parser accepted, or stopped early by design. Pair this check with the garak.log file, which carries the debugging detail that explains why a generation failed. Confirm these steps with your own engineering team, because the right timeout, parser path and generation count depend on how your application is built and deployed.

Handling auth headers and endpoint credentials during scans

The REST generator reads authentication headers and endpoint details from a YAML file, so that file becomes a credential store and a source of production load. The cards below cover how to keep secrets out of the configuration, which identity garak should use, how scans affect the target application, and how to handle the output that garak writes.

Keep keys out of the YAML file

The REST generator YAML describes headers such as Authorization, and a literal token written there is easy to commit, copy or paste into a ticket. Reference credentials from environment variables or a secrets store at run time, so the file holds only the endpoint shape. Before sharing a configuration, check it for any literal token, cookie or API key, and confirm your tooling reads the variable you intend.

Use a dedicated scan identity

Create a separate account or service credential for garak instead of reusing a developer or administrator token. Grant only the role the application needs to answer prompts, which is usually a standard end-user or chat role without access to configuration, model settings or other users' data. Confirm with the application owner which roles exist, because the lowest role that works may differ between a gateway and the application behind it.

Keep live configurations out of shared repositories

Store the endpoint YAML with placeholders in version control and keep the filled-in version on the machine that runs the scan. Shared repositories, chat threads and CI logs often expose files more widely than the security team intends. Add the working configuration file to ignore rules, and review pull requests for endpoint hostnames and header values that point at live systems.

Plan for rate limits on the target

garak sends many prompts per probe, and by default each prompt is generated multiple times, so a single run can exceed the request limits of a gateway or application. Ask the application owner which limits apply to the scan identity and whether a dedicated quota or an off-peak window is available. Agree a concurrency setting before the first run, and watch for throttling responses that could distort results.

Account for cost per request

When the target calls a paid model API, every probe attempt is a billable request, and the scan's cost depends on the number of probes selected and the generations per prompt. Ask who pays for the model usage behind the endpoint, whether a budget cap exists, and whether the scan identity is tagged so its spend can be separated from production traffic. Estimate the request count from the selected probes before any production run.

Expect probe content in application logs

Gateway, application and model provider logs will record the adversarial prompts garak sends, which can fill log storage and mix test traffic with real user activity. Ask the application team whether scan traffic can be tagged or filtered in logs, and whether retention rules for prompt logs apply. Check that the logs containing probe content do not feed analytics or customer-facing dashboards.

Handle JSONL reports and garak.log as sensitive

garak writes a JSONL report of every attempt and its evaluation, plus a garak.log file that continues across runs, and both contain generated text that may be harmful or may echo sensitive content. Store these files with the same access controls, retention rules and encryption you apply to other security test output, and limit who can share them. Review them before they leave the testing team, because a report can reproduce the exact responses that caused a failure.

What to keep after the first scan on a custom endpoint

Keep the garak version you tested with, pinned exactly, and record that pin next to every result. The garak releases page lists v0.17.1 as released on 16 October 2026. If your first run used that release, pin it in your requirements file or lock file and do not let a fresh install pull a newer version into the same pipeline. A later run on a different garak version may use different probes, detectors or defaults, so a change in results could come from the tool rather than the application. Install the tool from PyPI in a dedicated environment, and note the Python interpreter version as well, since the package requires Python 3.11 or newer.

Treat each YAML endpoint file as a test asset. Store it in version control alongside a short test note that says what the endpoint is, which gateway or service it fronts, which authentication header the file expects, how the response is parsed, and which probes were selected. Reviewers will need this file to understand why a finding was raised or why an attempt was classified as a failure. Remove any literal credentials from the file before committing it. Reference the environment variable or secret store entry that supplies the key instead, and confirm with your secrets management owner where that reference is kept.

For every run, record which build of the application was under test. The build identifier might be a container image digest, a git commit, or a release tag, depending on how your team deploys. Without it, a run from last month cannot be compared with a run from today in any meaningful way, because the model, system prompt, retrieval index or guardrail configuration may have changed. Write the build identifier into the run folder name or a run manifest, together with the garak version, the YAML file revision and the date and time.

Retain the JSONL report and the garak.log file for every run, and keep them together in one folder per run. The JSONL report holds each attempt and its evaluation, which is the basis for comparing failure rates across builds. The garak.log file is continued across runs, so it is useful for debugging but harder to use as a per-run record unless you archive it at the end of each run. Confirm how long these files must be kept, who may access them and whether they contain sensitive prompts or responses, with your own security and compliance policy owner. Scan output can include harmful generated text, so access controls on the archive matter as much as the retention period.

Re-check the YAML file whenever the application's API contract, authentication scheme or response format changes. A new header name, a token exchange step, a renamed JSON field or a switch from plaintext to structured output can all cause the generator to send requests that the application no longer understands, or to parse responses incorrectly. A broken parse can produce misleading results that look like a clean run. Before trusting a scan after any such change, send a small known request, confirm the response is read as intended, and note the check in the test notes. Assign an owner for this review, and make it part of the change process for the application's API.

Questions teams ask before testing a custom LLM endpoint

Is garak free to use for testing our own LLM application?

Yes. garak is licensed under Apache 2.0 and the maintainers describe it as a free tool, so the license itself carries no fee. The money goes elsewhere. Cost sits in three places: model or gateway usage for each scan, compute for the scan host, and engineering time to write the endpoint configuration, run scans and review results. Every probe sends prompts to your endpoint, and by default garak makes 10 generations per prompt, so usage scales with the probe set you choose. Ask any quote to separate one-time setup from recurring scan costs, and to state which token or request volumes it assumes.

How long does the YAML setup for a REST generator take?

The duration depends on how well your endpoint is documented and how many authentication and response details must be mapped. garak's REST generator needs a short YAML file that describes the endpoint, its headers and how to read the response, which works for plaintext or JSON. The main drivers are: the authentication scheme (static key, token exchange or signed requests), whether the response is plain text or nested JSON, any request body template your gateway requires, and whether a test account exists in a non-production environment. Ask for the setup steps and a test run against a single probe in the written scope before committing to a full scan.

What do we need to provide before garak can scan our endpoint?

You provide the access, the documentation and the ownership. Specifically: a test account or test API key with rate limits you approve, API documentation or an example request and response, a scan host inside a network that can reach the endpoint, and a named owner who receives the JSONL reports and decides what happens to findings. Confirm with your platform team whether the endpoint sits behind a gateway, a web application firewall or a content filter, since those controls change what garak sees. Also confirm which logging rules apply, because each run writes a JSONL report of every attempt and a garak.log file that persists across runs.

Can our in-house team run garak against a custom endpoint without outside help?

Yes, a team with Python and API experience can do it, and garak is installed from PyPI with pip. The PyPI package requires Python 3.11 or newer, and the project develops on Linux and macOS. The harder part is judgment, not installation: choosing probes that match your application, writing a YAML file that handles your authentication correctly, and interpreting failure rates in a probewise run. Teams often find that the first configuration takes longer than expected. Decide in advance who owns the configuration, who reviews results and how probe selection is documented so that later runs remain comparable.

What does a fixed-scope garak engagement from DeployOpen cover?

A fixed-scope engagement covers the work you agree in writing before it starts: the endpoints in scope, the probes and detectors to run, the REST generator configuration, the scan runs, and the report of findings and evidence. Most engagements are fixed-scope, so scope, deliverables and timelines are agreed before work begins. DeployOpen runs security testing of AI systems and agents, and prepares evidence for compliance assessments, but it does not issue audit reports or certifications. Ask the proposal to list excluded items, such as remediation of the application, and to name who receives and retains the reports.

Which factors make a custom LLM endpoint scan take longer?

Scan duration is driven by the number of probes, the generations per prompt, and how fast your endpoint responds. garak's default of 10 generations per prompt multiplies requests, and a probe set covering hallucination, data leakage, prompt injection, toxicity and jailbreaks produces a large volume of attempts. Endpoint latency, rate limits and any queueing on your side set the real pace. Authentication refresh logic and response parsing errors can also stop a run early. Ask for an estimate that states the probe set, the generation count, and the rate limit assumed, so you can compare quotes on the same basis.

Does ongoing support come with a garak engagement?

Ongoing support is available as a retainer, separate from a one-time fixed-scope engagement. garak releases frequently, and v0.17.1 was released on 16 October 2026, so probe sets and behavior can change between runs. A retainer can cover keeping the version current, pinning it for comparable results, re-running scans after endpoint changes and reviewing new findings. Ask whether the quote includes version pinning, how changes to the endpoint trigger new scans, and how reports are handed over after the engagement ends.

How DeployOpen scopes, builds and hands over garak endpoint testing

DeployOpen points garak at your own LLM application or gateway through its REST generator, with the endpoint described in a short YAML file. We plan, build, document and hand over that setup inside your infrastructure, under a written scope agreed before work starts. Ongoing support is available as a retainer once the scans are running.

How we scope

Confirm which application, gateway or endpoint garak will target and who owns it.
Record the request and response format, so the YAML describes the endpoint accurately.
Agree in writing which probe categories, runs and report outputs are in scope.

How we build

Write the REST generator YAML and test it against a non-production copy of the endpoint.
Configure authentication headers from your secret store, not from files in the repository.
Check response parsing on plaintext and JSON replies, then confirm a full probe run completes.

How we hand over

Deliver the YAML configuration, run instructions and a record of the pinned garak version.
Document how JSONL reports and garak.log are stored, reviewed and retained in your environment.
Walk your engineers through a sample run so they can rerun scans without us.

How we support

Offer a retainer for updates when you move to a newer garak release, such as v0.17.1.
Review changes to your endpoint and adjust the YAML when request or response formats change.
Confirm with your team the cadence for rerunning scans and comparing reports against earlier runs.

garak REST generator for testing custom LLM applications