Why garak security hardening for scan environments matters before the first run
A security or platform team about to run garak against a model or application should treat the scan host as a sensitive system before the first command is typed. garak, the Generative AI Red-teaming and Assessment Kit, is a command-line tool installed from PyPI with pip, and it probes for hallucination, data leakage, prompt injection, toxicity generation and jailbreaks. That means the scanner itself handles the material being tested. The host will hold credentials for the target, generate prompts and outputs that are deliberately harmful, and write reports that record every attempt. Hardening the environment is therefore a decision about where the tool runs, who can reach it and what it leaves behind, not an afterthought once results arrive.
The first risk is credential exposure. Scans against hosted models usually depend on API keys or tokens stored in environment variables, configuration files or a shell history. A scan host that other engineers share, or one that syncs to a laptop backup, can spread those keys beyond the team that approved the test. Before the first run, confirm where each key is stored, which account it belongs to, what spending and model access it grants, and whether a dedicated key with a narrow scope and an expiry date can be issued for scanning. Also confirm that run configurations and logs do not echo secrets back to the console or into files that outlive the test.
The second risk is that scan traffic looks like attack traffic to the target. garak sends large volumes of adversarial prompts by design, so a production endpoint, a shared gateway or a provider account with rate limits can see unusual spikes from a single source. Without an agreed target, a rate limit and a named owner who is watching the endpoint, a routine scan can trigger incident response, lock out legitimate users or consume quota that other workloads need. Decide in advance whether scans run against staging, a replica or a production endpoint, and get written approval from the owner of each target. Ask the provider or gateway team what limits, alerts and blocking rules will apply to the scan source.
The third risk is persistence. Each garak run writes a JSONL report of every attempt and its evaluation, and a garak.log debugging file that continues across runs. Those files can contain harmful generated text, fragments of sensitive prompts and, depending on configuration, details of the target. They are not temporary artifacts. Plan where reports are stored, who can read them, how long they are retained, how they are encrypted at rest, and how they are deleted or archived under your records policy. The default setting of 10 generations per prompt multiplies the volume of material written to disk, so the storage and handling plan should be based on the actual configuration you intend to use, not on a single test run.
Before the first run, the practical step is a short written checklist: a dedicated scan host or isolated container with a defined owner, scoped and rotated credentials, an approved target list with rate limits, a storage location and retention rule for reports and logs, and a review path for harmful output. Pin the garak version you tested, since garak v0.17.1 was released on 16 October 2026 and later releases may change probes and defaults. Confirm the Python requirement of 3.11 or newer on the host. Ask your security team to sign off on this checklist so that the first scan is repeatable and defensible.
Which garak parts carry risk on the scan host
A garak run touches several components, and each one handles data differently. Knowing which plugins load code, which generators hold credentials, and which files collect prompts and model output tells you where to restrict access and what to review before sharing results.
Probes and the prompts they send
Probes are the plugins that generate the adversarial prompts garak sends to a target, covering areas such as prompt injection, data leakage, jailbreaks and toxicity. Their content is usually harmless to the scan host, but the prompts are sent verbatim to whichever model or endpoint you configure, so they can reach third-party APIs. Check which probes your run loads, and confirm that the target is one your data policy allows to receive them.
Detectors that judge model output
Detectors inspect each model response and decide whether it counts as a failure, for example by looking for toxic text, leaked strings or unsafe instructions. Because they read raw output, they process the same potentially harmful content the probes elicit, and that content is written to disk. Review each detector the default probewise harness selects, and confirm that your team understands which detector flags which weakness before you act on a FAIL result.
Evaluators that score attempts
Evaluators turn detector results into the pass or fail outcome and the failure rate reported for each probe. They rarely hold sensitive data themselves, but they determine what appears in the summary you share with stakeholders. Check that evaluator settings are recorded with each run, so a reviewer can explain why a probe was marked FAIL when the report is read months later.
Generators and endpoint credentials
Generators are the connectors that send prompts to a model or application. Generators for hosted providers and the REST generator read API keys or authentication headers from configuration or the environment, which makes the scan host a credential store. Restrict who can read the configuration files and the shell environment on the host. The REST generator is where endpoint credentials are configured, and its setup is covered in the sibling topic on the REST generator for custom LLM applications.
The JSONL report of every attempt
Each run writes a JSONL report that records every attempt and its evaluation, which usually includes the full prompt and the full model response. This is the file most likely to contain harmful generated content and anything a model echoed back, including data leakage findings. Store it in an access-controlled location, decide a retention period before the first run, and review it before sharing it outside the security team.
The garak.log file across runs
The garak.log file is a debugging log that continues across runs rather than starting fresh each time. It can capture request details, errors and fragments of responses, so it may hold sensitive material even when the JSONL report does not. Because it persists, old content accumulates unless you rotate or clear it. Check the log for credentials or identifiers after each run, and confirm the log location in your scan environment before automation starts writing to it.
Where garak should run and how it should reach targets
This table compares five places to run garak and how each one reaches its target, from a dedicated segregated scan VM to a scan against a staging replica. Read each row for the fit, the operating burden on access control, patching, key handling and reproducibility, and the network facts you must confirm before choosing. The main takeaway is that the placement decision sets how far garak's probe traffic and API keys can reach, so choose it before configuring any target.
| Option | When it fits | Operational cost and what to confirm |
|---|---|---|
| Dedicated scan VM in a segregated network | Recurring scans against production-like endpoints where one team owns the scan host and its logs | Someone must own OS patching, Python environment pinning, and who can log in. Confirm that egress allows only the approved model or gateway addresses and that the JSONL reports land in an access-controlled store. |
| Container with restricted egress | Repeatable runs where the image pins garak and its dependencies and can be rebuilt for comparable results | Image rebuilds and registry access need governance, and secrets must be injected at run time rather than baked in. Confirm your container runtime can enforce egress allow-lists and that mounted report volumes have restricted permissions. |
| Shared engineering workstation | Early exploration by one engineer against a sandbox or non-sensitive target | Hardest to control: keys, reports and generated harmful content sit beside everyday work, and results are hard to reproduce. Confirm whether the workstation can reach production endpoints and whether disk encryption and screen locking are enforced. |
| CI runner | Gating checks on a model or prompt change where the pipeline already holds a scoped service identity | Runner pools are often shared across projects, so keys, logs and garak.log may be visible to other jobs. Confirm runners are ephemeral, that scan secrets are scoped per pipeline, and that failed-run artifacts are restricted before attaching JSONL reports. |
| Scan against a staging replica of the target | Active probing that could degrade, log or alter a live service, such as adaptive probes at high attempt counts | A replica must match the production model version, system prompt, guardrails and gateway rules, or findings will not transfer. Confirm with the owning team how closely the replica tracks production and who approves data and key differences. |
How to harden a garak scan host, step by step
This sequence turns a general-purpose machine into a contained environment for garak scans, with a pinned tool version, limited network reach, protected credentials and separated outputs. Each step ends with a check you can record as evidence that the control works.
Create a dedicated scan host or container
The platform or security engineer provisions a host or container used only for garak runs, with no shared workloads, personal browsing or unrelated tooling installed. Keep the image or build minimal and record who has administrative access. This step is done when the host inventory lists the machine as a garak scan environment with a named owner and a restricted administrator group. Confirm the access list against your identity provider before the first scan.
Set up a Python 3.11 environment
Create an isolated virtual environment using Python 3.11 or newer, because the garak PyPI package requires that version. Do not reuse a system interpreter shared with other applications. The engineer runs the interpreter's version command inside the new environment and saves the output with the build record. The step closes when the reported version is 3.11 or higher and the environment is the only one active for scan jobs.
Install and pin the garak version
Install garak from PyPI with pip inside the environment, then record the exact version in use with the package metadata command. garak is developed in the NVIDIA GitHub organization and is licensed under Apache 2.0, so keep the release notes for the version you approved. Store the version in your change record and in the scan report metadata so later runs can be compared. The step closes when the recorded version matches the approved entry. The latest published release at the time you install may differ from the one your team tested, so confirm the version against the NVIDIA releases page first.
Restrict outbound network access to approved endpoints
Configure host firewall rules, egress proxy or network policy so the scan host can reach only the target endpoints approved for the engagement and any required package or model repositories. garak can send large numbers of prompts, so unrestricted egress also risks unintended traffic to other systems. Test the policy by attempting a connection to an unlisted host from the scan host. The step closes when that connection fails and the attempt appears in your firewall or proxy log.
Inject credentials from the secret store at run time
Keep API keys and endpoint tokens in the host's secret store or your enterprise vault, and have the job retrieve them as environment variables when the scan starts. Do not write keys into generator YAML files, shell history, scripts or repository files. Restrict the secret to the service account that runs garak. The step closes when a configuration review finds no literal credentials in the scan directory and the key is removed from the environment after the run.
Set file permissions and report retention
Define a retention location for JSONL reports and the garak.log debugging file, with owner-only read and write permissions for the scan service account and read access limited to named reviewers. Garak reports contain generated model output, which may include harmful or sensitive content, so decide how long they are kept and how they are deleted. garak.log continues across runs, so plan rotation or archival. The step closes when a permissions listing shows the expected owners and modes, and a test user outside the reviewer group cannot read the files.
Separate output directories for each engagement
Create one output directory per engagement or target, named with the engagement identifier and the garak version, and direct each run's reports into that directory. This keeps results for different models or applications from mixing, and it lets you retain or delete one engagement without touching others. Confirm the output path in the run configuration before starting. The step closes when a test run writes its report and log only into the intended engagement folder and no other folder changes.
Verify that keys stay out of reports and logs
Before the first production-facing scan, run a short test using a dummy key value and search the generated JSONL report and garak.log for that value. Confirm that the output contains no credentials, headers or tokens. Record the search command and result with the scan record. The step closes when the search returns no matches, and the result is repeated after any change to the generator configuration or garak version.
Protecting API keys, JSONL reports and harmful generated content
A garak scan touches three kinds of sensitive material: provider credentials, the report files it writes, and the model outputs those reports contain. Each needs its own controls, and the team running the scans should agree on them before the first run.
Scope API keys to test projects
garak calls OpenAI, AWS Bedrock, Hugging Face, Replicate or a gateway using credentials you supply. Create keys in a dedicated test project or account with spending limits and only the model access the scan needs, so a leaked key cannot reach production data or unrelated models. Confirm each provider's options for project-level keys and usage caps, since these differ by vendor.
Rotate keys after each scan cycle
Set a rotation rule tied to scan activity, for example rotating after each scheduled campaign or after any engineer with access leaves the team. Store keys in your secrets manager and inject them at run time rather than keeping them in configuration files. Confirm that old keys are revoked at the provider, not only removed from the host, and that the scan pipeline can pick up the new key without manual edits.
Keep keys out of logs and reports
Check that API keys are not echoed into the console, garak.log, JSONL reports or CI output. Run a test scan with a dummy key, then search the output directory and pipeline logs for that string. Pay attention to verbose or debug flags, which can widen what gets written, and to REST generator headers, which carry authentication values.
Define access to JSONL reports
Each run writes a JSONL file recording every attempt and its evaluation, and garak.log continues across runs. Both contain the prompts sent and the outputs received, which can include toxic, leaked or sensitive text. Limit read access to named roles, restrict write access to the scan service account, and record who can view or copy files from the scan host.
Encrypt and set retention for reports
Store report directories on encrypted volumes or object storage with encryption at rest enabled, and confirm the setting with your platform team. Write a retention period for raw JSONL and garak.log files, plus a deletion procedure that covers backups and copies. Keep only the summarized findings that remediation needs, and delete raw outputs when the retention period ends.
Control who may share reports
Decide in advance who may send a report outside the test team, such as to a model provider, a vendor or an auditor, and how it is transferred. Share summaries or redacted extracts by default. Full JSONL files should move only through approved channels and only when the recipient has a documented need.
Handle disturbing generated content
Reviewers may read toxic, abusive or otherwise harmful model outputs. Give the team guidance on how to review them, including breaks, a named escalation contact and a rule that reviewers can flag material without reading all of it. Agree with your HR and wellbeing functions what support is available before scans begin.
Keep results out of tickets and chat
Do not paste raw garak results into tickets, chat channels or email by default, because those tools often have wider access and longer retention than the scan host. Reference the report location and a finding ID instead, and share a short written summary that describes the failure without repeating the harmful text.
Limiting unintended load on production endpoints during garak runs
The main risk is that a garak run becomes a load test nobody approved. garak sends many prompts per probe, and by default it makes 10 generations per prompt. A single probe run can therefore send far more requests to a model or application endpoint than a typical user session does. If that endpoint is production, the scan can slow real users, consume quota, and trigger alerts or incident response. Treat the target choice and the request volume as decisions to make before the first command runs, not as details to tune afterward.
Start by pointing scans at staging or a dedicated test deployment that mirrors the production configuration closely enough to produce meaningful results. Where production testing is unavoidable, agree a test window and a request budget with the endpoint owner in writing. The budget should state the maximum number of requests the scan may make and who can stop it. Probe selection matters as much as the target. Choose probes that match the threat model for this application, such as prompt injection or data leakage for a retrieval assistant, instead of running the full set. Fewer probes means fewer prompts, fewer generations and a smaller, more explainable footprint.
Give the scan its own credentials. Create a separate API key for garak, with its own quota or rate limit, so that a runaway scan exhausts only its own allowance and not the keys used by production services. Confirm with your provider how rate limits, quotas and billing apply to that key, including whether limits are per key, per organization or per project, and how overage is charged. These figures vary by provider and plan, so get them from the provider's current documentation or account team rather than from assumptions. The same check applies to any gateway or REST endpoint that garak calls on your behalf.
Watch the run while it is in progress. Before starting, define which signals will stop it: error rate increases, latency growth on the endpoint, rate-limit responses, alerts from monitoring, and spend against the expected cost of the run. Make sure someone on the endpoint owner's side is reachable during the window, and that the person running garak can kill the process and revoke the scan key immediately. Afterward, compare the request count in the garak.log file and in your provider's usage records against the planned budget. A mismatch is a finding to explain before the next run, not after the next invoice.
How isolation changes for workstations, CI runners, Kubernetes and air-gapped networks
A single engineer workstation is the fastest place to run garak, and it is the environment where isolation is weakest. The scan runs under the engineer's own user account, so any API key in environment variables, config files or shell history is readable by every process that user starts. Reports and garak.log are written to the working directory by default and remain there until someone removes them.
- Credentials: keep provider keys out of shell profiles and committed config files. Load them from a secret store for the session only, and confirm how your garak version reads keys from the environment.
- Reports: JSONL files record every prompt and model output, including harmful generated content. Keep them out of cloud sync folders and backup paths that index or copy the working directory.
- Logs: garak.log continues across runs, so one log accumulates the history of every engagement on the machine. Rotate or clear it between targets.
- Shared or personal devices: a laptop used for other work may sync, index or back up the scan folder. Use a dedicated scan account or an excluded directory.
- Load: one operator controls request volume, but default settings make 10 generations per prompt. Confirm the target's rate limits and billing before a full run.
Check which keys are present in the environment during a scan, and list every location where reports and logs are copied after the run.
A CI runner is ephemeral, so the host itself is discarded after each job. The exposure moves to the pipeline: secrets are handed to a job that many people can influence, and the job's logs and artifacts are often visible to a wider group than the security team that owns the scan. Isolation on a runner depends on who can edit pipeline definitions and who can read job output.
- Masked variables: masking hides a secret in log output but does not stop a pipeline step from printing it through a debug command or an error trace. Restrict who can edit the pipeline file that receives the key.
- Job logs: garak's console output and garak.log may include prompt text and model responses. Confirm the retention period and who holds project read access.
- Artifacts: uploading JSONL reports as artifacts makes them available to every member of the project. Limit artifact visibility and set an expiry.
- Fork and merge request triggers: runs triggered by untrusted changes should not receive production keys or reach production endpoints.
- Caches: pip and model caches persist between jobs on some runners. Confirm what is cached and whether a later job can read it.
Check who can read the job log, the artifact store and the pipeline definition, and confirm that the scan job receives only the keys it needs for its own target.
On a Kubernetes cluster, isolation is set by network policy, the secret delivery method and the storage that holds reports. The controls you need depend on how garak is packaged, such as a container image run as a Job or CronJob, or a longer-running pod your team maintains. Confirm the packaging model first, because it determines where keys sit and which volumes receive reports.
- Egress: a network policy should allow outbound traffic only to the target endpoints and any model hosts the scan needs. Default-allow egress lets a compromised scan pod reach other services.
- Secret mounts: mount API keys as files from a Secret with restricted read permissions, instead of exposing them as environment variables that appear in pod specs and debugging output.
- Service account: give the pod a service account with no cluster API permissions. The scan only needs to reach its target.
- Report storage: write JSONL reports and garak.log to a dedicated volume with access limited to the reviewing team. Define a retention and deletion step.
- Runtime settings: a read-only root filesystem with a writable volume for output limits what a compromised container can change. Confirm that the container image installs garak from a pinned version, since garak requires Python 3.11 or newer.
Check the packaging model, then verify with the cluster's policy tooling that the egress rules and secret mounts match the endpoints listed in the scan configuration.
An air-gapped network removes the easy paths that other environments take for granted. Model weights, Python packages and cloud API calls cannot be fetched during the scan, so everything the run needs must be staged in advance. Reachability becomes a planning task, and reports must follow an approved transfer process before they can leave the enclave.
- Model downloads: models from the Hugging Face Hub must be downloaded outside the enclave, verified and transferred in. Record the exact revision you staged so later runs test the same weights.
- Cloud API targets: OpenAI, AWS Bedrock and Replicate targets are unreachable from inside an enclave unless an approved route exists. Confirm whether an internal endpoint can be tested through the REST generator instead.
- Packages: mirror garak and its dependencies into an internal repository, along with a Python 3.11 or newer runtime.
- Reports and logs: JSONL files and garak.log may contain sensitive model output. Confirm the classification level and the approved method for moving them out.
- Reachability matrix: list every target, model host and package source, then record whether each one is reachable from the enclave.
Check every target and dependency against the reachability matrix before the enclave test begins, and confirm the transfer process for reports with the data owner.
What drives hardening effort for a garak scan environment
This table lists the factors that make garak host hardening larger or smaller, how to keep each one manageable, and which cost component it moves. Read each row as a question to answer about your own environment before estimating work. The main takeaway is that most effort comes from how many target types, teams and credentials the host must serve, not from garak itself, which is free and Apache 2.0 licensed.
| Driver | Why it changes effort | How to keep it down | Cost component moved |
|---|---|---|---|
| Number of target types | Hosted APIs, self-hosted models and custom REST applications each need their own endpoint configuration, outbound rules and authentication handling. | Standardize on one REST generator pattern where possible and keep one reviewed YAML file per target class. | Internal engineering time |
| Number of environments and teams sharing the host | Shared hosts mix test data, run histories and credentials across owners, which multiplies access reviews and isolation checks. | Use a standard host image and one scan host per trust zone rather than one shared host for everyone. | Infrastructure and internal engineering time |
| Sensitivity of prompts and generated content | Probes deliberately elicit harmful or leaked content, so JSONL reports and garak.log may need the same handling as sensitive data. | Write reports to a shared report location with access control and set retention rules before the first run. | Security review time |
| Compliance evidence the organization must keep | Assessors may ask for proof of who ran scans, which versions were used and how reports were stored and deleted. | Pin the garak version in the host image and record run metadata alongside each JSONL report. | Security review time |
| Network segmentation already in place | Scan hosts must reach model endpoints without gaining broad access to internal systems, so each segment needs a documented rule set. | Apply a reusable network policy template for egress to approved endpoints and confirm it with your network team. | Infrastructure and security review time |
| Number of credentials to manage | Each provider key or endpoint token adds rotation, scoping and revocation work, and leaked keys in configs or logs are a common failure. | Hold all keys in a single secret store and inject them at run time instead of storing them in YAML files. | Internal engineering time |
| Load on production endpoints | Default runs make 10 generations per prompt, so scans against production systems need rate limits and agreed windows. | Confirm limits and scan windows with the endpoint owner and configure a lower probe set for production-facing targets. | Internal engineering time and model API spend |
| Scan host sizing and patching | Python 3.11 or newer and the pinned garak version must be kept current on every host that runs scans. | Build the host from one image with a documented upgrade path and test each new garak release in a staging host first. | Infrastructure and internal engineering time |
How to verify that a garak scan host fails safely
Verify the scan host's failure behavior before you trust it with a production endpoint or a live API key. The checks below are designed to show that the host cannot leak credentials, reach systems it was not meant to reach, or flood a struggling target. Run them on a staging host that matches the production configuration, and keep the results with the change record so an auditor or reviewer can see what was tested and when. Confirm the network boundary first. From the scan host, attempt connections to a sample of endpoints that are not on the approved list, including an internal address, a metadata service address and an external host you do not expect to use. Each attempt should fail at the egress control, whether that is a firewall rule, a proxy allowlist or a security group. Then confirm that the approved target endpoint still answers. A test that only shows the approved endpoint works proves nothing about the boundary. Ask the network team to confirm which control enforces the rule so the evidence points at the right place.
Test credential failure paths. Replace the API key in the garak configuration or environment with a revoked key, then with a malformed key. Each run should stop with an authentication error and write that error to garak.log. The run must not retry with a different credential that happens to exist in the environment, a cached token or a default key from another profile. Check the environment variables, configuration files and any credential helper that garak's generator could read, and confirm that only the intended key is in scope. Record which credential source each test used.
Check that secrets stay out of output. After a normal run and after a failed run, search the JSONL report and garak.log for the key value and for its prefix. Use the exact string of the key you configured, not a pattern guess. Also check any shell history or wrapper script output that captured the command. If a key appears in a report, treat it as an exposed secret, rotate it, and fix the configuration path that produced it before the next scan. Confirm with the garak maintainers' documentation which fields each report records for your generator type, since a custom REST generator may echo headers you add.
Check file permissions and a bounded failure response. Confirm that the report directory, garak.log and any generated files have the owner and mode your data handling policy requires, and that scan output is not world-readable on a shared host. For the failure response, point the scan at a deliberately slow or failing target, such as a test stub that returns errors or delays. The run should end within the time and attempt limits you configured, and the log should show a finite number of retries. Ask the team that operates the target what request volume it can absorb during an incident, and set the scan's concurrency to stay under that figure.
Confirm report structure and record the run. Run a small scan against a staging target that you control and confirm that the JSONL report contains the expected entries for each attempt and evaluation, and that garak.log has the run's entries appended. Compare the structure to the documentation for the version you installed. Record the garak version, the exact command line, the probe and generator names, the configuration file used and the target identifier. Store that record with the results so the run can be repeated later on the same version, which is what makes a later comparison meaningful.
Questions teams ask before hardening garak scans
Is garak free to use for our internal LLM testing?
Yes, the tool itself is free. garak is licensed under Apache 2.0, and its maintainers describe it as a free tool. The license covers the software, so your costs sit elsewhere: model API or GPU time when you probe a hosted or local model, internal engineering time to configure targets and review results, and the infrastructure for the scan host. Each model provider you test against has its own terms and billing, which are separate from garak. Ask your legal team to confirm how Apache 2.0 applies to internal use and to any modified code you share. Then ask each provider what a full run against your target will bill.
How much effort does hardening a garak scan environment take?
Effort scales with how many hosts, targets and teams share the environment. The one-time work includes isolating the scan host, setting up how API keys are stored and scoped, defining report retention, and a security review of the scan design. Internal staff usually carry most of this, and an outside partner would price it from a written scope. Recurring work includes patching the host, reviewing new garak releases, rotating keys, and reviewing reports and logs after each run. Infrastructure and model usage are separate line items. Ask any provider to split the quote into one-time and recurring costs, and to name who is paid for each.
What drives the duration of a garak hardening project?
The main drivers are the number of targets, network complexity and whether production endpoints are involved. Duration also grows with the number of owners who must approve access, how mature your key management process is, how sensitive the generated content and reports are, and whether a staging endpoint exists for testing. A single host with one non-production target moves faster than several shared hosts with production endpoints and multiple teams. Any timeline should come from a written scope that lists these items. Ask the provider which assumptions the timeline depends on, and what changes if a driver grows during the work.
What do we need to provide before a garak hardening engagement starts?
Prepare four inputs before any outside work begins. First, network diagrams showing where scan hosts sit relative to model endpoints, gateways and key stores. Second, a list of targets with each owner, environment (production or not) and the model or application type. Third, access to your key management system so scoped credentials can be created and rotated. Fourth, a named owner for JSONL reports and the garak.log file, who decides retention, access and deletion. It also helps to have contacts for production endpoint owners who can agree on load limits and test windows before any scan runs.
Can our own team harden garak scans without outside help?
Yes, for a single small host with one target, a team can usually do it internally. The work is manageable when one dedicated host runs scans against one non-production model with a scoped key and a simple retention rule. It gets harder once several teams, targets and hosts share the environment, when production endpoints are in scope, or when reports contain sensitive or harmful generated content that needs controlled handling. At that point, the questions multiply: who owns each key, how logs are separated, and how load is limited. Decide early which of these your team can own, and where an outside partner would add review capacity.
How do we handle garak JSONL reports and log files safely?
Treat every report and log as sensitive data. Each garak run writes a JSONL report of every attempt and its evaluation, and the garak.log file continues across runs. Prompts and outputs may include harmful generated content or details of your system. Decide who can read these files, where they are stored and encrypted, how long they are kept, and how they are deleted. Because garak.log carries over between runs, plan rotation or archiving so older entries do not accumulate unmanaged. Confirm these rules against your data classification policy and name one owner who signs off on them.
Will garak scans put unintended load on our production endpoints?
They can, so confirm load before pointing garak at production. By default, garak makes 10 generations per prompt, and it marks a probe as FAIL when any attempt produces undesirable output. The total volume therefore depends on which probes run, how many prompts they contain and how many attempts are made. Before a run, ask the endpoint owner for rate limits, a test window and, where possible, a staging endpoint. Give garak a dedicated key with a quota so scans cannot starve other users. Estimate the request count from your probe selection and check it against the endpoint's capacity.
Does DeployOpen harden garak scan environments inside our own infrastructure?
DeployOpen works inside your infrastructure, including your cloud accounts, data centers or air-gapped networks. It plans, implements, hardens and documents self-hosted open-source platforms, and it runs security testing, including AI systems and agents. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. DeployOpen does not publish a price list, so a quote follows from the written scope. Ongoing support, upgrades and continuous testing are available as retainers. DeployOpen prepares systems and evidence for compliance assessments, but independent auditors and certification bodies issue the reports.
How DeployOpen scopes, hardens and hands over garak scan environments
DeployOpen plans, builds and documents garak scan environments inside your own cloud accounts, data centers or air-gapped networks. The work covers isolating scan hosts, protecting API keys, handling JSONL reports and keeping probe traffic within limits agreed for each target endpoint. Engagements are fixed-scope, with the scope agreed in writing before work starts.
How we scope
- The written scope lists which models, applications and REST endpoints garak will target.
- It names the probe families in use and the detectors that judge their output.
- Scope also records who may run scans, where JSONL reports are stored, and who reviews them.
How we build
- Scan hosts sit in a dedicated network segment with egress limited to approved model endpoints.
- API keys live in the client's secrets manager and are injected at runtime, not stored in YAML files.
- garak installs from PyPI into a pinned Python 3.11 or newer environment, with the version recorded.
How we hand over
- Handover includes the host build, endpoint configuration files and run instructions for the scan team.
- Reports from each run go to a controlled location, with retention and access rules documented.
- Runbooks cover handling harmful generated content in JSONL reports and garak.log, plus escalation steps.
How we support
- Ongoing support is available as a retainer covering scan host upkeep, key rotation and garak updates.
- Pinned versions stay in place until a reviewed upgrade is tested against the agreed baseline probe set.
- Ask in the quote which support hours, response terms and upgrade work the retainer includes.
