garak vs PyRIT: which to choose

garak or PyRIT: which suits your red team workflow

Choose garak when you need a broad, repeatable scan of a model endpoint from the command line, and choose PyRIT when your own red team needs to script attacks, score them and keep the history. garak, NVIDIA's Apache-2.0 scanner, is built from probes that generate attacks, detectors that judge whether the model failed, and generators that connect to targets. A single run covers hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks, and writes a JSONL report of every attempt plus a hit log of successful attacks. That suits a one-off assessment, a pre-release check or a quick run you can repeat after a model or configuration change. PyRIT, Microsoft's MIT-licensed framework, suits teams that want to write the attack logic themselves. It provides targets, prompt converters, scorers, multi-turn strategies such as Crescendo, TAP and Skeleton Key, and a memory layer that stores conversations and results in SQLite or Azure SQL.

Choose garak when the person running the test is not expected to build attack code and the output needs to be a standard report. Its default harness runs each probe with its recommended detectors, and the default of 5 generations per prompt marks a probe as FAIL when any attempt produces undesirable output. Choose PyRIT when the test depends on a conversation that evolves over several turns, when your team must write custom scorers, or when you need to query past results across engagements. Illustrative case: a team testing a customer-facing assistant may find that single-prompt scans look clean while a patient multi-turn approach succeeds, which points to PyRIT. Before committing, confirm the Python version each tool needs in your environment. The garak package requires Python 3.11 or newer, and PyRIT supports 3.10 up to but not including 3.15, with the documentation recommending 3.13.

The decision turns on three questions. First, who runs the tests: an assessor who needs results quickly points to garak, while an in-house red team that will maintain attack code points to PyRIT. Second, whether multi-turn scripting is required; if a scanner's single-prompt coverage will not reflect how your application is attacked, plan for PyRIT or pair it with another tool. Third, what report the buyer needs. garak's JSONL output and hit log are easy to hand to an auditor or a risk owner, while PyRIT's stored conversations and scores suit an internal team that reviews attack transcripts. Confirm the report format your stakeholders accept before you choose, because switching later means rewriting attack logic or re-running assessments.

Which tool fits which scenario

This table matches common garak vs PyRIT scenarios to the tool that fits better and the condition that decides it. Read each row as a starting point, then confirm the condition against your own targets, team skills and reporting needs before committing.

Scenario	Better fit	Condition that makes it so
One-off assessment of a model endpoint	garak	You need a broad sweep of probes against a model with a JSONL record of every attempt and a hit log, and you do not need custom attack logic.
Recurring in-house red team	PyRIT	Your team will write and version its own attack scripts, converters and scorers, and keep results in its SQLite or Azure SQL memory across campaigns.
Multi-turn jailbreak study	PyRIT	The study needs strategies such as Crescendo, TAP or Skeleton Key, plus scorers that judge each turn of a conversation.
Testing custom REST applications	garak, with the REST generator	The endpoint returns plaintext or JSON and can be described in a short YAML file. Confirm that your app's auth and session handling fit that config.
Testing custom REST applications with complex flows	PyRIT, with custom HTTP or WebSocket targets	The app needs multi-step sessions, stateful interactions or browser-driven steps through Playwright-based web app targets.
Team without Python engineers	garak	Operators run a command-line tool installed with pip and edit YAML, and do not need to write attack code. Python 3.11 or newer is still required.
Buyer who needs an audit-style report	Neither tool alone	Both produce raw records rather than a finished assessment report. Confirm with the buyer what report format and control mapping they expect, then plan the write-up separately.

How garak and PyRIT are built and what that means for operations

garak and PyRIT take different shapes: one is a pip-installed scanner you run against a target, the other is a Python framework you assemble into attacks. The cards below cover how each is put together, where results land and what your team takes on to host and run them.

garak's five plugin categories

garak is built from probes that generate attacks, detectors that judge whether the model failed, evaluators, generators that connect to targets, and harnesses that run them. The default probewise harness runs each probe with its recommended detectors, so a first assessment needs little design work. Check whether your evaluation needs custom probes or detectors, because that is where the plugin model starts to cost engineering time.

garak's pip-installed CLI

garak installs from PyPI with pip and needs Python 3.11 or newer, with development covered on Linux and macOS. It runs as a command-line tool on a workstation or CI runner, so there is no server to host. Confirm that your build image can provide the required Python version and that your security team approves installing packages from PyPI.

garak's report files

Each garak run writes a JSONL report of every attempt and its evaluation, plus a garak.log debugging file that continues across runs. Results are plain files on the machine that ran the scan, so retention, access control and encryption are your responsibility. Decide where these files go and how long they stay before the first scan produces sensitive prompts or model outputs.

PyRIT's component model

PyRIT is assembled from targets, converters, scorers, attacks, memory and datasets. Its targets cover OpenAI, Azure, Anthropic, Google and Hugging Face models, custom HTTP or WebSocket endpoints, and web apps driven through Playwright. Each piece is code your team owns, so the operating effort sits with the engineers who write and maintain the attack scripts.

PyRIT's memory layer

PyRIT memory records conversations, scores and results in SQLite or Azure SQL. SQLite keeps everything in a local file, which suits a single operator, while Azure SQL adds a managed database you must secure and back up. Ask who owns the database, where it is hosted and how multi-user access is controlled before you choose a backend.

PyRIT's three usage modes

PyRIT runs as a scanner through the pyrit_scan command, as CoPyRIT, a web GUI for human-led red teaming and team collaboration, or as a Python framework for custom attacks. CoPyRIT suits analysts who run attacks by hand, the scanner suits repeatable batch runs, and the framework suits engineers building new multi-turn logic. Pick the mode that matches your team, since each one implies a different hosting and access model.

Azure/PyRIT is archived

The original Azure/PyRIT repository was archived on 27 March 2026 and is read-only, and development continues at microsoft/PyRIT. Any internal fork, pinned install or documentation link that still points at the Azure path will not receive changes. Confirm that your install references and procurement records use the microsoft/PyRIT repository.

License terms and what each project includes

garak is licensed under Apache 2.0 and is developed in NVIDIA's GitHub organization. Its maintainers describe it as a free tool, and the user guide is published at docs.garak.ai. The most recent release in the supplied facts is v0.17.1, published on 16 October 2026. The open-source package is the command-line scanner itself, which installs from PyPI with pip and requires Python 3.11 or newer.

License: Apache 2.0, which permits commercial use, modification and redistribution under its terms.
Maintainer: NVIDIA's GitHub organization, so release decisions sit with that project rather than with a commercial vendor.
Cost model: the maintainers describe garak as a free tool, and no paid edition appears in the supplied facts.
Latest release: v0.17.1 on 16 October 2026, per the releases page.
Packaging: installed from PyPI, with a Python 3.11 or newer requirement.

Confirm with NVIDIA or your legal team whether any hosted or commercial offering exists before assuming one, since none is described in the supplied facts. Check the release notes for each version you adopt.

Capability comparison from supplied facts

This table sets garak and PyRIT against the same questions: license, install path, interface, attack style, scoring, targets, reporting and multi-turn support. Read each row as a fact the reader can check against the project, and treat any item marked to confirm as an open question for a proof of concept. The main takeaway is that garak is built around a scanner run, while PyRIT is built around scripted attack logic that your team writes and owns.

Capability	garak (NVIDIA)	PyRIT (Microsoft)
License	Apache-2.0	MIT
Language and install path	Command-line tool installed from PyPI with pip; PyPI package requires Python 3.11 or newer	Python framework installed with pip; PyPI package supports Python 3.10 up to but not including 3.15, documentation recommends 3.13
Interface	Command-line scanner driven by probes, detectors, generators and harnesses	Python framework, the pyrit_scan command, CoPyRIT web GUI for human-led red teaming
Attack types	Static, dynamic and adaptive probes covering hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks	Multi-turn attack strategies including Crescendo, TAP and Skeleton Key, built from targets and converters
Scoring	Detectors judge each attempt; a probe is marked FAIL with a failure rate when any attempt produces undesirable output	Scorers use LLMs, Azure AI Content Safety or custom logic
Target coverage	Hugging Face Hub, Replicate, OpenAI API, AWS Bedrock, LiteLLM, llama.cpp gguf models and REST endpoints via YAML config	OpenAI, Azure, Anthropic, Google and Hugging Face models, custom HTTP or WebSocket endpoints, web apps driven through Playwright
Reporting	JSONL report of every attempt and its evaluation, plus a hit log of vulnerable attempts and a garak.log continued across runs	Memory layer records conversations, scores and results in SQLite or Azure SQL
Multi-turn support	Not stated in supplied facts; confirm whether a probe holds a multi-turn conversation in your version	Built in through named multi-turn strategies
Custom attack logic	Add or modify plugins across the five categories; confirm the effort for your own detectors	Written directly in Python using targets, converters, scorers and attacks

How a garak scan and a PyRIT attack run end to end

A garak run is a fixed pipeline: you choose probes, a generator and a harness, and the tool works through the attacks without you steering each one. The harness is the layer that matters most for planning. The default probewise harness runs each probe with its recommended detectors, so the detector set is largely decided by the probe you pick. The generator then connects to the target, whether that is a Hugging Face model, an OpenAI or AWS Bedrock model, a llama.cpp model or a REST endpoint described in a short YAML file. By default garak makes 5 generations per prompt, and a probe is marked FAIL with a failure rate when any attempt produces undesirable output. Before a run, confirm the generation count you want, since it changes both the runtime and the signal you get from a single probe.

Each garak run writes a JSONL report of every attempt and its evaluation, and appends to a garak.log file that continues across runs. The report is the artifact you hand to a reviewer, so check that it captures the prompt, output and detector result for each attempt in the form your assessment requires. The log is for debugging and is not a substitute for the report. Because the log carries over between runs, record which version you ran (the current release is v0.17.1, published on October 16, 2026) and which probes and generator settings were used, so a later reader can reproduce the result.

A PyRIT attack is a scripted loop you control. An attack strategy such as Crescendo, TAP or Skeleton Key sends a prompt to a target, and converters transform the prompt before it is sent, for example to change its encoding or phrasing. Scorers then evaluate each response, using an LLM, Azure AI Content Safety or custom logic, and the strategy uses that result to choose the next turn. Memory records the conversations, scores and results in SQLite or Azure SQL. Your team decides which scorer is authoritative, because a scorer that is too lenient will report a clean run that is not clean. Confirm the scorer, the memory backend and the Python version (the documentation recommends 3.13) before you start, and plan where the SQLite or Azure SQL database will be stored and who can read it.

Where each tool fits in CI and an in-house red team

garak runs as a single command, which makes it the easier of the two to put on a schedule. Installed from PyPI on Python 3.11 or newer, it can be called from a pipeline step with a generator config for the target and a fixed probe selection, and it writes a JSONL report of every attempt plus a hit log of failures. A release process can treat that output as an artifact: fail the build when a probe reports FAIL, archive the report with the model version, and diff reports between releases. The default of 5 generations per prompt means results are sampled, so a team should set the run parameters once, record them, and keep them constant if it wants comparable trends. Confirm with NVIDIA's user guide at docs.garak.ai which probes are suitable for a nightly job versus a pre-release gate, since run time depends on the probe set and the target.

PyRIT asks more of the engineers who own it, and that cost recurs. It installs with pip on Python 3.10 up to but not including 3.15, and the documentation recommends Python 3.13, so the pinned interpreter should be set in the pipeline image from the start. Because attacks are Python code built from targets, converters, scorers and multi-turn strategies, someone must write, review and version the attack scripts, choose the scorer for each harm category, and decide where the SQLite or Azure SQL memory lives. A team that will maintain custom attacks, such as a Crescendo sequence tuned to its own application's domain, gets far more out of PyRIT than a team that only needs a baseline. A team without that capacity should expect the scripts to drift as the target changes, and should plan an owner and a review cadence before adopting it.

The choice comes down to who will run the test next quarter. A scanner lowers the barrier for a single team that needs a repeatable baseline on a model endpoint, and its output maps cleanly to a release gate. A framework rewards an in-house red team that will invest in attack code, multi-turn scenarios and custom scoring, and it can run through the pyrit_scan command or the CoPyRIT web interface for human-led sessions. For a one-off assessment, garak usually gets to a useful result sooner. For an ongoing internal program, budget for the engineering time to maintain PyRIT's attack code as a recurring cost, and ask any external provider to state in the written scope who owns the scripts after the engagement ends.

What transfers and what is rebuilt when moving between them

garak and PyRIT solve overlapping problems with different building blocks, so most of the work in a move sits in the attack logic, the reports and the environment rather than in a shared format. The cards below separate what can be reused from what has to be recreated before a team can trust the results of either tool.

Attack and probe logic

garak probes and detectors are plugins with their own prompts and pass or fail rules, while PyRIT attacks are Python objects built from targets, converters and scorers. A garak probe cannot be imported into PyRIT, and a Crescendo or TAP attack has no direct garak equivalent. Before switching, list the probes your current results depend on and confirm which PyRIT attack or custom script will reproduce each one.

Reports do not map to memory

garak writes a JSONL file with one record per attempt and a hit log of attempts that found a vulnerability. PyRIT stores conversations, scores and results in its memory layer, backed by SQLite or Azure SQL. Historical garak reports will not load into PyRIT memory as-is, so keep them as archived artifacts and confirm that your audit or review process accepts that format.

Target configuration must be rewritten

garak's REST generator is configured with a short YAML file describing the endpoint. In PyRIT, the same endpoint is described as a target in Python, including any custom HTTP or WebSocket handling you need. Each authentication header, request template and response parser must be rebuilt and tested against the live system, since a mismatch can produce results that look clean but test nothing.

Scoring baselines and evidence retention

Pass and fail thresholds, scorer choices and the retention rules for results do not carry over. garak reports a failure rate per probe based on its default five generations per prompt, while PyRIT scorers use LLM judges, Azure AI Content Safety or custom logic. Record the baseline from your last accepted run under each tool, and agree with your assessors where the evidence is stored and for how long.

Team skills move unevenly

Engineers who know the garak CLI can run scans quickly, but writing a PyRIT attack requires Python, familiarity with its converter and scorer model, and an understanding of multi-turn state. Analysts who read garak's hit logs may need training before they can interpret PyRIT conversation histories. Map which people will own each tool before committing, and confirm whether your team can maintain custom attack code over time.

Python environment and dependency pinning

garak installs from PyPI and requires Python 3.11 or newer, while PyRIT supports Python 3.10 up to but not including 3.15 and recommends 3.13. Running both in one environment is likely to create conflicts, so each tool needs its own virtual environment or container. Pin exact versions for both tools in your CI images, since the garak release from October 16, 2026 and the PyRIT v1.1.0 release from September 4, 2026 may change behavior between runs.

Order of work when moving from garak to PyRIT or back

This sequence moves an assessment from one tool to the other without losing the evidence trail, so the team can show what changed, what was retested on the same model version and what the old results still prove. It is written for either direction and assumes both tools are installed in a controlled environment.

Inventory existing probes, detectors and attack scripts
Start by listing everything the current pipeline does. For garak, record which probes run, which detectors judge each probe, and whether the default probewise harness or a custom harness is used. For PyRIT, record each attack script, the converters and scorers it uses, and the memory database it writes to. Owners sign off the list, and the inventory is done when every test case maps to a named probe, detector, attack or scorer and an accountable person.
Map each target and its generator or connection
Identify every model endpoint the tests reach and how each is reached today. In garak, that is a generator such as a hosted API, a local llama.cpp model or a REST endpoint described in YAML. In PyRIT, it is a target such as an OpenAI, Azure, Anthropic, Google or Hugging Face model, or a custom HTTP or WebSocket endpoint. Record the model name, version string, authentication method and any system prompt or wrapper logic. The map is complete when each target has one row showing the exact model identifier and the matching connection on the other tool, with gaps flagged for the security team to resolve.
Rebuild the target connection in the new tool
Recreate each mapped connection in the destination tool using the same model identifier and the same request shape, so both tools test the same system. Keep credentials in the secrets store your team already uses and do not paste keys into configuration files. Confirm the new connection with a single benign request and a response check. This step is done when a test prompt returns the same kind of response from the old and new configuration, and the difference in wrapper logic, if any, is written down.
Confirm the model version before the baseline
Pin the model version, endpoint revision and any safety settings before running anything. Many hosted models change behind a stable name, so the team should confirm with the provider that the identifier resolves to the same underlying version for both runs. Record the tool versions as well, such as the garak release or the PyRIT release in use. The check is complete when the pinned values are written into the test record and the owner confirms they will not change during the comparison.
Run a baseline on the same model version
Run the rebuilt tests on the pinned model version with the same probe or attack set that the old pipeline used. Use the same number of attempts per prompt where the tool allows it, and keep the seed or randomness settings documented. Save the complete output from each tool separately. The baseline is done when both tools have produced a full report for the same target and the run logs show no connection errors that would hide failures.
Compare results with a documented method
Agree on the comparison rules before reading the results. Garak reports a JSONL file of every attempt with its evaluation and marks a probe as failed when any attempt produces undesirable output. PyRIT stores conversations and scores in its memory layer. Because the two tools judge outcomes differently, compare categories and examples rather than raw counts, and record each disagreement with the prompt, the response and the scorer or detector that decided it. The comparison is finished when a written method, a disagreement log and a short list of open findings have been reviewed by the security lead.
Retire the old pipeline in a controlled change
Once the baseline is accepted, switch scheduled jobs and CI steps to the new tool and disable the old jobs, without deleting their configuration. Name the change owner, the date of the switch and the rollback path. Confirm that the first scheduled run of the new pipeline completes and writes its report to the expected location. The step is done when the old jobs are disabled, the new jobs are running and the change record is approved.
Keep the old reports for audit
Archive the final reports, logs, configuration files, tool versions, model pins and the comparison record from both tools in a retained location with access controls. Garak's garak.log is continued across runs, so keep the full file, and keep PyRIT's memory database or exports under the same retention policy. Store a short index that maps each archived report to its date, target and tool version. The sequence is complete when an assessor can trace any past finding to the exact tool, model version and report that produced it.

What drives the running cost of each tool

Neither garak nor PyRIT carries a license fee in the supplied facts, so the running cost comes from people, model calls, storage, upgrades and optional support. The cards below show where each tool places that work and what to ask for before you commit.

Engineering time to build tests

garak gives you a scanner that runs probes against a target with recommended detectors, so most early effort goes into the generator YAML for your endpoint and choosing which probes match your application. PyRIT asks you to script attacks in Python using targets, converters, scorers and multi-turn strategies, so the build effort is larger but fits custom logic. Ask your team to estimate hours for both paths using a pilot target you already run.

Judge and target model calls

Every probe run sends prompts to a target model, and detectors or LLM-based scorers may send more calls to a judge model. garak defaults to 5 generations per prompt, so call volume scales with probe count and that setting. PyRIT's LLM-based scorers and multi-turn attacks add calls per conversation turn. Charges come from whichever provider you choose, so price each model you plan to call before a run.

Storage for reports and memory

garak writes a JSONL report of every attempt and evaluation, plus a garak.log file that continues across runs, so storage grows with each scan you keep. PyRIT stores conversations, scores and results in SQLite or Azure SQL, which means database sizing, backup and retention are your responsibility. Decide how long to keep raw attempts, who can read them, and where they sit under your data rules.

Upgrade effort across releases

garak has moved through several releases, and the latest in the supplied facts is v0.17.1 from October 16, 2026, so probe and detector names may change between versions. PyRIT is on v1.1.0 after v1.0.0, and its Python API is what your custom attacks depend on. Pin versions in your pipeline, keep a regression set of expected results, and budget time for each upgrade your team runs.

Internal staff for ongoing runs

Someone must own scheduled scans, triage the hit log or scorer output, and decide which failures block a release. With garak, that role is mostly reviewing reports and tuning probes. With PyRIT, it includes maintaining attack code and scoring logic, which usually needs a red teamer who can write Python. Estimate the share of an existing security or platform role these tasks will take each week.

External support and retainers

Neither tool carries a license fee in the supplied facts, so any vendor spend is for services. DeployOpen offers fixed-scope implementation, testing and ongoing support as retainers, and quotes follow from a written scope. Ask any provider to separate one-time setup, recurring retainer hours, and model or infrastructure charges you pay directly, so you can compare the two tools on the same basis.

Questions teams ask before choosing between garak and PyRIT

Can garak run multi-turn attacks?

garak's probes generate attacks from its plugin library, and its supplied documentation describes static, dynamic and adaptive probes, but it is not described as a multi-turn attack scripting framework. If your assessment depends on escalating a conversation over several turns, check the current probe catalog in the garak user guide before you commit. PyRIT is the stronger fit for that case because its documentation lists multi-turn strategies such as Crescendo, TAP and Skeleton Key. Confirm the exact behavior of any strategy against your own target before you rely on it in a report.

Is PyRIT suitable for a one-off assessment?

Yes, but it asks more of the operator than a scanner does. PyRIT offers a pyrit_scan command, a web GUI called CoPyRIT for human-led testing, and a Python framework for custom attacks, so a one-off run can be set up quickly if the target is simple. Its memory layer stores results in SQLite or Azure SQL, which means you own that data store after the engagement. For a single time-boxed check of one model endpoint, garak usually needs less scaffolding. The right choice depends on whether the assessment needs custom multi-turn logic.

What does an in-house red team need to run PyRIT?

An in-house team needs Python skills, the ability to write targets and scorers for its own systems, and a place to keep conversation memory. The PyRIT documentation recommends Python 3.13, and the PyPI package supports Python 3.10 up to but not including 3.15. Targets can include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP or WebSocket endpoints, and web apps driven through Playwright. Scorers can use LLMs, Azure AI Content Safety or custom logic, so the team must decide how judgments are validated. Ask the team to confirm these skills before selecting PyRIT.

Does either tool replace a formal audit?

No. Neither garak nor PyRIT produces an audit report, attestation or certification. They generate attack attempts and evidence that a team can use in its own risk review, and an independent auditor or certification body decides whether controls meet a standard. DeployOpen prepares systems, controls and evidence for compliance assessments but does not issue audit reports, attestations or certifications. Ask any assessor which evidence they will accept from automated red team output before you design the test plan.

How long does setup take for garak or PyRIT?

Setup time depends on target complexity and team skill, so no duration can be given here. A public model behind a supported API needs little more than installation and credentials, while a custom REST endpoint, an authenticated web app or a multi-turn scorer requires configuration and validation work. garak's REST generator uses a short YAML file to describe an endpoint, and PyRIT requires Python code for custom targets. Ask the provider for a written scope that lists each target, each authentication path and each detector or scorer you need before agreeing on timing.

What does outside help cover, and what drives the cost?

Outside help is usually paid for the scope agreed in writing before work starts: target setup, probe or attack selection, custom scoring, running the tests, and writing up findings. Tool licenses are not the main cost, since garak is Apache 2.0 and PyRIT is MIT licensed, so the fees are for people and time. Costs rise with the number of targets, the depth of multi-turn testing, the need for custom detectors, and the evidence format you require. DeployOpen does not publish a price list, so ask for a quote that separates one-time setup, the test run and any recurring retesting.

Can we run both garak and PyRIT?

Yes. garak gives you a broad scan of a model endpoint with probes and detectors, and PyRIT gives you scripted, multi-turn attacks with its own scorers and memory. Running both can cover different failure modes, but results will use different formats and judgments, so decide in advance how you will merge and deduplicate findings. garak writes a JSONL report of every attempt plus a hit log, and PyRIT stores conversations in its memory layer. Assign one owner to reconcile the two outputs before the report leaves the team.

Official sources for garak and PyRIT

Start with the garak README on the NVIDIA/garak repository before you commit to a scanner. It sets out the probe, detector, generator and harness model that this comparison depends on, and it names the Apache 2.0 license you will need to clear with legal. Then check the garak releases page for the current version and its release notes, since probe coverage and generator support change between versions. The garak user guide covers the command-line options and the generator configuration for a REST endpoint, which is where most of the setup effort sits. The garak package on PyPI confirms the Python 3.11 minimum that your build image must meet.

For PyRIT, the current source of truth is the microsoft/PyRIT repository, which carries the MIT license and the active development history. Do not rely on the older Azure/PyRIT repository for anything current. It is archived and read-only, so its issues and code will not reflect fixes or new attack strategies. Use the PyRIT releases page to pin a version in your environment, and confirm which release your red team tested against before you record results. The PyRIT documentation for version 1.1.0 describes targets, converters, scorers, attacks and memory, and it is the reference for the multi-turn and scoring work that separates PyRIT from garak. The pyrit package on PyPI states the supported Python range, which you should compare with your own build standard.

Before the first run, record the exact version, license and Python runtime for each tool in your assessment file, and confirm them against these pages on the day you install. Ask your security engineers to store the garak JSONL report and hit log, and the PyRIT memory database, under the same change record so that results can be reproduced. If your team plans to fork either project for internal use, check the license text in the repository and get your own legal review, because this page does not assess license obligations beyond the names stated in the sources.

How DeployOpen helps choose, implement or migrate AI red teaming tools

DeployOpen plans, implements, documents and supports self-hosted open-source platforms inside your own infrastructure, and runs security testing that includes AI systems and agents. For garak vs PyRIT, we start from which tool fits your assessment model, then build and hand over the setup in your environment with written scope.

How we scope

We confirm whether the work is a one-off model assessment, a recurring in-house red team program, or both
We list the model endpoints, generators and REST targets that garak or PyRIT must reach
We agree in writing which probes, attack strategies, scorers and report formats are in scope

How we build

We install garak from PyPI or PyRIT from PyPI in your environment, pinned to versions you approve
We configure generators, REST endpoint definitions and PyRIT memory storage against your own targets
We run agreed probe sets or multi-turn attack scripts and review detector or scorer results with your team

How we hand over

We deliver runbooks covering configuration, run procedures, JSONL reports and PyRIT memory outputs
We document which findings need human review and how results map to your internal risk records
We walk your security engineers through the setup so they can rerun and extend the tests

How we support

Ongoing support, upgrades and continuous testing are available as retainers after handover
Retainers can cover tracking new garak and PyRIT releases and re-running agreed test suites
Scope changes are priced and agreed in writing before any additional work starts