AI red teaming as ISO 42001 evidence, what auditors accept

Does AI red teaming count as ISO 42001 evidence?

AI red teaming and security testing can serve as evidence for A.6.2.4 verification and validation and for related controls, but the organization owns its controls, and an independent certification body decides whether the evidence is accepted. A customer questionnaire or contract that asks for AI security testing under ISO 42001 is really asking three things: whether testing is defined as part of verification and validation, whether the results are recorded, and whether they lead to decisions. A.6.2.4 asks the organization to define and document verification and validation measures for the AI system and to set the criteria for using them. Red teaming, model scanning and other security tests are only evidence once they are tied to those documented measures and acceptance criteria. A test run with no stated criteria shows activity, not control.

The answer also depends on scope and the statement of applicability. Under clause 6.1.3, the organization justifies which Annex A controls it includes and which it excludes, so the first check is whether A.6.2.4, A.6.2.6 (ongoing operation, including monitoring) and A.6.2.8 (event log records) are included and why. Ask the team to confirm which AI systems are in scope, which test types apply to each, and who signs off on criteria. Testing that covers only a chatbot front end will not support a claim about a model pipeline the organization also operates. Confirm these points with your certification body early, because its reading of scope governs what it will accept.

Results only count when they feed clause 8. Under 8.2 and 8.3, AI risk assessment and risk treatment, a finding should change a risk rating, a treatment decision, an accepted residual risk or a control update, and each of those changes should be recorded. Keep the chain visible: test plan, raw output, finding, risk entry, treatment decision and retest. Findings should also reach management review, because that is where accepted residual risk is owned. For the threat framing behind these tests, ISO/IEC 27090:2026 gives guidance on threats such as data poisoning and model theft. Treat it as a source of threat categories for your risk register, not as a checklist that testing must satisfy.

Which testing activities produce which evidence

Each testing activity below produces a different kind of record, and each record supports a different Annex A control. Map the activity to its evidence before you run it, so the test plan, criteria and results can be shown to an assessor without reconstruction.

Pre-release red teaming

Before an AI system is released, a red team tests it against a written plan that names the threat scenarios, the test sets, and the pass or fail criteria. The test plan and criteria are the evidence that matter most here, because A.6.2.4 asks the organization to define and document verification and validation measures and the criteria for using them. Keep the plan, the criteria version and the signed results together, and record which findings were accepted, fixed or deferred.

Model vulnerability scanning with garak

garak probes language models for hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks. Its run produces a JSONL report of every attempt and evaluation, plus a hit log of attempts that found a vulnerability. Store both files with the model version and probe configuration used, since the attempt report is the record an assessor can sample against your stated test criteria.

Prompt and output testing

Prompt and output testing checks whether the system holds its intended behavior when inputs are adversarial or unusual, including injected instructions in retrieved content. Log the prompt set, the expected behavior for each case, and the observed output, so the results trace back to the test criteria. Treat repeated failures in one category as a risk treatment input under clause 8.3, not as isolated bugs.

Application and API testing

The AI model sits inside an application, and the application has its own attack surface: authentication, authorization, API endpoints and data stores. Testing these produces a scope statement, a findings register with severity, and retest records showing closure. This evidence supports the operational controls for the surrounding system, and confirming the scope with your assessor early prevents gaps about which components the tests covered.

Continuous testing after release

Once the system is live, testing has to repeat whenever the model, prompts, retrieval sources or configuration change. A.6.2.6 asks the organization to define what ongoing operation requires, including performance monitoring, repairs and updates. Record each retest with the change that triggered it, so the operation record shows that the system was tested against its current version rather than the one approved at launch.

Log review and remediation records

Event logs from testing and production are only useful if someone reviews them and acts on what they show. A.6.2.8 asks the organization to decide at which life cycle phases event logs are enabled, and at minimum when the system is in use. Keep the log configuration, review dates, reviewer and the remediation tickets that link each finding back to a risk treatment decision and to management review.

How ISO/IEC 27090 frames the threats your tests should cover

ISO/IEC 27090:2026 gives guidance on identifying, detecting and reducing security threats that are specific to AI systems across their life cycle. Its examples include data poisoning (corrupting training or fine-tuning data so the model behaves wrongly) and model theft (extracting a model's behavior or parameters). Use these threat categories as the scaffold for your test plan. For each category, write down which life cycle phase it affects, which system components are exposed (training data pipeline, model artifact store, inference endpoint, prompt handling, agent tool access), and which test activity will probe it. A test that cannot be traced to a threat category is hard to defend as evidence for the verification and validation objective in A.6.2.4.

ISO/IEC 27090 is guidance, not a certifiable requirement. An auditor will not check whether you followed its clauses line by line. What an auditor will check is whether your own documented threat picture is reasonable for the system, whether the tests cover the threats you chose to treat, and whether the results flow into risk treatment under clause 8.3. Treat the standard as a vocabulary and checklist that helps you show your reasoning, and record in the statement of applicability how you used it.

Confirm the threat mapping with your assessor before you rely on it. Ask which threat categories they expect to see for your system type, whether they accept a mapping that combines ISO/IEC 27090 with your own threat model, and how they want exclusions justified. An illustrative example: a customer-facing assistant that calls internal tools may need prompt injection and tool misuse tests mapped to the agent components, while a model you only fine-tune on licensed data may need poisoning and model extraction tests mapped to the training pipeline and model registry. Your assessor decides what is sufficient for your scope.

How test results feed risk treatment and management review

This sequence turns raw red teaming and scanning output into a traceable chain from test criteria to risk register entries, treatment decisions, retests and management review, so an assessor can follow each finding to a documented outcome.

Scope the AI system and its statement of applicability
Start by naming the AI system, its intended purpose and its life cycle stage, then check which Annex A controls in the statement of applicability (clause 6.1.3) cover testing, monitoring and event logging for it. The owner of the AI system and the security function agree the boundary, including models, prompts, tools, data pipelines and any third-party components. The output is a short scope record that names the controls the testing is meant to evidence, so each later test can be traced back to a stated requirement.
Define pass criteria under A.6.2.4
Document the verification and validation measures for the system and the criteria that decide whether a result passes, fails or needs review. Criteria can reference the test sets, metrics and tools used, and the acceptance thresholds the business has approved, so the reasoning is written down before anyone sees results. The NIST crosswalk maps this work to the MEASURE subcategories for test documentation and validity, which helps you show the criteria are deliberate. The output is an approved criteria document with a version and an approver.
Run the tests and record every attempt
Execute the agreed tests, such as garak probes for prompt injection, data leakage and jailbreaks, against the scoped system and retain the tool configuration, model version and test date. Keep the full JSONL report of attempts and evaluations, not only the hit log of successful findings, because an assessor will ask how many attempts were made and what did not succeed. Record who ran each test and under which account. The output is a versioned evidence package that can be reproduced.
Log findings into the AI risk register
Compare each result with the pass criteria and open a risk register entry under the AI risk assessment in clause 8.2 for every failed or inconclusive result. Each entry should state the affected system, the threat category (for example, from the framing in ISO/IEC 27090), the likelihood and impact estimate, the owner and a link to the raw evidence. Findings that stay within criteria are still recorded with their test references. The output is a register with no unassigned items.
Decide risk treatment and document the choice
Under the AI risk treatment process in clause 8.3, the risk owner selects a treatment for each entry, such as modifying the system, adding guardrails or output screening, restricting the intended use, transferring the risk through a contract, or accepting it with a documented reason. Where a treatment changes a control, update the statement of applicability and any affected impact assessment under clause 8.4. The output is a treatment plan with the chosen option, the approver and the residual risk stated.
Retest and close the loop
After a treatment is implemented, rerun the same tests against the changed system using the original criteria, plus any new cases that the change could affect, and record the new results beside the originals. Close the register entry only when the retest meets the criteria or when the residual risk has been formally accepted. Keep the before and after results linked so the change history is visible. The output is a closed or re-opened register entry with retest evidence attached.
Report results to management review
Prepare a summary for management review covering test coverage, open and closed findings, accepted residual risks, recurring failure themes and any test criteria that were changed, and confirm the outcomes against the AI policy and objectives. Management decides on resourcing, scope changes and whether the testing cadence and criteria still fit the system. Record the decisions and actions in the review minutes. The output is signed minutes with assigned actions, which is the evidence that the results were used at the level the standard expects.

Event logs and monitoring: what testing can and cannot show

Red teaming and scanning tools generate their own records, and those records only become useful evidence if the production logs can show the same activity. A probe that sends a prompt injection attempt to a model endpoint should leave a trace that links the request, the model version, the response and the calling identity. Without those fields, a later investigation cannot tell a test run from a real attack. ISO/IEC FDIS 24970 covers AI system logging, so use it as a reference point when you define the fields, then check the design against your own threat model and retention rules.

Request identity: Record the caller, session, API key or service account, and the source address, so test traffic can be separated from production users by a documented tag rather than by guesswork.
Model and configuration version: Capture the model identifier, system prompt version, guardrail policy version and tool configuration for each call, so a finding can be tied to the exact system that produced it.
Input and output handling: Log enough of each prompt and response to reproduce a finding, and apply redaction rules for personal data before storage.
Security events: Flag blocked prompts, jailbreak detections, tool-call denials and unusual output lengths as distinct event types that a monitoring rule can alert on.
Test run markers: Tag each red team or garak run with an identifier that appears in both the scanner report and the platform logs.

Before you rely on these fields, confirm with your logging platform's documentation that each one is actually captured by default, because many AI gateways and tracing tools log only part of the request.

Which testing option fits your AI system

This table compares four ways to produce security testing evidence for ISO/IEC 42001 controls such as A.6.2.4 and A.6.2.8: in-house testing, an external red team, a platform-based scan with open-source tools, and a combined model. Read each row for what the option proves, where it falls short, and what the auditor will ask to see.

Testing option	What it proves	Where it falls short	What the auditor will ask for
In-house testing	Your team tested the AI system against defined criteria, and the results are repeatable across releases	Testers may share the design assumptions of the system they test, and results are hard to show as independent	Documented test plan, criteria for use, test records, and who signed off the results
External red team	An independent party tested the system and reported findings you can compare against your own risk register	Point-in-time evidence only, and findings often lack a link to the controls they affect unless you specify that scope	Statement of work, scope and rules of engagement, final report, and proof that findings entered risk treatment
Platform-based scan with open-source tools	Repeatable probes ran against each model version, with machine-readable records of every attempt and any vulnerability hit	Automated probes miss context-specific misuse, and the tool's output needs human review before it counts as a verified result	Tool versions, configuration, run logs or JSONL reports, and records showing how a person reviewed the hits
Combined model	Automated scans run continuously, and periodic human testing covers the threats the scans cannot reach	Requires coordination between teams, and gaps appear when the two schedules or scopes drift apart	One test calendar covering both, linked findings, retest records, and evidence that changes to the system triggered testing

What weakens security testing as ISO 42001 evidence

Security testing produces evidence only when the organization can show what it tested, what counted as a pass, and what changed afterward. The pitfalls below are the ones that most often leave a test report with no clear link to a control, a risk or a decision.

Pass criteria never defined

A red team report that lists attack categories and a count of successful prompts does not show whether the system is acceptable. Under A.6.2.4, the organization must define verification and validation measures and the criteria for using them before testing starts. Check that each test suite names its metrics, thresholds and the owner who decides a finding is acceptable or must be fixed. Write the threshold down before the first run.

Tests not linked to the statement of applicability

Clause 6.1.3 requires a statement of applicability that justifies each included or excluded Annex A control. If security testing runs but the SoA does not point to it, an assessor cannot tell which control the test supports or why a control was excluded. Map each test campaign to the A.6.2.4 and A.6.2.8 entries and to the clause 8 operational records it feeds.

Findings missing from the risk register

Clause 8.2 and 8.3 expect AI risks to be assessed and treated. A vulnerability found by a scanner or red team that never reaches the risk register has no recorded treatment decision, so the test shows a weakness without a response. For each finding, record the risk, the accepted or mitigated status, the owner and the retest date. Confirm the register is the place management reviews, not a side spreadsheet.

Third-party models left out of scope

Many AI systems call a hosted model, an API or a vendor-supplied component that the organization does not train. If the test scope covers only the wrapper application, the evidence says nothing about prompt injection or data leakage through the model. Define the scope to include each third-party model version in use, and record what the supplier contract and documentation allow you to test. Coordinate with A.10 supplier controls for anything you cannot test directly.

Tests not rerun after model updates

A passing test against one model version says little about the next. Under A.6.2.6, operation covers updates and repairs, so changes to the model, prompts, retrieval data or guardrails should trigger a defined retest. Ask the team to state which changes require retesting, and keep the test date and model identifier in each record so an assessor can match results to the deployed version.

Reports not retained or traceable

Raw outputs from tools such as garak, including its JSONL attempt log and hit log, are only useful if they survive. Teams often overwrite runs, delete logs after a release or store results in a personal workspace. Set a retention period in the organization's own records policy, store reports with the model version and configuration, and make sure the event logs required by A.6.2.8 can be linked back to the test run.

Who owns the testing, the evidence and the decision

The organization owns its AI controls, its risk decisions and the conclusion that its AI management system meets ISO/IEC 42001. Red teaming and security testing produce findings, but the organization decides which findings are accepted, which are treated, and which residual risks it carries. An implementation partner such as DeployOpen can plan the test scope, run the security testing, prepare the evidence and document the results against the Annex A controls the organization has chosen in its statement of applicability. That work prepares the organization for assessment; it does not replace the organization's own accountability for the controls.

Only an independent certification body decides whether the evidence meets ISO/IEC 42001. ISO does not certify organizations. External certification bodies do, and accreditation of those bodies is voluntary, so the reader should confirm whether a candidate body holds accreditation for AI management system certification from its national accreditation body. ISO/IEC 42006:2025 sets additional requirements, on top of ISO/IEC 17021-1, for bodies certifying AI management systems, and it is the right reference when asking a certification body how it qualifies its auditors and assesses AI-specific evidence.

Before engaging anyone, agree in writing who signs off test plans, who approves exceptions to findings, and who holds the test reports as records. Ask the certification body early whether it will accept test reports prepared by a third party as evidence, and under what conditions, so the testing scope is designed around what the auditor will actually sample. Keep the separation clear in contracts: the partner delivers tested systems and documented results, the organization owns the decisions, and the auditor issues the outcome.

Questions buyers ask about AI security testing for ISO 42001

Does passing AI red team tests satisfy ISO 42001?

No. Test results are evidence that a control operates, not proof that the organization meets ISO/IEC 42001 on its own. The organization must define its verification and validation measures under A.6.2.4, document the criteria for using them, and show that results were reviewed and acted on. An independent certification body decides whether that body of evidence meets the standard. A clean test report with no linked risk decisions, owners and follow-up will usually fail to persuade an auditor, while a modest test program tied to a documented risk assessment and treatment record can be more persuasive.

What effort does AI security testing add, and what drives it?

Testing adds effort in three places: scoping the systems and use cases, running the tests, and turning findings into risk treatment records. The main drivers are the number of AI systems in scope, whether they are built in-house or bought, how many interfaces and data sources they touch, and whether the team must test agents with tool access as well as chat models. Open-source scanners such as garak reduce tooling cost but still require people to design test sets, interpret reports and document decisions. When you request a quote, ask the partner to break the work into scoping, test design, execution, reporting and remediation support, and to state which items are one-time and which recur.

How often should AI systems be retested for ISO 42001?

ISO/IEC 42001 does not set a fixed retest interval, so the frequency is a decision the organization must justify and document. Retesting is usually triggered by a change to the model, its prompts or retrieval sources, its tools or permissions, its intended purpose, or a new threat identified in the risk assessment. Operational monitoring under A.6.2.6 and event logging under A.6.2.8 help show when a change occurred. Confirm the expected cadence and triggers with your certification body before the first stage 2 audit, and record the rationale in your statement of applicability and internal procedures.

Are open-source tools like garak acceptable evidence for ISO 42001?

The standard does not name tools, so acceptability depends on how the output is used and whether the certification body agrees. garak is released under the Apache-2.0 license and probes language models for issues such as prompt injection, data leakage, toxicity and jailbreaks. It writes a JSONL report of every attempt and evaluation plus a hit log of attempts that found a vulnerability, which gives auditors raw records to sample. Document the version, configuration, probe selection, test sets and who reviewed the results. Ask your certification body whether it accepts open-source tool output and what additional traceability it expects, since that position is not settled by the standard itself.

What must our team supply to make security testing usable as evidence?

Your team owns the decisions the tests inform, so it must supply the AI system inventory, intended purpose, data and tool access details, and named owners for each system. It also needs to define acceptance criteria, approve the test scope, keep test configurations under version control, and record how findings enter the risk register and treatment plan. Management review should cover open findings and their status. An implementation partner can help design the test plan, run tests in your environment and prepare the evidence pack, but it cannot accept risk on your behalf or sign off your control operation.

Can DeployOpen issue an ISO 42001 certificate after testing?

No. DeployOpen does not issue audit reports, attestations or certificates. ISO does not certify organizations either. Accredited certification bodies perform the independent assessment, and their accreditation is voluntary, so confirm that the body you choose holds the accreditation your customers expect. DeployOpen can run security testing, help prepare systems and evidence, and document controls in your environment, and it works on fixed-scope engagements agreed in writing before work starts. Keep the implementer and the certification body independent so the auditor can assess your controls without relying on work it designed.

How DeployOpen scopes testing and prepares ISO 42001 evidence

DeployOpen treats AI red teaming and security testing as a source of evidence for ISO/IEC 42001 verification, validation and monitoring. We scope the testing in writing, run it inside your own infrastructure, and hand over documented results that your organization owns and your independent auditor can review.

How we scope

Agree in writing which AI systems, models, agents and environments are in the test scope before any work starts.
Map each test to the Annex A controls it supports, such as verification and validation and event logging, so evidence has a clear purpose.
Confirm the testing rules, permitted attack techniques and data handling with your security and AI governance owners before testing begins.

How we build

Set up open-source scanning and red teaming tools, such as garak, inside your own cloud accounts or data centers.
Define test sets, metrics and pass criteria so results can be repeated and compared across model versions.
Configure logging of test runs so every attempt and evaluation is kept as traceable records for later review.

How we hand over

Deliver the written test plan, raw tool outputs, findings with severity and reproduction steps, and a summary mapped to the relevant controls.
Document which findings feed your AI risk assessment, risk treatment plan and management review records.
Explain to your team which evidence an independent auditor or certification body is likely to ask for, while noting that DeployOpen does not issue audit reports or certificates.

How we support

Fixed-scope engagements have agreed deliverables and timelines, so you know what evidence you receive and when.
Ongoing support, retesting after model or system changes, and continuous testing are available as a retainer.
Ask any provider to confirm which operational monitoring and repair activities remain with your team after handover.

AI red teaming and security testing as ISO 42001 evidence