Why AI red teaming is different from a checklist test
A launch date, a customer security questionnaire, an insurer's application or a due diligence request often asks for adversarial testing of an AI system, and the buyer then has to decide what that means. AI red teaming is objective-based adversarial testing: testers pursue specific harm scenarios that the client has agreed on, using techniques from the MITRE ATLAS knowledge base, rather than working through a fixed list of checks. A checklist test asks whether a known weakness is present. A red team asks whether an attacker could make the system do something the business cannot accept, and then tries to find the path that gets there.
The starting point is the list of harms, not the tool. Before testing, the client and the testers agree on scenarios such as a customer-facing assistant revealing another customer's records, an agent sending money or data to an attacker-controlled address, or a model output that triggers an action in a downstream system. Each scenario is mapped to the relevant weakness categories. For LLM applications, those include prompt injection, sensitive information disclosure and excessive agency in the OWASP Top 10 for LLM Applications 2026. For agents that call tools, the OWASP Top 10 for Agentic Applications for 2026 adds goal hijack, tool misuse and identity and privilege abuse. Those categories tell the testers where to look. The scenarios tell them what success means.
For the buyer, the practical decisions come before any attack work starts. Confirm which harm scenarios the report must address, who approves the rules of engagement (including which data, tools and production environments are in bounds), and whether the test runs against production or a representative staging system. Ask the provider how each finding will be tied to a scenario, a MITRE ATLAS technique and the evidence needed to reproduce it. If a provider offers only a scan result or a list of generic probes, the output will not answer the question the customer, insurer or regulator is asking.
What an AI red teaming scope includes
Scope defines what the engagement tests, against which harms, and under what limits. Each card below is a decision that must be settled and agreed in writing before any testing starts, so the findings can be traced back to a system, a harm and an owner.
The system and its components
Name every component in the test: the model endpoint, the prompt and system layer, retrieval sources, tools and agents, and any MCP servers. Red teaming an endpoint alone misses the places where an attack actually lands, such as a document in a retrieval index or a tool that runs with broad permissions. Ask the provider to list each component in the scope document and state which ones are in and out.
Harm scenarios and their owners
Agree a short list of concrete harm scenarios, such as a customer receiving another customer's records or an agent issuing a refund it should not. Each scenario needs a named business owner who can judge whether an outcome counts as a failure. Without owners, findings stay technical and nobody can prioritize the fix or accept the residual risk.
Threat actors and attack objectives
Decide which adversaries the test models, such as an anonymous user of a public chatbot, a malicious insider with a low-privilege account, or an attacker who controls a document the system ingests. Each actor maps to techniques in MITRE ATLAS and to a realistic set of starting access. Confirm with the client which actors are in scope so the test answers the questions the risk owner actually has.
Test accounts and roles
List the accounts the testers will use, including standard users, privileged users, and any tenant or role boundaries that the system is meant to enforce. Access control failures are among the most common findings in AI systems, so the test needs accounts that sit on both sides of each boundary. Confirm who issues the accounts, how they are revoked at the end, and what logging applies to them.
Data handling and environments
Set the rules for test data, including whether testers may see real customer records, whether prompts and outputs are retained, and where logs are stored. Decide whether testing runs in staging, a production-like copy or production itself, since production testing carries more risk to live users. Ask for a written statement of where test artifacts are kept and when they are deleted.
Rules of engagement and stop conditions
Define the permitted techniques, the hours, the contacts for escalation, and the conditions under which testing pauses or stops, such as service degradation, exposure of real personal data or unexpected action against a live system. Agree who can call a stop and how quickly testers must respond. These rules protect the client's systems and make the test defensible to the client's own change and incident processes.
How an AI red teaming engagement runs, phase by phase
The sequence turns a broad concern about an AI system into agreed harm scenarios, tested adversarial paths and findings the owner can fix and prove fixed.
Agree scope and harm scenarios
The client, DeployOpen's testers and, where relevant, the application owner meet in a workshop to define the system under test, its tools and data, the trust boundaries, the users and the harms that matter most, such as leaking customer records, an agent taking an unauthorized payment action or unsafe content reaching end users. Rules of engagement cover permitted test accounts, environments, data handling and stop conditions, and are written down before work starts. The step closes when the client signs off a scenario list and scope document.
Map threats to MITRE ATLAS
Testers build a threat model for the agreed scenarios and map each plausible path to tactics and techniques in MITRE ATLAS, using the v2026.09 data release dated 15 September 2026 as the reference. Entries such as LLM Prompt Injection: Indirect (AML.T0051.001) and LLM Jailbreak (AML.T0054) are matched to the components they would touch, and the team identifies which OWASP GenAI Top 10 categories apply. The output is a test plan that lists each scenario, its ATLAS references and the expected evidence of success or failure. The client reviews and approves the plan before any adversarial test runs.
Frame risks with NIST AI 100-2 and AI 600-1
Before testing, testers record how each scenario maps to the adversarial machine learning taxonomy in NIST AI 100-2 E2025, which covers evasion, poisoning and privacy attacks, and to the risk categories in NIST AI 600-1, the Generative AI Profile published on 26 July 2024. This gives the report a shared vocabulary that risk committees and auditors can follow. The step is complete when every scenario has a stated risk category and a named owner on the client side who accepts the risk framing.
Run controlled adversarial tests
Testers execute the approved scenarios against the system in the agreed environment, typically a staging copy with test data and monitored logging, and adapt their approach based on what the system reveals. They record every attempt, input, output and tool call so that each result can be reproduced. Testing stops if a scenario would touch live customer data, affect production availability or exceed the agreed rules of engagement, and the client is told at once. The step closes with a complete attempt log and a list of observed behaviors for analysis.
Analyze findings and rate impact
Testers group observed behaviors into findings, confirm each one by reproducing it, and assess impact against the harm scenarios agreed in the first step. Guardrails that held are recorded alongside failures, so the client sees which controls worked as well as which did not. Each finding is linked to its ATLAS technique, OWASP category and, where relevant, the MCP or agentic risk it corresponds to. The step closes when every finding has a reproduction record, an impact statement and a suggested remediation the client's engineers can act on.
Deliver the red team report
The report sets out the scope, the methodology, the threat model, each finding with its evidence and remediation, and a summary of which harm scenarios were tested and what happened. It states clearly what was not tested and the limits of the test, since a red team exercise samples adversarial behavior rather than proving its absence. A readout meeting lets engineering, security and product owners agree on remediation priorities. The step closes with a delivered report and an agreed remediation plan with owners assigned.
Retest fixes and close findings
After the client applies fixes, testers rerun the reproduction steps for each closed finding and the related scenarios, then record whether the issue is resolved, partly resolved or still open. Retest results are added to the report as a dated addendum so the change history is traceable for auditors and customers. Because model, prompt and tool changes can reopen old paths, the client should confirm in advance whether a retest is in the agreed scope or part of a retainer. The step closes with a retest record that states the status of every finding.
Harm scenarios and ATLAS techniques tested in practice
This table maps the weakness categories an AI red team typically pursues against a target system, showing what each looks like in production and why it matters to the buyer. Read each row as an objective the test works toward, with the ATLAS or OWASP reference that anchors it. The main takeaway is that red teaming scenarios should be agreed against the harms your business and regulators care about, not run as a generic checklist.
| Weakness category | What it looks like in a target system | Why it matters to the buyer |
|---|---|---|
| Prompt injection, direct and indirect (AML.T0051.001) | Untrusted input, including documents, web pages or tool outputs the model reads, changes how the model behaves; testers check whether instructions in retrieved content override the system's intended behavior. | Indirect injection through content the system ingests is the most common route to unintended actions, so the test should cover every input channel, not only the chat box. |
| Jailbreak (AML.T0054) | Attempts to get the model to ignore its safety or policy constraints through role framing, multi-turn pressure or encoded requests; testers record which guardrails held and which failed. | A jailbroken model can produce content your policy prohibits, which creates brand, legal and conduct exposure even when no data is taken. |
| Sensitive information disclosure and hidden context exposure | The model or its surrounding services reveal personal data, credentials, system prompt content or internal instructions to users who should not see them. | Disclosed prompts, keys or customer records trigger notification duties and undermine any claim that the system's confidential configuration is protected. |
| Excessive agency | An agent holds more tools, permissions or autonomy than its task needs, so a manipulated request can trigger writes, payments or messages the owner never approved. | Scope of granted permissions determines the worst outcome of any successful attack, so this row often drives the most important design fix. |
| Tool misuse and goal hijack | Attackers steer an agent away from its assigned goal and use legitimate tools in harmful sequences, tested in a controlled environment with logged actions. | Hijacked agents act with the user's or service's authority, so the test should show whether approval gates and action logging catch the misuse. |
| Data and model poisoning | Tampered training, fine-tuning, retrieval or memory data alters later outputs; reviewed through data pipelines and sampled tests rather than live exploitation. | Poisoned sources can persist silently across sessions, so buyers need to know which data sources are trusted and how changes are tracked. |
| Misconfigured or publicly exposed AI services (AML.T0132) | Model endpoints, vector stores, notebooks or admin consoles are reachable from places they should not be, or lack authentication and network restriction. | Exposure is often the cheapest path to compromise, and the finding is usually fixable in configuration within the client's own environment. |
| Discovery of agent runtime capabilities (AML.T0133) | An adversary or tester maps which tools, credentials, network paths and runtime functions an agent can reach once it is running. | Capability discovery reveals the real blast radius of an agent, so the test should confirm whether the reachable set matches the approved design. |
What the EU AI Act's adversarial testing duties mean for buyers
The EU AI Act creates two distinct adversarial testing duties, and they fall on different parties. Article 15(5) requires high-risk AI systems to be resilient against attempts to exploit their vulnerabilities, including data poisoning, model poisoning, adversarial examples, model evasion and confidentiality attacks. Article 55(1) requires providers of general-purpose AI models with systemic risk to perform documented adversarial testing. Read the regulation itself in the official text of *Regulation (EU) 2024/1689* on EUR-Lex before scoping anything, because the obligations are written against defined roles and categories rather than against testing methods.
Before you commission a red team, confirm with counsel which category your system falls into. A system that is high-risk under the Act, a general-purpose model with systemic risk, or neither will lead to different test scopes and different evidence. Ask counsel to state in writing which provisions apply to your system, which role you hold (provider, deployer, importer or distributor) and whether the threat categories in Article 15(5) map to your use case. If you build on a third-party model, also confirm which party carries the Article 55(1) duty and what documentation that party will share with you.
DeployOpen can plan and run objective-based adversarial tests against your AI system and document the methods, scenarios, findings and retests in a form your compliance and legal teams can use. DeployOpen does not determine whether the Act applies to you, does not issue legal conclusions, and does not issue certifications or attestations of conformity. Ask your counsel or notified body to decide how the test evidence fits your obligations. In the scoping call, ask the test provider to map each agreed harm scenario to the threat categories your counsel has confirmed, so the report answers the question your auditor or regulator will ask.
What drives AI red teaming cost and duration
This table lists the scope factors that move the effort of an AI red teaming engagement, why each one matters, and how to keep it proportionate. Read it before requesting a quote, because a quote follows from the written scope rather than from any fixed price list. The main takeaway is that the number of AI components, harm scenarios and agent privileges usually counts for more than the size of the underlying application.
| Driver | Why it changes effort | How to scope efficiently |
|---|---|---|
| AI components in scope | A single model endpoint can be tested through one interface. A RAG pipeline, agents and MCP servers add ingestion paths, retrieval boundaries, tool connections and token flows, each needing its own adversarial scenarios. | Name each component and its trust boundary in the request. Exclude components that are tested elsewhere, such as the MCP server under its own engagement. |
| Number of harm scenarios | Each agreed harm scenario, such as data exfiltration, unauthorized actions or harmful output, needs objectives, attack paths and evidence. Scenario count is the most direct input to effort. | Limit the list to harms the business and its regulator or customer care about most. Rank scenarios so the lower-priority ones can be deferred or dropped in the scope. |
| Agent tool count and privilege levels | Every tool an agent can call and every privilege it holds widens the set of goal hijack and tool misuse paths. High-privilege write tools need more careful, controlled test design. | Group tools by consequence (read only, reversible write, irreversible action). Test the highest-consequence group in depth and sample the rest. |
| User roles and tenants | Each role and tenant boundary creates authorization and cross-tenant leakage tests, especially for RAG retrieval and memory. Tests must be run with separate accounts and data. | Provide a short matrix of roles, tenants and the data each should see. Restrict testing to the boundaries that carry real isolation requirements. |
| Environment access | A staging environment that matches production lets testers iterate freely. A production replica needs extra data handling, change control and sometimes a longer setup period. Testing against live production usually needs strict controls. | Confirm whether a staging or replica environment with representative models, prompts, tools and data exists. Agree in writing what data may be used and what must be masked. |
| Depth of adversarial iteration | Objective-based testing adapts each attempt to what the system reveals, so effort rises with the number of rounds needed to reach or rule out each objective. A shallow pass checks known patterns and costs less but gives weaker assurance. | Decide in advance whether the goal is a bounded sample across scenarios or sustained effort on a few high-value objectives. State the stopping rule in the scope. |
| Client-side preparation | Access setup, test accounts, system and prompt documentation, architecture diagrams, tool schemas and logging access all determine how much time testers spend on discovery rather than on adversarial work. | Prepare architecture diagrams, a tool inventory, role matrix, test accounts and logging access before kickoff. Ask the provider to list the prerequisites in the proposal. |
| Retest scope | A retest of fixed findings is a separate piece of work from the first test and is usually scoped and priced on its own. Continuous testing under a retainer is priced differently again because it recurs. | Ask whether retest is included, how many rounds it covers and what evidence closes a finding. If testing must recur as models or prompts change, scope a retainer rather than repeated one-time tests. |
What the report contains and whether an auditor or customer accepts it
An AI red teaming report should let a reader trace each agreed harm scenario to the ATLAS techniques tested, the outcome, the evidence and the fix. The cards below cover the sections a buyer should expect, followed by the question that decides whether the document is accepted as evidence: what the auditor or customer requires.
Executive summary for leadership
A short, plain-language account of the systems tested, the agreed harm scenarios, the overall result and the decisions leadership needs to make. It states what was out of scope and which assumptions the results depend on. Check that it names the test window and the model, prompt and tool versions tested, because an AI system can behave differently after a model or prompt change.
Harm scenarios and outcomes
Each harm scenario agreed before testing appears with its success criteria and whether the system met, partly met or failed it. Results are written against the objective, such as unauthorized disclosure of customer records through a chat interface, not as a count of payloads tried. Ask for a per-scenario outcome table so the reader can see which harms were attempted and which were not reached.
MITRE ATLAS technique mapping
Each test step is mapped to the MITRE ATLAS tactic and technique it exercised, so an auditor can match the work against a recognized threat model. Confirm which ATLAS data release the mapping uses, since the knowledge base changes over time. A mapping that lists techniques not actually exercised is a sign of padding, so check that every mapped technique links to a recorded test.
Findings, severity and defensive reproduction
Each finding gives a severity rating, the affected component, the conditions under which it occurred and the reproduction context at a defensive level, such as the category of input and the observed system behavior. Operational exploit steps are kept out of the shared report and held under restricted access. Confirm the severity scale in advance and whether it follows your own risk register or the scoring the provider uses.
Recommended mitigations
Each finding is paired with a mitigation that addresses its cause, such as input and output filtering, tool permission limits, human approval for sensitive actions or tighter retrieval access controls. Good recommendations name the owner and the layer where the fix belongs, whether that is the prompt, the application, the tool gateway or the model supply chain. Ask whether the provider will discuss feasibility with your engineering team before the report is final.
Retest results and sign-off letter
After fixes are applied, the agreed findings are retested and the report records whether each one is closed, partly addressed or still open, with the system version tested. A sign-off letter summarizes the engagement, but it is a statement by the testing provider, not an audit opinion. Acceptance depends on what the auditor or customer requires, so confirm those requirements in writing before testing starts.
How AI red teaming changes by system architecture
A hosted or self-hosted LLM application places a model behind an API, a system prompt and an application layer that handles user input and model output. The main trust boundaries sit between the user and the prompt, between the model output and the downstream code that renders or executes it, and between the application and the model provider. Red teaming here focuses on whether adversarial input can override instructions, extract the system prompt or sensitive context, or push the output into unsafe handling.
- Input and instruction boundary: Test whether user-supplied content can override the system prompt, using objectives such as policy bypass or disclosure of hidden instructions.
- Output handling boundary: Check whether model output reaches HTML, SQL, shell or template contexts without encoding or validation.
- Provider and logging boundary: Confirm which prompts, completions and metadata are sent to the model host and retained in logs.
- Consumption limits: Test whether a single user can drive unbounded token use or repeated expensive calls.
Environment: use a staging deployment with synthetic customer data, a dedicated test API key with spend limits, and read access to application and gateway logs so testers and your team can trace each finding. Confirm with your provider what the model host retains.
A retrieval-augmented generation pipeline adds an ingestion path, a vector store and a retrieval layer between the user and the model. The defining boundary is that retrieved content is untrusted input: documents, web pages and uploaded files can carry instructions into the prompt. Red teaming therefore covers how content enters the index, who can read which chunks, and whether retrieved text leaks to users who lack permission for the source document.
- Ingestion boundary: Test whether documents from less-trusted sources can carry indirect prompt injection into the context window.
- Access control boundary: Check that retrieval filters enforce the same permissions as the source system, including tenant isolation in shared indexes.
- Embedding and leakage risks: Assess whether queries can recover content from chunks the caller should not see, through ranking, citations or summaries.
- Index administration: Review who can write to or reindex the vector store and whether changes are logged.
Environment: build a seeded corpus with marked test documents at several permission levels, so every leak is attributable. Confirm with your team which vector store is in use and whether its access controls and audit logs are enabled.
An agent takes actions through tools such as ticketing, email, databases or cloud APIs, and may keep memory across sessions. Here the consequences of a successful attack are actions, not only text. Trust boundaries sit between the model's plan and each tool call, between the agent identity and the privileges it holds, and between one session's memory and another's. Harm scenarios focus on goal hijack, tool misuse, privilege abuse and persistent memory poisoning, aligned with the OWASP Top 10 for Agentic Applications 2026.
- Goal and plan boundary: Test whether instructions in tool outputs or documents can redirect the agent toward an objective the user did not set.
- Tool authorization boundary: Check that each tool enforces its own checks and that high-impact actions need explicit approval.
- Identity boundary: Confirm the agent runs under a least-privilege identity separate from any user's full rights.
- Memory boundary: Test whether content written to memory in one session influences later sessions or other users.
Environment: run tool actions against sandbox accounts and non-production systems, with write operations simulated or reversible. Confirm with your team which actions are irreversible and ensure logs capture every tool call with its arguments.
An MCP-connected deployment lets a model client reach external tools and data through MCP servers, which may be remote services or local processes. The Model Context Protocol security best practices name concrete boundaries: the authorization server and token audience, the client's trust in servers it connects to, and the sandbox around local servers. Red teaming tests whether a server accepts tokens not issued for it, whether a proxy can be used as a confused deputy, and whether a local server can run with more privilege than it needs.
- Token audience boundary: Check that each MCP server rejects tokens not explicitly issued for it, which the specification forbids passthrough of.
- Confused deputy boundary: Test whether a proxy server forwards third-party consent or identity without validating the request.
- Discovery and SSRF boundary: Assess whether OAuth metadata or URL handling can be steered toward internal addresses.
- Local execution boundary: Confirm that clients show the exact command before one-click installation and that local servers run with minimal default privileges.
Environment: use test identity providers and test tenants, a sandboxed host for local servers, and network egress rules that log outbound connections. Confirm with each server vendor which scopes and tokens it issues.
What you need to prepare before testing starts
Before the first test, the client needs to name the people who own the outcome. Assign one contact for each harm scenario in the agreed list, such as a customer receiving an unauthorized refund or a clinician seeing another patient's notes, and one technical operator per component under test: the model endpoint, the retrieval layer, each agent tool and any MCP server. The harm scenario owner decides what counts as a meaningful failure, and the operator can answer questions about tool definitions, logging and rate limits during testing. Confirm both people will be reachable during the test window, not only during the scope workshop. Decide internally who can approve a change to scope, because testers will need an answer quickly when an objective turns out to be unreachable or when a finding touches a live customer.
The client supplies the test environment, test accounts and documentation. Ask for a sandbox that mirrors production controls, including the same guardrail and content filter configuration, authentication path, tool permissions and data sources, and state clearly where it differs from production. Provide test accounts for each role in scope, with the permissions a real user, a privileged staff member and an integration service would hold, so that privilege and tenant boundaries can be tested. Share system prompts and tool definitions under the confidentiality terms agreed in the statement of work, and give access to logs and monitoring dashboards so testers and operators can see what the system recorded. Confirm which records are retained, for how long, and who can read them. Agree a change freeze window so that model versions, prompts, tools and filters do not change mid-campaign, because a moving target makes results impossible to reproduce.
Agree the stop conditions and escalation path in writing before testing begins. Define what makes a tester stop immediately, such as exposure of real personal data, a tool call that changes a production record, or signs that a third-party provider's service is degrading, and name the person who can halt the test at any hour. Decide internally whether the organization wants retesting of fixed findings within the same engagement, and who will sign off that a fix has been applied. Finally, check with your legal and compliance leads whether any test prompts or outputs will be stored in a way that creates new personal data obligations, and whether your AI provider's terms allow adversarial testing of the service.
What to check when choosing an AI red teaming provider
A red team on an AI system is only as useful as the scenarios it pursues, the evidence it produces and the independence of the people producing it. The cards below cover the checks that separate an objective-based engagement from a checklist exercise, and what to request in writing before work starts.
Documented methodology mapped to ATLAS
Ask for the written method and how each test step maps to MITRE ATLAS tactics and techniques, and to the OWASP lists that apply to the system, such as the OWASP LLM Top 10 2026 or the Top 10 for Agentic Applications for 2026. A provider that cannot show this mapping is likely running a checklist. Confirm whether the method is reviewed and versioned, and which ATLAS release it targets.
Experience with this architecture
Red teaming a single chatbot differs from testing a retrieval pipeline wired to tools, memory and a multi-agent workflow. Ask for named examples of comparable architectures, described without client names, and for the specific attack surfaces the provider has tested, such as tool-calling permissions, vector store access or MCP connections. Ask how the team keeps up with changes in the threat catalogues.
Harm scenarios agreed and owned
Objective-based testing starts with harm scenarios the client owns, such as an agent taking an unauthorized payment or a model disclosing another customer's records. Ask who writes the scenarios, who approves them, and how out-of-scope harms are handled. The rules of engagement should state which outcomes count as a success for the tester and which count as a finding for the client.
Rules for sensitive data and artifacts
Red team work produces prompts, model outputs and sometimes real personal data. Confirm where test traffic runs, which data sets are used, how long transcripts and logs are kept, who can access them and how they are deleted at close. Ask whether testing uses synthetic data in a non-production environment that mirrors production controls, and get the answer in the statement of work.
Reporting and retesting
Ask for a sample report before buying. It should list each finding with the affected scenario, the ATLAS or OWASP reference, evidence, severity reasoning, and a recommended fix that the client's engineers can act on. Confirm how retesting is priced and scoped, and whether the provider reruns the exact scenario that failed. A report that only lists categories gives an auditor or customer little to verify.
Independence and named testers
A provider that built or configured the system is poorly placed to test its own guardrails objectively. Ask whether the testers were involved in the build, which roles each named tester will hold, and who signs off on findings. Confirm the reporting line for the engagement, and whether any tester has a conflict with the vendor of the underlying model or platform.
Common questions about AI red teaming services
How much does AI red teaming cost?
Cost depends on the written scope, not on a standard rate. DeployOpen does not publish a price list, so a quote follows from the agreed scope. The main drivers are the number of AI systems and interfaces in scope, whether agents can call tools or reach internal data, how many harm scenarios and ATLAS techniques are tested, how much access testers receive, and whether a retest is included. Ask each provider to itemize testing days, scenario design, environment setup, reporting and retest effort, and to state which items are one-time and which recur if you run testing continuously. A quote that cannot explain its drivers is hard to compare.
How long does an AI red teaming engagement take?
Duration follows from scope and from how much time the testers need to reach the objectives you agreed. Expect the schedule to depend on the number of agents and tools, the depth of the harm scenarios, the availability of a test environment that matches production, and the time needed to review and fix findings. Ask the provider for a timeline in the written scope with named milestones, including when access, test accounts and a staging copy will be ready. Delays in access usually cost more time than the testing itself.
What does the client need to provide for an AI red teaming test?
Plan to provide the system's architecture, a description of intended use and the harm scenarios your business cares about, such as data leakage, unauthorized actions or harmful output. Testers also need test accounts at each privilege level, a staging environment that mirrors production controls, logging access so findings can be traced, and a named contact who can approve tests against live tools. Confirm in writing which systems are off limits, which third-party model providers are covered by their own terms, and what happens if a test triggers an incident.
Can our own team run AI red teaming without an outside provider?
Your team can run exercises, and many organizations should, but an independent test carries weight with auditors, customers and insurers because the testers do not build the system they assess. Internal teams are well placed for continuous checks against known scenarios. An external engagement adds an adversarial view, documented methodology and an independent report. Whichever route you choose, agree who owns the harm scenarios, who signs off on findings and how results feed your risk register.
How does AI red teaming differ from an LLM application test or a prompt injection test?
AI red teaming is objective-based. Testers start from harm scenarios you agreed and try to reach them using MITRE ATLAS techniques, chaining steps across the model, retrieval layer, tools and users. An LLM application security test checks the application against the OWASP LLM Top 10 2026 and the web and API layers around it. A prompt injection test focuses on one risk category in depth. Red teaming can include both, but its question is whether a defined harm can happen, and that shapes the scope and the report.
Does AI red teaming satisfy the EU AI Act's adversarial testing duties?
Red teaming can produce evidence toward those duties, but it does not by itself establish compliance. Article 15(5) of Regulation (EU) 2024/1689 requires high-risk AI systems to be resilient against attempts to exploit vulnerabilities, including data poisoning, model poisoning, adversarial examples, model evasion and confidentiality attacks. Article 55(1) requires providers of general-purpose AI models with systemic risk to perform documented adversarial testing. Confirm with your legal and compliance advisers which duties apply to your role and system, and agree with the provider that the report documents scope, methods, results and remediation in a form your auditor can review. DeployOpen does not issue attestations or certifications.
How does retesting work after an AI red teaming engagement?
Retesting checks whether the fixes for each finding work and whether they introduced new paths. Ask the provider to confirm in the written scope which findings are included in a retest, how many rounds are covered, and what evidence closes a finding, such as a rerun of the same scenario or a documented control change. Model updates, prompt changes and new tools can reopen risk, so plan retests after material changes. Ongoing testing can be arranged as a retainer if you need repeated coverage.
How DeployOpen scopes, tests, reports and retests AI systems
DeployOpen runs AI red teaming as a fixed-scope engagement, with objectives, harm scenarios and rules of engagement agreed in writing before any adversarial testing starts. Findings are documented so your team can act on them, and ongoing retesting is available as a retainer.
How we scope
- Written scope names the AI system, its tools, data sources and user roles under test
- Agreed harm scenarios and adversary objectives are mapped to MITRE ATLAS techniques before work begins
- Rules of engagement cover test environments, data handling, rate limits and stop conditions
How we build
- Test scenarios are built from the agreed objectives rather than from a generic checklist
- Attacks on agents and tool calls run in a controlled environment with test accounts and seeded data
- Testers record which guardrails held, which failed and the conditions needed to reproduce each result
How we hand over
- Report maps each finding to the relevant OWASP GenAI or ATLAS entry and states its business impact
- Remediation guidance is specific to your architecture, with owners and priorities suggested for each item
- Evidence of adversarial testing is packaged so your compliance lead or auditor can review it directly
How we support
- Retesting confirms whether agreed fixes close each finding and documents the outcome in writing
- Ongoing support and continuous testing are available as a retainer after the initial engagement
- Confirm with DeployOpen during scoping which retest rounds and support terms are included in your written scope

