Real-world testing under the EU AI Act: plan and approval

Can we test high-risk AI in real-world conditions under the EU AI Act, and what does it take?

Yes, but only on a narrow path. Providers of Annex III high-risk systems or Annex I Section A products may test them in real-world conditions outside a regulatory sandbox at any time before market placement, alone or with deployers, under a real-world testing plan approved by the market surveillance authority of the Member State concerned. The test may run no longer than necessary and at most six months, and it can be extended once by six months after the authority is notified with reasons. The organization owns that plan, the consent process, the deployer agreements, the logging and the incident handling. An implementation partner can prepare the systems, test environment, logs and evidence packs, but the partner does not approve anything. Only the authority does, and silence for 30 days counts as approval unless national law requires an explicit decision.

The answer changes with four variables. The first is the system's Annex category, since Annex III and Annex I Section A products follow the same testing route but carry different conformity paths and dates. The second is the Member States involved, because each has its own market surveillance authority, its own approval practice and possibly its own rules on explicit authorization. The third is whether deployers take part, which triggers a written agreement with each participating deployer and shifts some operational duties to them. The fourth is whether an Annex I Section B framework applies, such as machinery, where a Member State may have adopted a separate real-world testing framework under Article 60a, with its own mandatory testing plan. Confirm each of these with the relevant authorities before you design the test.

Before committing engineering time, check four things: that the registration with a Union-wide identification number and the EU establishment or legal representative are in place, that vulnerable subjects are protected, that qualified human oversight can reverse or disregard outputs, and that informed consent records are dated and documented. A reasonable first step is a written checklist mapping each condition to an owner, an artifact and a test date. Ask any implementation partner to show how its environment isolates test subjects, captures evidence and routes serious incidents to the reporting process, and to state in writing which tasks remain with your team.

What the real-world testing process runs through, from plan to close-out

The sequence turns a test idea into a documented, approved and bounded real-world test, with each phase leaving evidence that an assessor or market surveillance authority can check. Providers own the sequence, and deployers and test subjects each have defined roles within it.

Prepare the testing plan and metrics
The provider writes a real-world testing plan covering the intended purpose, the population and deployment setting, the duration, the oversight arrangements and the reversal process. The metrics and probabilistic thresholds are set in advance and tied to the testing duties in Article 9, which expect testing against defined metrics before market placement. Output to hold: a signed plan version with its metric set and a record of the risk management results that the test builds on.
Confirm the EU establishment or legal representative
Before submission, the provider confirms that it has an establishment in the Union or has appointed a legal representative there, since the testing conditions require one. For a non-EU provider, this is the step where the representative's appointment and contact details are verified and placed in the file. Output to hold: a written confirmation of the establishment or representative, with the named contact for the market surveillance authority.
Submit the plan to the market surveillance authority
The provider submits the testing plan to the market surveillance authority of the Member State where the test will run. The submission should match the approved plan version exactly, so later changes can be traced. Confirm with the authority which channel and format it uses, because the Act sets the requirement but national procedures determine the mechanics. Output to hold: a dated submission receipt and the plan reference the authority assigns.
Wait for approval or the silence rule
The authority either approves the plan or responds with a request for change. If it stays silent for 30 days, that silence counts as approval unless national law requires an explicit authorisation, so the provider must check the national rule before relying on it. The provider does not start testing until the approval route is settled. Output to hold: a written approval, or a dated record showing the 30-day period ran out under a national law that permits silence to count.
Register the test and sign deployer agreements
The test is registered under a Union-wide identification number, and each participating deployer signs a written agreement that sets out roles, oversight, the ability to reverse or disregard outputs, and how incidents are passed on. Deployers should review their own duties and confirm that their internal governance can run the test. Output to hold: the registration number and a signed agreement for every deployer in the test.
Recruit subjects and document informed consent
Subjects are recruited under the protections for vulnerable people, and each one receives the information the Act requires: the nature, objectives, duration and conditions of the test, the right to withdraw at any time without detriment, and how to have outputs reversed or disregarded. Consent must be freely given, dated and documented. Output to hold: a consent record for each subject that can be produced on request, with withdrawals logged.
Run the test under qualified oversight
The test runs with qualified human oversight, the ability to reverse or disregard outputs, and continuous checks against the plan's metrics. Serious incidents are reported under Article 73, and the provider must mitigate immediately or suspend or end the test. A recall procedure should be ready before the first subject is included. Output to hold: an oversight log, incident records and a current record of metric results.
Close out, or extend once with reasons
Testing may last no longer than necessary and at most six months. It can be extended once by six months, but only after the provider notifies the authority with reasons. At close-out, the provider ends subject involvement, confirms that outputs are reversed or disregarded where required, and records what the test showed. Output to hold: a closure record or an extension notice with reasons, plus the final results file that supports the next compliance step.

Which high-risk systems qualify, and who can run the test

Only providers can start a real-world test under Article 60 of the AI Act, and only for two groups of systems: high-risk AI systems listed in Annex III, and products covered by Annex I Section A. The provider may run the test alone or together with deployers, but the testing plan is the provider's document. A deployer that wants to put a system in front of its own users before launch cannot begin that test on its own initiative. It needs the provider's plan, the provider's registration and the written agreement that the Act requires between the parties. Before drafting anything, the organization should confirm which role it holds and which Annex category its system falls under, because that determines whether Article 60 applies at all. The Annex III category depends on the intended purpose of the system, so the intended purpose statement should be fixed first.

Annex I Section B covers products such as machinery, where the AI is a safety component of a product regulated by other EU legislation. Article 60a does not create a general right to test these products. It allows Member States to adopt their own frameworks for real-world testing under the relevant product law, and any such framework requires a mandatory testing plan agreed with the authority responsible for that product law. The applicable framework therefore depends on the Member State where testing will take place, and on the sector legislation that governs the product. Organizations should confirm with the national market surveillance authority, and with the authority behind the product legislation, whether a framework exists, what it requires and which authority approves the plan. Those answers should be in hand before the testing plan is written, not discovered during review.

Which conditions every real-world test must meet under Article 60

Article 60(4) sets a short list of conditions that apply to every real-world test of a high-risk system, whether the provider tests alone or with deployers. Each condition needs its own evidence and a named owner, because a missing item can stop the test before it starts.

Union-wide registration identification number

Every real-world test must be registered and given a Union-wide identification number before subjects are exposed to the system. The registration record is the evidence: the number, the test plan reference and the authority that received the filing. Confirm with the market surveillance authority which registration channel applies in the Member State concerned, and assign ownership to the regulatory affairs or compliance lead.

EU establishment or legal representative

The provider must have an establishment in the EU or appoint a legal representative there before testing starts. The evidence is a signed appointment or a documented establishment record that a regulator can contact. Ownership usually sits with legal counsel or the head of compliance, who should confirm that the appointed contact can receive authority requests during the full test period.

Protection of vulnerable subjects

Test subjects who may be vulnerable, such as minors, patients or people in financial hardship, need extra safeguards before they are enrolled. Evidence includes a documented vulnerability assessment, selection criteria, and the safeguards applied to each group. The clinical, product or human factors lead should own this, with review from an ethics or data protection function.

Written agreement with participating deployers

Where deployers take part in the test, the provider needs a written agreement that sets out roles, responsibilities, data handling, monitoring, and how incidents are reported. The signed agreement and its schedules are the evidence. Procurement and legal teams usually draft it, while the test owner confirms that operational duties match what the deployer can actually perform.

Qualified oversight during the test

A test must run under qualified human oversight, meaning people with the training and authority to monitor outputs and stop the test. The evidence is a named oversight roster, training records, and an escalation procedure with stop criteria. The head of the test program owns this, and should confirm that oversight staff can act outside normal working hours if the test runs around the clock.

Ability to reverse or disregard outputs

Participants must be able to have the system's outputs reversed or disregarded, so a decision made by the system does not stick if it was wrong. The evidence is a tested override or rollback procedure, logs showing that reversals were carried out, and a record of the time taken. Engineering owns the mechanism, and the test lead should run a drill before enrolment to confirm it works.

Testing outside a sandbox or inside one: which route fits

This table compares real-world testing under Article 60 with participation in a regulatory sandbox, covering legal basis, approval, duration and fit. Read each row against your system's risk category and your timeline, and treat the sandbox column as a prompt to confirm conditions with your national authority before you plan around it.

Route	Real-world testing under Article 60	Regulatory sandbox
Legal basis	Article 60 of the AI Act, as amended in 2026, for Annex III systems and Annex I Section A products	Sandbox rules in the AI Act, applied through a national or EU sandbox programme
Approval step	Testing plan submitted to and approved by the market surveillance authority of the Member State concerned	Admission and sandbox plan agreed with the sandbox authority, which the organization should confirm directly
Approval timing	Silence for 30 days counts as approval unless national law requires explicit authorisation	Depends on the sandbox call and authority process, so confirm the schedule with the national authority
Duration limit	No longer than necessary and at most six months, extendable once by six months with notice and reasons	Set by the sandbox plan and exit conditions, so confirm the limit for the specific programme
Registration	Union-wide identification number required before testing starts	Confirm with the authority whether registration applies in addition to sandbox admission
Subjects and consent	Informed, dated and documented consent from testing subjects, with protection of vulnerable people	Consent and personal data handling set out in the sandbox plan, confirm specifics with the authority
Liability and incidents	Provider remains liable for damage and must report serious incidents under Article 73	Liability and fine protection depend on the sandbox conditions, see the sandbox page for detail
When it fits	A provider that needs live-user evidence before market placement and can meet the plan, registration and consent conditions	Organizations that want regulator guidance on the system before testing, and can accept the sandbox's schedule and conditions

Who owns each duty in a real-world test: provider, deployer, authority and partner

The provider of the Annex III or Annex I Section A system holds the core legal duties for a real-world test under Article 60. The provider writes the real-world testing plan, submits it to the market surveillance authority of the Member State concerned, and keeps the test within the conditions the Act sets. Those conditions include registration with a Union-wide identification number and an EU establishment or legal representative. The provider also owns the consent records required by Article 61, the handling of serious incidents, the recall procedure, and liability for damage caused during testing.

Testing plan: the provider drafts, submits and maintains the plan, and must not start testing before approval or the applicable silence period.
Registration and EU presence: the provider holds the Union-wide identification number and an EU establishment or legal representative.
Consent records: the provider keeps dated, documented informed consent from each testing subject and records withdrawals.
Incident reporting and recall: the provider reports serious incidents under Article 73 and maintains a recall procedure.
Liability: the provider remains liable for damage caused during testing, whichever party operated the system.

Before testing, confirm with your national market surveillance authority which approval route applies, whether silence counts as approval, and what evidence it expects with the plan.

How the six-month limit, the 30-day clock and liability shape the plan

The timing rules set the outer limits of any real-world test plan. Testing must last no longer than necessary and at most six months, and it can be extended once by six months, but only after the provider notifies the market surveillance authority with reasons. Build the plan around the shortest test that answers the metrics defined under Article 9, and write down the decision point at which you would ask for an extension. An extension request that arrives late or without a clear justification is an avoidable risk to the schedule. The 30-day approval route deserves caution. Under Article 60, a testing plan submitted to the authority is treated as approved if the authority stays silent for 30 days, unless national law requires an explicit authorization. Whether your Member State applies silence as approval, and what the authority expects in the submission, are questions to settle before you count on that route. Confirm the national rule and the authority's submission format directly with the authority, and plan for an explicit approval timeline if the answer is not clear.

Liability runs for the whole test. If a serious incident is found during testing, it must be reported under Article 73. The provider must mitigate the problem immediately, or suspend or end the test, and must have a recall procedure ready. The provider remains liable for damage caused during testing, so insurance, contractual allocation with deployers and the incident response runbook should be reviewed before the first test subject is enrolled. Ask your legal and insurance advisers to confirm how these duties interact with the written agreement with each participating deployer. For planning, treat suspension as a designed-in outcome, not a failure. Name the person who can suspend the test, the threshold that triggers suspension, and the time within which the test team must act. Those decisions, recorded in the testing plan, are the evidence an authority will look for when it reviews how the provider handled problems during the test.

What must be in place before the testing plan goes to the authority

A real-world testing plan is only as strong as the records behind it. The six items below are the documents and controls an authority will expect to see before it reviews the plan, and each one has to be checked against the actual test rather than assumed from the product.

Defined test purpose and metrics

State what the test will establish, for which intended purpose, and which prior metrics and probabilistic thresholds decide success or failure. Article 9 requires testing against defined metrics suited to the intended purpose, and Article 60 testing should be traceable to those same measures. Check that each metric has a pass threshold, a data source and a named owner who can stop the test if it is breached.

Risk management record

Provide a risk management record that covers the risks identified for the system, the mitigations applied, and the residual risks accepted before real users are involved. The record should explain why the proposed population and conditions are proportionate to those risks. Check that it is dated, versioned and was updated after the most recent test results, not written once for the original development.

Consent forms and withdrawal process

Prepare the informed consent documents and the procedure for using them. Each form must explain the nature, objectives, duration and conditions of the test, the right to withdraw at any time without detriment, and how outputs can be reversed or disregarded. Check that forms are dated and documented per subject, that withdrawal can be carried out in practice, and that vulnerable subjects receive the protections the Act requires.

Deployer agreement templates

Where deployers take part, the written agreement sets out each party's roles, the sites and users involved, oversight duties and reporting obligations. Check that the template covers how deployers will apply qualified oversight, how they will flag incidents to the provider, and who holds the consent and logging records. An authority will expect to see the signed or ready-to-sign version for each participating deployer.

Logging and output reversal capability

Show that the system can log test activity in enough detail to reconstruct decisions, and that operators can reverse or disregard the system's outputs for affected subjects. Verify this in a dry run, not only in design documents. Check that logs are protected from unauthorized change, that the reversal step has a documented owner, and that the test can be halted without losing the records needed for later review.

Incident and suspension procedure

Document how a serious incident found during testing is detected, assessed, reported under Article 73, and handled. The procedure must allow the provider to mitigate immediately or suspend or end the test, and it should include a recall procedure. Name one accountable owner with authority to suspend the test, and confirm that person is reachable during testing hours and knows the reporting route to the market surveillance authority.

Questions buyers ask about real-world testing under the AI Act

Can we start real-world testing before the high-risk rules apply?

Yes, for systems covered by the testing route. Providers of Annex III high-risk systems or Annex I Section A products may test them in real-world conditions outside a sandbox at any time before market placement, alone or with deployers, under a written real-world testing plan (Article 60(1) and (2)). The date that matters is therefore not only when the high-risk rules bite, but whether the test itself meets Article 60 from its first day. Confirm the current application dates against the consolidated text, because the 2026 amendment moved the Annex III and Annex I high-risk dates. Plan the testing timeline around the approval step, not around the deadline.

Who decides whether our testing plan is approved?

The market surveillance authority of the Member State concerned decides, not the implementation partner. The testing plan must be submitted to and approved by that authority (Article 60(4)(a)). Silence for 30 days counts as approval unless national law requires an explicit authorisation (Article 60(4)(b)), so check your national rules before relying on that default. A partner can prepare the plan, the registration data and the supporting evidence, but the organization submits it and answers any authority questions.

Does a partner have to be involved in the testing plan?

No. The organization owns the plan, and a partner can help prepare it. Work that a partner commonly prepares includes the description of the system and its test environment, the monitoring and logging design, the human oversight arrangements, the procedure for reversing or disregarding outputs, and the evidence pack. Work that stays inside the organization includes the decision to test, the choice of participating deployers and subjects, the signing of agreements, and the accountability for results. Ask any partner to state in writing which parts of the plan it drafts and which parts your team must approve.

Do deployers have to sign something before testing starts?

Yes. Real-world testing requires a written agreement with each participating deployer (Article 60(4)). The agreement should set out the roles of each party, who runs human oversight, how outputs can be reversed or disregarded, how incidents are reported to each other, and how the test ends. Ask your legal team to confirm the wording, since the agreement must match what the testing plan promises the authority. Expect the negotiation to take time, and build that into the schedule before the authority review begins.

What must informed consent cover for test subjects?

Each subject must give freely given, dated and documented informed consent after being told the nature, objectives, duration and conditions of the test (Article 61). Subjects must also be told that they can withdraw at any time without detriment, and how to have the system's outputs reversed or disregarded. Records should show who consented, when, to which version of the information, and how withdrawals are handled. Vulnerable subjects need additional protection under the conditions in Article 60(4), so consent design should be reviewed by someone who understands those groups in your sector.

What happens if a serious incident occurs during testing?

The provider must report the serious incident under Article 73, and must either mitigate it immediately or suspend or end the test (Article 60(7)). The provider also needs a recall procedure. Incident handling should be rehearsed before the first subject is enrolled, with named owners, reporting contacts for authorities and deployers, and a clear trigger for stopping the test. Your incident response runbook should be ready and tested before any subject is enrolled, so that the first reporting decision does not depend on a process written under pressure.

Who is liable for harm caused during testing?

The provider remains liable for damage caused during the test (Article 60(9)). Deployers and partners may hold their own obligations under contracts or national law, so the written agreements should allocate responsibilities clearly, but they do not move the provider's liability under the Act. Check insurance coverage for testing activities with your broker, and ask your legal advisers how national tort law applies in each Member State where subjects are located. Do not assume a partner carries this risk unless the contract says so in writing.

Can the real-world testing period be extended?

Yes, but only within limits. Testing may last no longer than necessary and at most six months, and it can be extended once by six months after notifying the market surveillance authority with reasons (Article 60(4)(f)). A second extension is not provided for in the Act, so an organization that needs longer should plan the exit, the evidence and any placement on the market accordingly. Build the test schedule with a realistic end date from the start, and record the reasons for any extension request so the notification is ready when needed.

How DeployOpen prepares systems and evidence for a real-world test

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, so the systems and records behind a real-world test are ready before a testing plan goes to the market surveillance authority. We prepare the systems and evidence; the provider keeps control of its obligations, and an independent authority or assessor reviews them.

How we scope

Engagements are fixed-scope, with the systems, test population boundaries and deliverables agreed in writing before work starts.
The written scope states which logging, human oversight and output reversal mechanisms the test environment must provide.
Scope records which deployers, sites and data flows participate, so the written agreements and consent records match the test.

How we build

We implement the self-hosted platform and test environment inside your cloud accounts, data centers or air-gapped network.
We configure access controls, kill switches and the ability to suspend the system or disregard its outputs during testing.
We run pre-launch testing against the metrics and thresholds you define, and record the results as evidence for the testing plan.

How we hand over

We deliver the documentation set: system description, testing plan inputs, consent and deployer agreement templates, and incident procedures.
We hand over logs, test records and configuration baselines in a structure your compliance team and assessors can review.
We walk your staff through the runbooks for suspending the test, reporting serious incidents and executing a recall procedure.

How we support

Ongoing support, upgrades and continuous testing are available as retainers, agreed in writing separately from the initial scope.
Retainers can cover monitoring of the test environment, updates to documentation when the testing plan is extended, and incident support.
Confirm with your legal and compliance teams which decisions, registrations and submissions remain with your organization and its authorities.

Testing high-risk AI in real-world conditions under the AI Act