A.7.4 Quality of data for AI systems: requirements and evidence

What A.7.4 quality of data for AI systems asks for

A.7.4 asks the organization to define and document requirements for the quality of data used in AI systems, and to show that the data used to develop and operate the system actually meets those requirements. When an auditor or customer asks how you know your training, evaluation and operational data is fit for purpose, the answer should be a written quality specification for each dataset plus records showing it was measured against that specification. Poor data produces unreliable, biased or unsafe outputs, and those failures often surface only after deployment, when they are expensive to trace. Quality is also hard to pin down because it depends on the use. A dataset that is complete enough for a fraud model may be far too sparse for a clinical triage tool, and a label set that works for training may be unfit for evaluation.

ISO/IEC 42001 is a management system standard, so it does not prescribe quality dimensions or numeric thresholds. The organization sets them, usually by choosing dimensions such as completeness, accuracy, consistency, timeliness and representativeness, defining an acceptable level for each, and stating what happens when a dataset falls short. Confirm with your certification body how much rationale it expects behind each threshold, because an auditor will test whether the thresholds are justified for the intended use. Reference material such as the ISO/IEC 5259 series on data quality for analytics and machine learning can inform the measures you choose. A.7.4 sits within the nine Annex A objective areas and the 38 controls in total, alongside A.7.2 on data management processes, A.7.3 on acquisition, A.7.5 on provenance and A.7.6 on preparation. Its quality requirements feed the preparation steps and the provenance records, so define them before those controls are documented.

Which datasets, systems and teams fall under A.7.4

A.7.4 applies wherever data shapes how an AI system is built or behaves, so scoping starts with the systems, datasets and teams that touch that data. The decisions below determine how much quality work the organization owns, and each one should be recorded in the scope statement and the statement of applicability under clause 6.1.3.

AI systems the organization builds or provides

Start with an inventory of every AI system the organization develops, fine-tunes or offers to customers, including features embedded in other products. A.7.4 applies to each one that depends on data the organization controls. For each system, record the intended purpose, the data it consumes and who signs off on quality requirements. Systems that only call a third-party model through an API still need a quality decision for the inputs they send and the outputs they store, even if the model's own training data is out of reach. Ask engineering to confirm the list is complete, since shadow prototypes often reach production unrecorded.

Training and tuning datasets

Training, fine-tuning and validation datasets are the core of A.7.4. Each dataset needs documented quality requirements, such as completeness, accuracy, consistency, timeliness and representativeness for the population the system serves. Record which dataset versions were used for each model release so an assessor can trace a quality threshold to the data that was tested against it. Labeled data deserves separate attention, because labeling errors are a common and hard-to-detect quality failure.

Evaluation and test data

Evaluation sets measure whether the system meets its performance criteria, so their own quality determines whether those measurements mean anything. Confirm that evaluation data is independent of training data, is kept current enough to reflect deployment conditions, and has documented quality checks. Teams often apply looser standards to test sets than to training sets, which weakens the evidence an auditor expects to see. Include these sets in the same inventory as training data.

Retrieval and operational inputs

Retrieval-augmented systems, scoring models and agents read data at runtime, and that live data needs quality controls too. Identify the knowledge bases, feature stores, logs and databases each system queries, then decide which quality dimensions and thresholds apply to them. Stale or duplicated records in a retrieval index can degrade outputs without any change to the model, so monitoring for freshness and error rates belongs in scope.

Third-party and customer data

Data supplied by customers, data brokers, open datasets and suppliers enters the quality process at the point of ingestion. Decide whether the organization will accept supplier quality claims or verify them, and document which contractual or technical checks apply. Customer data used to improve a model raises both quality and rights questions, so coordinate with legal before deciding whether it is in scope. Record each external source so the quality requirement and its verification are traceable.

Teams that own quality decisions

Data engineering owns pipelines and checks, ML teams own thresholds and evaluation results, product owners define the purpose that quality requirements must serve, and legal and privacy staff review data that includes personal information. A.7.4 fails when each team assumes another owns the requirement. Name one accountable owner per dataset and record how the others contribute, consistent with the roles defined under the internal organization controls.

Statement of applicability justifications

Clause 6.1.3 requires the organization to justify including or excluding each Annex A control. A.7.4 is rarely excluded for a system that trains or runs on data, but an organization with no AI development and only a supplier-managed API may justify a narrower scope. Any exclusion must state the reason and the residual risk, and the auditor will test whether the reason holds. Revisit the justification when a new system or dataset appears.

How an assessor tests A.7.4

This table walks through the steps an assessor typically takes to test A.7.4, the evidence each step asks for, and the findings that most often surface. Read it as a checklist for building your own evidence trail before the audit starts, since assessors work from the documented requirements outward to the datasets and the records of what happened when quality fell short.

Test step	Evidence requested	Common findings
Review the written data quality requirements	Documented data quality requirements covering the quality dimensions chosen for each AI system, with an owner and approval record	Requirements exist only as general statements such as data must be accurate, with no dimensions or criteria defined
Trace requirements to specific datasets	A mapping from each requirement to the datasets used for development and operation, showing which requirement applies to which data	Datasets used in production have no recorded link to any quality requirement, so the assessor cannot tell what was checked
Sample measurement reports	Quality measurement outputs for a sample of datasets, showing the metric, the value, the date and the measurement method	Reports are one-off spreadsheets with no method, no date or no link to the dataset version that was actually used
Check thresholds and their rationale	Defined thresholds for each measured dimension, with a written reason for each value and approval by a named owner	Requirements are never measured, or thresholds are set with no rationale and are never revisited after the system changes
Review remediation records when thresholds fail	Tickets or logs showing the failed check, the root cause, the corrective action, the approver and the re-test result	Quality issues are found and logged but not resolved, or data is used anyway without a documented decision to accept the risk
Confirm checks run before data is used	Pipeline configuration, gate definitions and run logs showing quality checks executed before training, tuning or operational use	Checks run only on request or after deployment, so data reaches development or operation without passing any gate
Confirm reviews after data or system changes	Records of re-measurement after new data sources, schema changes or model updates, with updated thresholds where needed	Quality requirements were set at launch and never re-assessed when data sources or use changed

Implementing A.7.4 step by step

This sequence turns A.7.4 from a written intention into a working data quality process: requirements are defined per AI use, thresholds are measured, checks run inside the pipeline, failures are remediated, and every dataset and release carries a record an assessor can follow.

Scope the AI uses and their data
Start by listing each AI system in scope and the datasets it uses to develop, enhance and operate, including retrieval sources and evaluation sets. The data owner and the system owner agree on which datasets feed which decisions, since quality needs differ between a fraud model and a document summarizer. The step is done when every dataset has a named owner and a link to the AI system it supports. The output is a data inventory that feeds the quality requirements document.
Define quality dimensions for each use
For each AI use, decide which quality dimensions matter and why. Typical dimensions include completeness, accuracy, consistency, timeliness and representativeness, but a given use may need only some of them, and representativeness often matters most where the model will meet populations absent from the training data. Write the reasoning next to each choice, because an assessor will ask why a dimension was excluded as well as why it was included. The output is a quality requirements document, approved by the accountable owner for the AI system.
Set measurable thresholds
Turn each dimension into a measure and a pass or fail threshold. Examples include the share of required fields populated, the rate of records that fail a validity rule, or the maximum age of a source before it is treated as stale. Thresholds should come from the use case and from the risk assessment, not from what the current data happens to achieve. Record each threshold, its rationale and its approver in a threshold register. Confirm with your own risk owners how tolerances are set, since the standard does not supply numeric values.
Build measurement into the data pipeline
Implement the checks where data enters, moves and is transformed, rather than as a one-off review before release. Validation rules, schema tests, distribution comparisons and freshness checks can run as pipeline steps that fail a job or quarantine a batch when a threshold is breached. Keep the rules under version control so the check that ran on a given date can be reproduced. The step is complete when a failed run stops or flags the batch and writes a timestamped entry to the pipeline check log.
Define remediation and escalation
Decide in advance what happens when a check fails: who triages, which fixes are allowed (correcting, excluding, re-sourcing or accepting a documented deviation), and when an issue goes to the AI system owner or governance committee. Remediation should never silently change a threshold to make a dataset pass. Document the path in a procedure and test it with at least one simulated failure. The output is a written remediation procedure and an escalation contact list.
Run checks against each dataset and release
Execute the defined checks on each dataset version used for development, evaluation and operation, and on each release that changes the data. Compare results with the threshold register and record whether the dataset met requirements, was remediated, or was accepted with a documented exception. Where data is used to operate a system, such as retrieval corpora or live inputs, schedule recurring checks and record their results as well. The output is a per-dataset, per-release quality result.
Log remediation and accepted exceptions
Maintain a remediation log that links each failed check to its root cause, the action taken, the approver and the date the dataset was re-checked. Accepted exceptions need an explicit risk decision, an expiry or review point, and a note in the impact assessment where relevant. Review the log at the management review cycle to spot recurring defects in sources or pipelines. The output is a closed remediation log and a short list of systemic issues for the improvement plan.
Review requirements and retain evidence
Revisit the quality requirements when the AI use changes, when new data sources are added, or when monitoring shows drift, and keep the superseded versions. Keep the requirements document, threshold register, pipeline check logs, remediation log and per-release results together so an assessor can trace one dataset from requirement to result. Ask your assessor which retention period and sampling approach they expect, since these depend on the audit and the organization's own policy. The output is a dated, version-controlled evidence set for the A.7.4 control.

Choosing quality dimensions and thresholds that hold up in an audit

The dimensions and thresholds you pick for A.7.4 must follow from what the AI system is for. Start by writing down the decision the system supports, the people or processes that act on its output, and the harm a poor input could cause. Completeness, accuracy, timeliness, consistency and representativeness are common dimensions, but an assessor will ask why each one was chosen and why it matters for this system. A fraud model and a document summarizer need different dimensions, and a list copied from another project is the first thing an auditor will question. Record the link between each dimension and the system's intended purpose in the data quality requirements document, since A.7.4 asks you to define those requirements and show that data used to develop and operate the system meets them.

Thresholds need a justification you can defend, not a round number chosen for convenience. The NIST AI RMF to ISO/IEC FDIS 42001 crosswalk, contributed by Microsoft, maps the guidance for this control to MAP 2.3 and MEASURE 2.3, which concern performance or assurance criteria that are measured and demonstrated under conditions resembling deployment. In practice, that means setting thresholds against held-out data drawn from production-like sources, then keeping a record of the test population, the measurement method and the result. Where a threshold is a trade-off between data volume and error rate, document the sensitivity analysis behind it. The ISO/IEC 5259 series on data quality for analytics and machine learning provides shared terminology and measures. Part 1, published in July 2024, gives the overview and vocabulary, and parts 2 to 5 cover models, measures and guidelines. Confirm the current status and titles of each part with the ISO catalogue before you cite them in your documentation.

Evidence that satisfies A.7.4 and how to keep it current

An assessor testing A.7.4 looks for a written quality standard, evidence that the data measured against it, and a record of what happened when it fell short. The six artifacts below link those pieces together, and each one needs a review rhythm and a version tied to the AI system it supports.

Data quality requirements document

This is the organization's own statement of what good data means for each AI system, covering the quality characteristics it cares about, the data sets in scope and who approved the requirements. It is the baseline the assessor checks everything else against. Confirm that it names the AI system and model release it applies to, that it records an owner, and that it is reviewed whenever the intended purpose or deployment context changes. Guidance in ISO/IEC 42001 for this control is the reference to read alongside the text.

Dimension and threshold register

This register turns the requirements into measurable terms: each quality dimension, how it is measured, the pass threshold and the action taken on a breach. Thresholds should be set before measurement begins, not adjusted afterward to fit results. Keep a change history for each threshold, with the reason and approver. Review the register at each material model change, since a threshold that suited one training set may not suit a retrained model.

Measurement reports per dataset and version

Each measurement run should produce a report tied to a specific dataset version, its source and the date measured. The report shows results against each threshold, the method used and any sampling approach. Retain reports for every dataset used in production and for a sample of prior versions so an assessor can trace a past release back to its data. Retention periods should be set in the organization's own policy and confirmed against its contractual and regulatory obligations.

Pipeline validation logs

Automated checks inside the data pipeline, such as schema validation, null rate checks and range tests, generate logs that show whether each run passed or failed. These logs are the continuous proof that quality controls operate between formal reviews. Check that logs are immutable or access-controlled, that they record the pipeline version and that failed runs are linked to a ticket. Gaps in log coverage during an audit period are a common finding.

Remediation and exception records

When data fails a threshold, the record should show the root cause, the corrective action, who approved it and when the data was re-measured. Exceptions, where data is used despite a shortfall, need documented justification, a risk assessment and an expiry date. Auditors test whether exceptions are rare, approved by the right owner and closed on time. An open exception with no owner usually signals a weak control.

Sign-off records

Sign-off records show that a named accountable person reviewed the quality results and accepted the data for development or operation at each stage. They should reference the dataset version, the measurement report and any exceptions. Keep sign-offs linked to model release approvals so that every production model traces to an approved data quality decision. Confirm that the approver has the authority and competence described in the organization's AI roles documentation.

What meeting A.7.4 costs in money and effort

This table breaks down the main cost components of meeting A.7.4 Quality of data for AI systems, showing who is paid for each, whether it is one-time or recurring, and what pushes it up or down. Use it to build a scope before asking for quotes, since no price or duration is published for this control and every figure depends on your data estate.

Cost component	Who is paid or who bears it	One-time or recurring	What drives it
Defining quality requirements and thresholds	Internal data owners and AI governance staff, sometimes an external advisor	One-time, then reviewed periodically	Number of data domains, number of AI systems sharing datasets, and how many business owners must agree on thresholds
Engineering time to build quality checks	Internal data engineers or an implementation partner paid under a fixed-scope agreement	One-time build, with ongoing changes	Number of pipelines, data formats, and whether checks already exist in current ETL or data platform tooling
Open-source or commercial quality tooling	Software licensing to a vendor, or hosting and staff time for self-hosted open-source tools	Commercial licenses recur; open-source tools carry hosting and maintenance costs	Data volume, number of users, and whether the organization already runs a self-hosted data stack that can host the checks
Remediation of failing data	Data engineering, domain experts and sometimes third-party data providers	Recurring whenever checks fail and upstream sources are corrected	Quality of source data, frequency of data change, and how far upstream remediation is possible
Evidence upkeep	Internal compliance or data governance staff	Recurring, usually tied to each review cycle	Frequency of dataset refreshes, number of quality reports retained, and how automated the evidence trail is
External review and readiness support	External assessors, consultants or implementation partners	Typically one-time for readiness, then periodic	Scope of the review, maturity of existing data governance, and whether evidence is ready when the review starts
Auditor sampling and testing	The certification body or independent auditor, paid by the organization	Recurring at each audit cycle	Number of AI systems in scope, number of datasets sampled, and how quickly the auditor can trace quality records

Who owns A.7.4 and what can be outsourced

The organization owns A.7.4. It must define and document the data quality requirements for each AI system, decide which quality dimensions matter for the intended purpose, and set the thresholds that data must meet before it is used to develop or operate the system. It also decides what happens when data falls below a threshold, including who may approve an exception and who signs off the remediation. These are management decisions tied to the organization's risk appetite and to the people affected by the system's outputs, so they cannot be delegated to a vendor.

Requirements and thresholds: Written quality criteria per dataset and per AI system, approved by an accountable owner.
Remediation authority: Named roles that decide whether data is fixed, excluded, re-sourced or accepted with documented risk.
Review cadence: A schedule for revisiting thresholds when the system's purpose, population or data sources change.
Evidence ownership: Responsibility for the quality reports, exception records and sign-offs an auditor will request.
Staff competence: Trained data owners who can explain the criteria and the results.

Before engaging any partner, confirm that your own owners can state each threshold, justify it and show who approved it.

Where organizations fall short on data quality for AI

Most organizations have some data checks, but few can show an assessor a defined quality standard for the data behind each AI system. The gaps below recur across teams, and each one leaves A.7.4 unevidenced even when the underlying data is sound. Each card names the gap and the action that closes it.

No written quality requirements

The control asks the organization to define and document data quality requirements before it can show that data meets them. Teams often rely on informal expectations held by the data scientists who built the model. Ask for a single document per AI system that lists the quality requirements, who approved them and the date they took effect. Close it by writing requirements before the next dataset is accepted.

Dimensions not linked to the AI use

Generic data quality dimensions such as completeness and accuracy are easy to adopt but say little about whether an AI system works for its purpose. A credit model and a document search tool need different emphasis on timeliness, representativeness and label correctness. Map each dimension to the decision the system supports and record why it matters there. Close it by tying every dimension to a stated use.

Thresholds without rationale

Teams often set a threshold, such as a maximum share of missing values, because a tool default or a colleague suggested it. An assessor will ask how the number was chosen and what happens when a dataset falls below it. Record the threshold, the reasoning, the people who accepted the risk of a lower value, and the review date. Close it by documenting the reason behind each threshold.

Checks run once, not on new versions

A dataset that passed validation at training time can drift when upstream sources change, labels are re-issued or a pipeline is altered. Quality checks that run once at the start leave the operating data untested. Schedule checks on each new data version and on the data the system uses in operation, and keep the results. Close it by running the same checks on every new version.

No remediation trail

Finding a quality problem is only half of the control. Assessors look for what the team did next: the defect logged, the root cause, the data corrected or excluded, and the decision to retrain or not. Without this trail, the organization cannot show that its requirements lead to action. Keep a ticket or log entry for every failed check with its disposition. Close it by logging each failed check through to closure.

Quality evidence scattered across notebooks

Quality results often live in notebook outputs, personal folders and chat threads, which change or disappear as people move on. An assessor cannot trace a result back to a dataset version, a rule or a sign-off from those sources. Store check outputs in a shared, versioned location linked to the dataset and model record. Close it by moving results into a versioned evidence store.

Questions buyers ask about A.7.4

How much effort does A.7.4 data quality take to meet?

The effort depends mostly on how many AI systems use data, how many datasets each one draws on, and whether quality requirements already exist in any form. A.7.4 asks the organization to define and document data quality requirements and show that the data used to develop and operate each AI system meets them. Teams with data governance practices in place usually extend them to AI datasets, while teams starting from nothing spend most of their effort agreeing on dimensions such as completeness, accuracy, consistency and timeliness, and on the thresholds that count as a pass. Ask any partner for a quote built from a written list of systems, datasets and pipelines in scope, since that list determines the work.

Can we meet A.7.4 in-house without an outside partner?

Yes, an organization can meet A.7.4 with its own staff, because the control is about the organization's own requirements and records. The work needs people who understand the data, people who can run measurements, and someone who owns the documented requirements. Outside help is most useful when the internal team lacks experience with data quality measurement, when several AI systems share pipelines that need one consistent approach, or when the timeline is tight. Whatever you choose, the organization owns the requirements and the decisions. An implementation partner can prepare systems, pipelines and evidence, but it does not decide whether your data is good enough for your use.

How long does it take to prepare A.7.4 evidence for an audit?

The duration depends on how far your data pipelines already produce measurable records. Preparing evidence takes longer when quality checks are manual, undocumented or run only once, and it moves faster when checks run automatically and their results are stored. Plan time for four things: writing the quality requirements, setting up measurements, running them across the in-scope datasets, and recording what happened when data fell below a threshold. Ask your certification body or auditor which evidence period it expects, because a single snapshot is weaker than a history of results. No timeline should be accepted without a written scope that names the datasets and systems involved.

What does A.7.4 need from data and ML engineering teams?

A.7.4 needs data engineers and ML engineers to make quality visible in the pipeline rather than only in a report. Data engineers typically define checks on completeness, validity, duplication and freshness, and log their results. ML engineers check that training and evaluation data match the requirements, and that the data used in production inputs stays within the documented thresholds. Both groups need a shared definition of each dimension, a named owner for each threshold, and a remediation path when a check fails. Confirm with your teams which tools already record check results, since that record is the evidence an assessor will ask to see.

Can a consultant or implementation partner replace the auditor for A.7.4?

No. A consultant or implementation partner can help you design quality requirements, set up measurements and collect evidence, but it cannot issue an audit report or certificate. Under ISO/IEC 42001, certification is granted by an external certification body, and ISO does not certify organizations itself. DeployOpen prepares systems, controls and evidence for assessments, and it does not issue audit reports, attestations or certifications. Keep the roles separate in your contract and in your project plan, and confirm that the certification body you select is independent of the firm that helped you prepare.

Is A.7.4 mandatory for every organization using AI?

A.7.4 is not automatically mandatory. ISO/IEC 42001 does not require every Annex A control, and clause 6.1.3 requires a statement of applicability that lists the necessary controls and justifies including or excluding each one. Whether A.7.4 applies depends on your AI systems, the data they use and the risks they create. An organization that trains or operates AI on data whose quality affects outcomes will usually need to address it, while an organization that only consumes a third-party service may justify a different treatment. Record the decision and its reasoning in your statement of applicability, and have your auditor review it.

How does A.7.4 relate to the EU AI Act quality management system?

A.7.4 is part of an AI management system, while the EU AI Act's quality management system is a separate obligation that the European Commission says ISO/IEC 42001 does not match. The Commission states that ISO/IEC 42001 helps you set up an AI management system but does not replace the quality management system the Act requires. CEN and CENELEC approved EN 18286 for that purpose in June 2026, and the Commission cited EN 18286:2026 in the Official Journal on 14 October 2026. Work done for A.7.4, such as documented quality requirements and recorded checks, can support parts of a quality management system, but you should map it to the Act's requirements with your legal and compliance advisers rather than assume it covers them.

How DeployOpen prepares data quality evidence

DeployOpen plans, implements and documents the self-hosted systems and checks that produce data quality evidence for A.7.4, and it does so inside the client's own infrastructure, whether that is their cloud accounts, data centers or air-gapped networks. The work covers the pipelines that move training, tuning and operational data, the validation jobs that test it against the organization's quality requirements, and the records that show those jobs ran. DeployOpen does not issue audit reports, attestations or certificates. Those come only from an independent auditor or certification body, and the organization owns the decision to accept or reject a dataset against its own thresholds.

Most engagements are fixed-scope. Before any work starts, the parties agree in writing which data sources and AI systems are covered, which quality dimensions and thresholds apply, which checks and logs will be built, what documentation will be delivered, and what the organization must provide, such as data owners, access and sign-off authority. Ask for that written scope to name each deliverable and the evidence it produces, so you can test it against what an assessor will request. Ongoing support, upgrades and continuous testing are available as retainers, which suits datasets that change often and need their checks re-run after each change.

Before you engage any implementation partner, confirm three points with the vendor or your own team: which data quality tooling the platforms in scope support, who on your side will approve thresholds and remediation decisions, and how the evidence will be stored and retained for the assessor. A partner can prepare these systems and records, but accountability for the quality requirements and the ISO/IEC 42001 statement of applicability stays with your organization.

How DeployOpen scopes and implements A.7.4

DeployOpen helps organizations define the data quality requirements behind A.7.4 and build the measurement, thresholds, remediation records and evidence that show those requirements are met. The organization keeps ownership of its controls and its data, while DeployOpen prepares the systems and documentation inside the client's own infrastructure. Independent auditors or certification bodies issue any report or certificate.

How we scope

Agree in writing which AI systems, datasets and pipelines fall inside the A.7.4 scope.
Document the quality dimensions and thresholds the organization must define and approve.
Confirm which parts DeployOpen builds and which the organization's own data owners will run.

How we build

Set up automated quality checks in the client's own environment for the agreed dimensions and datasets.
Configure logging of quality measurements and threshold breaches so each result can be traced.
Test the checks against known good and known bad samples before they gate any training or operational data.

How we hand over

Deliver the quality requirements, measurement procedures and threshold register as versioned documents.
Provide remediation records showing how failed checks were investigated, corrected and re-measured.
Walk the organization's owners through the evidence an independent assessor is likely to request.

How we support

Offer ongoing support as a retainer to maintain checks, update thresholds and document changes.
Review quality results after model or data source changes, within the agreed support scope.
Confirm with the organization's auditor which evidence formats and sampling they expect before handover.