What NIST AI 600-1 asks for on harmful bias and homogenization
NIST AI RMF and its Generative AI Profile, NIST AI 600-1, are voluntary, and there is no certification that confirms an organization handles harmful bias correctly. The work is evidenced through the organization's own testing records, the decisions it documents from them, and any independent assessment it chooses to commission. An assessor will ask for proof of testing, not a badge. Start by confirming with your legal and risk teams which bias obligations apply to your system, since the framework itself does not set them.
The profile defines harmful bias or homogenization as the amplification of historical, societal and systemic biases, performance disparities between subgroups or languages, and undesired homogeneity that skews system or model outputs. It notes that a large language model may perform less well for non-English languages or certain dialects, and that over-reliance on synthetic training data can cause model collapse. Cost and effort depend on three things: the system's scope and the decisions it informs, the number of languages and user groups it serves, and how much TEVV evidence already exists. A team with mature evaluation records has far less to build than one starting from nothing. When requesting a quote, ask what share of the work is new test design versus running and documenting existing evaluations.
The core tests sit under MEASURE 2.11 of the AI RMF Core. The profile's first action, MS-2.11-001, asks teams to apply use-case appropriate benchmarks that quantify systemic bias, stereotyping and hateful content in outputs. The second, MS-2.11-005, asks teams to check the proportion of synthetic data in training sets, which addresses model collapse concerns. Keep dated benchmark results per language and per user group, along with the rationale for each metric chosen. Confirm with your assessor which of these records they expect to see, and which of the Playbook's suggested fairness metrics your use case warrants.
How MEASURE 2.11 tests harmful bias step by step
The sequence moves from defining which harms and groups matter, to measuring them, to examining the data that produced the system, and finally to checking the synthetic data share. Each step leaves a record that an assessor or internal reviewer can inspect. The Playbook suggestions are a menu for this work, not a checklist.
Identify harm types and affected groups
The AI or product team, with input from people who understand the use case and the communities it touches, first names the harm types the Playbook lists: allocational, representational, quality of service, stereotyping and erasure. For each harm type, the team identifies the groups that might be harmed in this specific deployment, such as language communities, dialect speakers, or demographic groups affected by the output. The output is a harm register that records each harm type, the groups involved, the reason each one applies to this system, and who reviewed the list. Your evidence that this step is done is a signed register that later steps can reference.
Select fairness and context-specific metrics
Next, the team chooses how each harm in the register will be quantified. The Playbook points to general fairness metrics such as demographic parity or equalized odds, and to context-specific metrics that fit the task. A generative system may need measures of output quality or refusal behavior across languages instead of classification error rates, so the choice depends on what the system produces. The output is a metric plan that names each metric, the harm it measures, the population slices it applies to, and the reason it fits. Record any metric considered and rejected, with the reason, so reviewers can see the choice was deliberate.
Analyze bias in training and TEVV data
The team examines the data used to train or adapt the model and the data used for testing, evaluation, verification and validation (TEVV). It checks how groups, languages and dialects are represented, where labels or sources may encode historical or societal bias, and whether test sets cover the same groups as the intended deployment. Where the organization did not train the model, this analysis relies on documentation from the provider and on what the provider will disclose. The output is a data bias report that lists the representation gaps found, the sources reviewed, and the remediation or residual risk accepted for each gap.
Find proxy features for group membership
The team searches for input features, prompts, metadata or retrieved content that correlate with protected or sensitive group membership, even when group labels are removed. In generative systems this includes names, locations, writing style, or language choice that can steer outputs. Findings are tested by comparing outputs with and without the suspect feature, and by checking whether the feature changes results across groups. The output is a proxy analysis that lists each candidate feature, the test used, the result, and the mitigation chosen, such as removal, restriction or documented acceptance by the accountable owner.
Quantify harms against the metrics
The team runs the chosen metrics on the evaluation sets for each group and language identified in the register. Results are compared across groups, and gaps are recorded with the sample sizes behind them, since small samples can make differences look larger or smaller than they are. Where a gap exceeds a threshold the team set in advance, it is escalated under the risk management process. The output is a measurement results record that ties each number to its metric, group, dataset version and test date, so the result can be reproduced later.
Apply use-case benchmarks for MS-2.11-001
Under MS-2.11-001 in NIST AI 600-1, the team applies benchmarks suited to the use case to quantify systemic bias, stereotyping and hateful content in generated outputs. Teams choose benchmarks that match the tasks and languages in scope, because a general benchmark may not reflect the deployment. Scores are recorded against the same groups and languages used earlier so results can be compared over time. The output is a benchmark report that names each benchmark, its version, the outputs tested, the scores and any limits the team noted. Benchmark results support the findings but do not replace the harm-specific analysis above.
Check the synthetic training data proportion
Under MS-2.11-005, the team checks what proportion of the training data is synthetic, that is, generated by models rather than collected from people or records. Because over-reliance on synthetic data can contribute to model collapse, the team compares the synthetic share against the amount it considers acceptable for the system and records the source of each synthetic dataset. Where the organization buys a pre-trained model, it asks the provider what synthetic data was used. The output is a synthetic data record that gives the share, the generating models, and the decision taken about the share, with a date and owner.
Document results and feed decisions
The team consolidates the register, data report, proxy analysis, measurement results, benchmark report and synthetic data record into one file tied to the system version. It confirms that each identified harm has a measurement, a result and a decision, and it sends unresolved gaps to the go or no-go and risk response process. Confirm with your legal and compliance leads which records must be retained and for how long, since no retention period is set by the researched facts. The output is a dated bias assessment package that an internal reviewer or independent assessor can follow from harm to evidence.
What evidence shows harmful bias is measured and managed
An assessor or customer asking about harmful bias usually wants proof that the organization defined the harms, measured them across groups and languages, and decided what to do about what remained. The cards below cover the artifacts that connect those steps, from the initial harm register to the records of residual risk.
Harm-type register and affected groups
This register lists each harm type the system could cause, such as allocational, representational, quality of service, stereotyping and erasure, and names the groups that could bear each one. It answers the basic question of what harm the organization looked for and why. Check that the register covers each language and user population the system serves, not only the primary market. Confirm that it was reviewed when the use case, user base or model changed.
Metric definitions and subgroup results
This artifact records which fairness metrics were chosen, such as demographic parity or equalized odds, and which context-specific measures were added for the use case. It also shows results broken out by subgroup and by language, including the gaps that were found. Ask whether the metric choice was justified in writing, since a single aggregate score can hide performance disparities between groups. Generative outputs also need benchmarks for systemic bias, stereotyping and hateful content, as the profile's MEASURE 2.11 actions describe.
Training and TEVV data bias analysis
This analysis examines the training data and the test, evaluation, verification and validation datasets for skewed representation, historical bias and gaps in coverage. It answers whether the data that shaped the model could produce the disparities seen in testing. Ask who reviewed the datasets, which sources were checked, and how findings were fed back into data selection or filtering. Third-party datasets and pre-trained models need the same review, with their documentation requested from the supplier.
Proxy feature findings
A proxy feature is an input that stands in for a protected or sensitive group membership, such as postal code standing in for ethnicity. This record shows which features were tested for proxy behavior, the method used, and what was found. It answers whether the system could reach a biased outcome without ever seeing the protected attribute. Confirm that the findings led to removal, transformation or documented acceptance of each proxy that was identified.
Synthetic data proportion records
Where synthetic data enters training, the organization should record its share of the training mix and the checks run on it. The profile links over-reliance on synthetic data to model collapse, so this record answers whether the proportion was limited and monitored. Ask for the version history of the training mix, the source of each synthetic dataset, and the tests used to detect degraded diversity in outputs over successive training rounds.
Residual bias decisions
After mitigation, some disparity usually remains. This decision record names the residual gaps, the groups affected, the owner who accepted or rejected the risk, and the conditions for re-review, such as a new language launch or a model update. It answers whether a person with authority made an informed choice about the remaining bias. Confirm that it is linked to the go or no-go determination and to post-deployment monitoring.
How the bias work differs for builders and buyers
An organization that builds or fine-tunes a generative model controls the most levers on harmful bias and homogenization. It chooses the training and fine-tuning data, the benchmarks used to test outputs, and the share of synthetic data that enters training. That control also makes it the party accountable for showing how those choices were made. Under the MEASURE 2.11 actions in NIST AI 600-1, the organization should be able to show how it looked for systemic bias, stereotyping and hateful content in its own outputs.
- Training data review: Document the sources, the coverage of languages and groups, and the proxy features checked during analysis of training and TEVV data.
- Use-case benchmarks: Select benchmarks suited to the intended use and record the results for systemic bias, stereotyping and hateful content.
- Synthetic data controls: Track the proportion of synthetic data in each training run, since MS-2.11-005 ties this to model collapse concerns.
- Harm and group inventory: Record which harm types (allocational, representational, quality of service, stereotyping, erasure) and which groups were considered for each release.
- Owned evidence: Keep the datasets, evaluation runs and decisions, because no vendor can supply them on the organization's behalf.
An implementation partner typically prepares the evaluation harness, the data lineage records and the evidence templates, while the organization signs off on the benchmark choices and risk decisions.
An organization that deploys a third-party model owns the use case but not the model's training data or internal benchmarks. It cannot fully test how the vendor curated the data, so its main job is to request evidence and then test the system in its own setting. NIST AI 600-1 notes that generative AI may be inappropriately trusted to perform similarly across subgroups, so a vendor's general results should not replace the buyer's own tests.
- Vendor evidence requests: Ask for documentation of training data sources, bias evaluations, the benchmarks used and any known performance gaps by language or group.
- Own use-case testing: Run tests that reflect the actual inputs, users and decisions, and record harm types and affected groups for that deployment.
- Language coverage checks: Confirm how the model performs in each language and dialect your users write in, since an LLM may perform less well for non-English languages or certain dialects.
- Change monitoring: Ask how the vendor notifies customers when the model version, training data or safety tuning changes, then retest after each change.
- Contract terms: Seek rights to receive evaluation updates and incident information that affect bias findings.
An implementation partner typically prepares the test plan, the test datasets and the evidence pack, but the vendor's cooperation determines how much of the evidence can be obtained. Confirm this in the contract before deployment.
An organization serving several languages or regions carries the performance gaps that NIST AI 600-1 describes across each one. A model can look sound in its main language and still underperform for other languages, dialects or regional user groups. The organization therefore needs a per-language view of results, not one aggregate score, and it should define which languages are in scope for each control.
- Per-language baselines: Measure outputs separately for each supported language and dialect, using the same harm categories throughout.
- Regional context: Adapt the harm and group lists to local legal and social contexts, and record where a region is out of scope.
- Quality of service comparisons: Compare error rates and response quality across languages and flag gaps that exceed the level the organization has accepted.
- Homogenization checks: Look for outputs that flatten viewpoints or styles across regions, since homogenization can flow from a foundation model into every downstream system.
- Ongoing sampling: Sample live traffic by language and region to catch drift that pre-release tests missed.
An implementation partner typically builds the per-language test sets, the sampling pipeline and the reporting dashboards. Ask the organization to confirm which languages it must support and which gaps it will accept, because those decisions are its own.
How the planned AI RMF revision and Texas law frame this topic
NIST says AI RMF 1.0 is being revised under the White House AI Action Plan, and as of 17 October 2026 it has not published a revised version. The July 2025 Action Plan recommends that the Department of Commerce, through NIST, remove references to misinformation, diversity, equity and inclusion, and climate change from the framework. The Playbook is also due for an update once the framework is revised, so its current wording for MEASURE 2.11 may change. Teams should record the version of AI 100-1, the Playbook and the Generative AI Profile they relied on when they assessed bias, and should check NIST's AI RMF pages for the revision as it is released. Any reference to bias testing in internal policy should cite the version in force on the date of each assessment.
Revision risk matters most where a policy or contract names a specific version or wording. Confirm with the business owner and counsel which documents cite the AI RMF, and whether those citations point to version 1.0 by name. Keep the bias measurement methods that your organization chose, including the harm types, groups and fairness metrics, in your own documentation so they remain defensible if the Playbook wording changes. The underlying concern, which is that AI systems can produce unequal results across groups and languages, does not depend on the revised text.
Texas Section 552.056, enacted through HB 149 (the Texas Responsible Artificial Intelligence Governance Act), bars developing or deploying an AI system with the intent to unlawfully discriminate against a protected class. It also states that a disparate impact is not sufficient by itself to show intent to discriminate. For bias testing, this means measured performance gaps across subgroups or languages are evidence to investigate, but the statute's intent standard turns on what the organization knew and did. Ask counsel to interpret how this section applies to your deployments, including which systems fall within Texas jurisdiction and how your testing records would be read. The statute text is available in the enrolled HB 149.
What makes harmful bias testing larger or smaller
The effort behind MEASURE 2.11 testing depends less on the bias method itself than on how much ground the system covers. The cards below cover the scope drivers that most often expand or shrink a harmful bias and homogenization program, along with the controls an organization can apply to each.
Number of languages and dialects
NIST AI 600-1 notes that an LLM may perform less well for non-English languages or certain dialects, so each language a system serves is a separate performance surface to test. Each added language needs its own benchmarks, reviewers and evaluation data. To control this, launch with a short list of languages that have validated test sets, and document which languages are out of scope and how users are told.
Number of user groups and use cases
Each distinct user group and use case adds potential harm types and affected groups that the Playbook asks organizations to identify. A system used for drafting marketing copy and one used to summarize benefits claims carry different harm profiles even on the same model. Narrowing the approved use cases before testing starts is usually the most effective way to limit scope, with unapproved uses blocked by policy and technical controls.
Foundation model or downstream application
NIST AI 600-1 says homogenization can flow from foundation models to downstream systems, which makes the origin of the model a scope question. A team that builds an application on a third-party model must test its own outputs and also collect what the model provider documents about its bias evaluations. Ask providers for their evaluation reports and what they cover, then decide which tests you must repeat yourself.
Volume and provenance of data
Training and TEVV data determine what the bias analysis can see. Large, poorly documented corpora take longer to analyze for skew, and the Playbook asks teams to analyze bias in both training and TEVV data. Synthetic data adds another check, because NIST AI 600-1 links over-reliance on synthetic training data to model collapse and points to checking the proportion of synthetic data under MEASURE 2.11. Record the source, licence and synthetic share of each dataset.
Maturity of evaluation pipelines
A team with automated benchmark runs, versioned test sets and stored results can rerun bias tests after every model or prompt change. A team without these pipelines must build them first, which is a one-time cost that then reduces the recurring effort of each test cycle. Before scoping, ask what evaluation tooling already exists, who maintains it, and whether results are retained for comparison across releases.
How a harmful bias evaluation moves from scoping to evidence
This sequence turns the harmful bias and homogenization concern in NIST AI 600-1 into a documented evaluation, from the systems in scope through subgroup and language test results to a named decision on mitigation or residual risk that an assessor or customer can review.
Scope the systems and use cases
The organization, usually the product or AI governance owner, lists each generative AI system in scope, every foundation model or third-party component it depends on, the use cases and user populations, and the languages and dialects in which the system operates. Engineering and the business owner confirm the list. The output is a scoping record tied to the AI inventory, and the step is done when every deployed use case has a named owner and a stated population. Teams should also confirm which downstream systems inherit outputs from a shared foundation model, since homogenization can flow from that model to them.
Map harm types and affected groups
The team identifies the harm types relevant to the use case, such as allocational, representational, quality of service, stereotyping and erasure, and the groups that could be harmed. Domain experts and representatives of affected communities provide input through structured sessions or feedback channels, and their input is recorded with the reasons it changed or did not change the scope. The output is a harm register that lists each harm type, the affected groups, and the sources of evidence for each. The step is complete when the register has been reviewed by someone outside the build team.
Select benchmarks and fairness metrics
The evaluation lead chooses use-case appropriate benchmarks that measure systemic bias, stereotyping and hateful content in outputs, along with general fairness metrics such as demographic parity or equalized odds where they fit the task, plus context-specific metrics for the harms in the register. Each metric is paired with a stated reason and a known limitation. The output is a metric specification that names the subgroups and languages each metric covers. The step is done when a reviewer can trace every harm in the register to at least one metric or documents why none exists.
Analyze training and test data
The data team examines the training data, evaluation sets and any synthetic data for representation gaps, skewed label distributions and features that could act as proxies for group membership. For synthetic data, the team records the proportion of synthetic content, which is the check NIST AI 600-1 ties to model collapse concerns. The output is a data analysis report with findings and the data changes or exclusions they triggered. The step closes when every proxy feature found is either removed, justified or tracked as a known risk.
Run tests and record subgroup and language results
The evaluation team runs the selected benchmarks and metrics against the system as deployed, not only against the base model, and reports results for each subgroup and each language or dialect in scope. Results are recorded with the test version, model version, configuration and date so they can be reproduced. Where a language performs less well, the gap is recorded as a finding rather than averaged away. The output is a test record with raw results, the thresholds applied and any failures. The step is done when every metric in the specification has a recorded result or a documented reason it could not run.
Decide on mitigation or residual risk
A named risk owner, with input from legal, product and the affected-community representatives where relevant, reviews the test record and decides whether each finding is mitigated, accepted as residual risk, or blocks release. Mitigations such as data changes, output filtering, language-specific evaluation or human review are assigned owners and re-tested. The output is a signed decision record that states the residual risk and who accepted it. The step is complete when no finding is left without a decision and an owner.
Keep records for assessors and customers
The organization assembles the scoping record, harm register, metric specification, data analysis, test record and decision record into one evidence package, keyed to the relevant AI RMF subcategories and to the generative AI actions it addresses. The package is version-controlled and refreshed when the model, data or use case changes, and the team confirms with its independent assessor or customer reviewer which format they expect. The output is a maintained evidence set. Note that an implementation partner can prepare this package, but only an independent auditor or assessor can issue an opinion on it.
Where harmful bias programs usually stall
Harmful bias programs rarely fail because teams ignore fairness. They stall on measurement shortcuts, unexamined language and data assumptions, and unclear ownership. Each card below names one failure, why it weakens the evidence behind MEASURE 2.11 and the Generative AI Profile, and the check that closes the gap.
Trusting one aggregate accuracy score
A single overall accuracy or quality figure can look healthy while one subgroup, language or dialect performs far worse. The Playbook's MEASURE 2.11 actions call for quantifying harms with general fairness metrics such as demographic parity or equalized odds, plus context-specific metrics. Check that results are broken out by group, by language and by use case, and that the document states which metric was chosen and why.
Testing only in English
AI 600-1 notes that an LLM may perform less well for non-English languages or certain dialects, and that teams may wrongly assume similar performance across subgroups. A test set written and scored only in English leaves that gap invisible. Confirm which languages and dialects your users actually write in, whether benchmarks exist for each, and whether reviewers can judge outputs in those languages.
Ignoring proxy features
A model can reproduce bias without receiving any protected attribute, because postal code, school, job title or language patterns can stand in for group membership. The Playbook suggests finding features that act as proxies and analyzing bias in training and TEVV data. Ask the team to list which input features correlate with sensitive characteristics, and record the decision made about each one.
No record of synthetic data share
When training or evaluation data includes model-generated content, homogenization and model collapse become a concern. AI 600-1's MEASURE 2.11 actions include checking the proportion of synthetic training data. Many programs cannot answer the question because provenance was not logged. Keep a dataset register that records the synthetic share for each training and fine-tuning run, with its source and date.
Unclear ownership between teams
The model team often runs the metrics, while the business owner decides whether an outcome is acceptable. When neither side owns the harm threshold, findings are logged but not acted on. Assign a named business owner to accept or reject each bias finding, and a named model owner to fix or document it. Record both sign-offs in the same file so the decision trail is complete.
Questions buyers ask about harmful bias under NIST AI 600-1
Is NIST AI 600-1 certifiable for harmful bias?
No. NIST AI 600-1 is the Generative AI Profile of the NIST AI Risk Management Framework, and the AI RMF is voluntary. NIST does not validate or endorse any individual organization or its approach to using the framework, and no certification, accreditation or conformity assessment program exists for it. What a buyer can ask for is evidence: documented harm types, bias metrics, test results and the decisions made from them. Any party claiming a NIST AI 600-1 certificate should be asked to name the issuing body and the standard it tested against.
Does a contract or regulator require a NIST AI 600-1 bias audit?
Usually the requirement comes from the contract, a customer questionnaire or a specific state or sector rule, not from NIST itself. Because the framework is voluntary, no federal rule mandates an audit under it. Counsel should read the exact wording of each clause and any statute cited, including Texas Section 552.056 if it applies, which bars developing or deploying an AI system with intent to unlawfully discriminate and states that disparate impact alone is not sufficient to show intent. Confirm with counsel what the clause actually asks for, and whether it names an independent assessor.
What drives the cost and effort of bias testing?
Effort depends on how many use cases, languages and user groups the system serves, how many harm types you must cover, and whether you train or fine-tune models or only deploy a vendor model. Costs fall into staff time for mapping harms and choosing metrics, compute and evaluation data, subject matter review of outputs, and documentation. Most of this is recurring, because testing must repeat after model updates, prompt changes and new data. Ask any provider for a written scope listing the systems, languages, subgroups and metrics covered, the deliverables and the retest triggers. NIST does not publish a price for this work, so treat any figure you see as a quote for your scope only.
Can our own team test for harmful bias without outside help?
Yes, in principle. NIST's Playbook suggestions for MEASURE 2.11 are voluntary and can be carried out by an internal team, provided it has the skills to identify harm types, select fairness metrics such as demographic parity or equalized odds, analyze training and evaluation data, and find proxy features. The harder parts are independence and coverage: the team that built a system is poorly placed to find its own blind spots, and language and dialect gaps need reviewers who speak those languages. Many organizations keep ownership of decisions and bring in outside testers for an independent pass. DeployOpen performs security testing of AI systems and agents, but it does not issue audit reports or attestations.
What should we ask a model vendor about bias before we buy?
Ask for evidence rather than assurances. Request the intended use cases and out-of-scope uses, the benchmarks used to measure systemic bias, stereotyping and hateful content, and the results broken down by language and dialect where your users operate. Ask how the vendor handles the proportion of synthetic data in training, since NIST links over-reliance on synthetic data to model collapse, and what changed in each model version. Also ask for documented known limitations, the process for reporting harms, and whether they will notify you of material changes. Put these requests in the contract and confirm with counsel which warranties and audit rights you can secure.
How does the planned NIST revision affect bias work?
NIST says AI RMF 1.0 is being revised as part of the White House AI Action Plan and has not yet published a revised version. The July 2025 AI Action Plan recommends that NIST remove references to misinformation, diversity, equity and inclusion, and climate change. Because the Generative AI Profile's bias actions relate to harmful bias and homogenization, your documentation should track which version of the framework you cite. Confirm with counsel whether any law or contract names a specific version, and plan for a change in terminology while keeping the underlying measurements that show fairness performance.
What evidence will an assessor or customer expect to see?
Expect a documented harm analysis, the groups and languages considered, the metrics chosen and why, test datasets and their composition, and results by subgroup and language. Also expect records of the decisions taken, such as go or no-go determinations, mitigations, and residual risk accepted by a named owner. Monitoring records after deployment matter as well, since the Generative AI Profile treats bias as something that can emerge over time. Confirm with your assessor or counsel which of these items they require, the format they accept and how long records must be retained.
Who pays for bias testing and who owns the results?
Your organization pays for everything it commissions, including internal staff time, tooling, external testing and any remediation. Costs are one-time for the initial harm mapping and test design, and recurring for re-testing after changes and for production monitoring. Ownership should be settled in the written scope: the organization owns its controls, decisions and resulting evidence, while the implementation or testing partner delivers the prepared systems, test artifacts and documentation. Only an independent auditor or assessor issues a report or opinion. Confirm the handover terms, data retention and access rights before work starts.
How DeployOpen prepares bias evidence for your assessment
DeployOpen prepares the systems, test environments and evidence that your own bias testing relies on, under a fixed scope agreed in writing before work starts. In practice that means building or configuring the evaluation environment inside your infrastructure, setting up repeatable runs of the benchmarks and subgroup comparisons you select, keeping the datasets, prompts, configuration and results under version control, and handing over the records in a form your team and your assessor can read. Typical scope decisions include which languages and subgroups are covered, which harm types from the Playbook are in view, which metrics are computed, and how often the runs repeat after a model or prompt change. Each of these choices should be written down before testing begins, because it determines what the evidence can and cannot show.
You own the controls, the risk tolerance, the interpretation of results and every go or no-go decision that follows from them. DeployOpen does not decide whether a disparity is acceptable, does not certify that your system is unbiased and does not guarantee any measured result. Where your assessor or auditor needs independent review of the bias evidence, only that independent party issues a report or opinion. Your internal reviewers, legal counsel and any affected stakeholders still need to sign off on the conclusions. Before engaging, ask the provider to confirm in writing who holds the raw test data, who approves changes to the evaluation plan and what remains your responsibility after handover.
Security testing belongs in the same conversation only where it touches the bias evidence. If adversarial prompts or data poisoning could skew subgroup results, the test plan should say so and reference the separate security testing scope. Ask for a list of every artifact handed over, including the evaluation code, the datasets or their hashes, the run logs and a short method note that explains the metrics used. Confirm the retention period and where the records will live after the engagement ends, so the evidence remains available to your assessor in the next review cycle.
