MEASURE 1.3 independent review: requirements and evidence

What NIST AI RMF MEASURE 1.3 asks for and why it matters

An auditor, customer questionnaire or internal review may ask a simple question: is your AI testing independent? MEASURE 1.3 is the subcategory in the Measure function of the NIST AI RMF that addresses this. It expects that internal experts who were not front-line developers of the system, independent assessors, or both, take part in regular assessments and updates. It also calls for consulting domain experts, users, external AI actors and affected communities as the organization's risk tolerance requires. The subcategory describes an outcome, not a fixed process, so the organization decides how often, how deeply and with whom it consults. NIST does not certify, validate or endorse any organization against the AI RMF, so a claim of MEASURE 1.3 alignment is a claim the organization must support with its own records.

The risk behind this subcategory is that the people who build a system are poorly placed to judge whether it works. Section 5.3 of NIST AI 100-1 notes that independent review can improve the effectiveness of testing and can mitigate internal biases and potential conflicts of interest. A developer who tuned a model may test it against the same assumptions that shaped it. Small teams find this hard because the pool of people who understand the system but did not build it is often one or two engineers, and budgets rarely fund outside assessors for every release. The practical response is to decide which systems and risk tiers require independence, name who provides it (an internal reviewer outside the build team, an external assessor, or both), and record that decision.

A workable starting point is a short register that lists each AI system, its risk tier, the testing role assigned to each reviewer, and the consultation steps taken with domain experts, users or affected communities. Confirm with your own risk owners what tolerance applies, because the subcategory ties consultation to that tolerance rather than to a fixed list. The NIST AI RMF Playbook lists suggested actions for this subcategory, including assessing the independence, stature and resources of the people doing testing and oversight. Those are judgments you must document. Budget for independent reviewers as a recurring cost, since each major model change or new use case may need another review.

Which AI systems, teams and third parties MEASURE 1.3 reaches

MEASURE 1.3 asks who checks the AI system and who gets consulted, and the answer depends on what the organization can tolerate losing. The cards below cover the scoping decisions that shape an independent review program, from which systems qualify to who may count as independent. Each decision should be written down and tied to the testing teams the Playbook links to GOVERN 2.1 and 4.1.

Systems that need independent review

Select systems by the impact they can have on people and by the organization's risk tolerance, not by whether they use a neural network, a rules engine or a large language model. Start with systems that make or strongly shape decisions about individuals, affect safety or critical services, or would be costly to get wrong. Record the tier for each system in the AI inventory so the review frequency and reviewer type follow from a documented rule. Confirm with the business owner which harms the tolerance statement covers.

People who are not front-line developers

MEASURE 1.3 expects internal experts who did not build the system to take part in regular assessments and updates. Someone who wrote the model code, tuned its prompts or owns its release pipeline is too close to judge it alone. Staff from a separate security, model risk, quality or compliance function can qualify if they have enough independence from the delivery schedule. Ask who can stop a release and whether that person reports outside the delivery line.

When an external assessor is warranted

Bring in an independent assessor when internal reviewers share the delivery team's incentives, when the system carries high impact, or when a customer, regulator or contract asks for outside testing. AI 100-1 Section 5.3 notes that independent review can improve testing effectiveness and reduce internal bias and conflicts of interest. An external assessor issues findings but does not own the controls, so the organization still decides on remediation and risk acceptance. Ask the assessor to state their independence from the implementation work.

Third-party models and generative AI services

Vendor models and hosted generative AI services still fall under MEASURE 1.3 when your organization deploys them. Confirm which evaluations the vendor provides, which ones you must run yourself, and whether the vendor's testers are independent of the model's development. For generative AI, NIST AI 600-1 action MS-1.3-003 asks you to verify that people running structured human feedback exercises are not directly involved in developing the same model. Write these split responsibilities into the contract.

Affected communities and domain experts

Identify affected communities by mapping who the system touches, including people who are not customers, such as applicants, patients, staff under monitoring or residents in an area served. Pair that list with domain experts who know the legal, clinical or operational context and with end users who see the system's output daily. Record why each group was chosen and what they are asked to review. Under the risk tolerance set in GOVERN, decide which groups must be consulted before a release and which may be consulted after one.

Testing teams from GOVERN 2.1 and 4.1

The Playbook suggests reusing the separate testing teams established under GOVERN 2.1 and 4.1 for MEASURE 1.3, and assessing the independence, stature and resources of those TEVV and oversight actors. Check whether each team has written authority, a named lead, time allocated outside delivery deadlines and access to the system's logs and documentation. Also evaluate whether the feedback channels for external stakeholders are working, since reviewers need those inputs to judge impact.

How assessors test independent review and expert consultation

This table compares what an assessor does, what evidence they ask for and where organizations usually fall short on each part of MEASURE 1.3. Read it as a checklist for preparing your own review file before an assessor arrives, and note that NIST AI 100-1 describes the outcome, not a fixed test procedure.

Test area	Evidence requested	Common findings
Reviewer separation from development	Org chart or role register for each reviewer, development and deployment history for the system, and a written statement of who approved the review scope	Reviewers are on the same team as the model builders, or the same engineer signs both the build and the review, so independence is asserted but not shown
Independence, stature and resources of TEVV and oversight actors	Charter for the testing team, reporting line to a function outside delivery, staffing and budget records, and evidence of authority to halt a release	Testing staff report to the product owner, lack a documented right to block deployment, or have no protected time for assessments
Consultation against documented risk tolerance	Risk tolerance statement, criteria that trigger consultation with domain experts, users or affected communities, and records showing which triggers fired	Consultation happens informally or only for launches, with no link to the stated tolerance, so the assessor cannot tell why some groups were consulted and others were not
Effectiveness of external feedback channels	Intake logs for external feedback, response times recorded in the log, examples of changes traced to feedback, and a review of whether the channel reaches affected groups	Feedback forms exist but nothing is logged, or no one reviews the channel for reach, so the organization cannot show the channel works
Human feedback exercises run by independent people (MS-1.3-003, NIST AI 600-1)	Roster of people running structured human feedback exercises, their roles in building the same generative model, and a signed attestation of non-involvement	Exercise leads also tuned prompts or fine-tuning data for the same model, or the roster is missing, so independence cannot be verified
Documentation of consultation outcomes	Meeting notes or written responses from consulted experts, decisions taken, and reasons for any input that was not adopted	Input is gathered but no record shows how it changed the system or why it was declined

How to set up independent review in six steps

This sequence turns MEASURE 1.3 from a principle into a working review process, with named reviewers, documented independence and a dated record of expert and community input. Each step produces an artifact that an internal auditor, customer or regulator can inspect.

Set review tiers from risk tolerance
Start by reading the organization's documented risk tolerance for each AI system, which should come from the GOVERN 1 policy and the MAP 1 context record. Group systems into review tiers, so that systems with higher potential impact on people receive deeper independent review and more frequent consultation. Owners are the risk function with the AI governance lead. The output is a tier table that lists each system, its tier, the review depth it receives and the trigger that moves a system to a higher tier. The step is done when every system in the AI inventory has a tier and a named approver for that tier.
Build the reviewer register
Create a register of everyone who will assess or challenge AI testing results. Include internal experts who did not build the system in question, and any independent assessors the organization engages. For each entry, record the person or firm, their domain, the systems they may review, and whether they are internal or external. Use the separate testing teams described under GOVERN 2.1 and 4.1 as the starting pool. The output is a reviewer register owned by the AI governance lead, reviewed whenever the team or vendor list changes.
Document reviewer independence
For each reviewer, assess independence, stature and resources. Ask whether the reviewer contributed to building, labeling, fine-tuning or deploying the system, whether they report to the team that owns it, and whether they have the authority to escalate findings. Have each reviewer sign a conflict-of-interest declaration for each system they will review, and record any recusal. AI 100-1 notes that independent review processes can mitigate internal bias and conflicts of interest, so this declaration is the evidence that the review is independent. The step is done when every assigned reviewer has a signed, dated declaration on file.
Assign reviewers to each assessment
Match reviewers to the regular assessments and updates for each system tier. Assign a primary reviewer who is independent of development and a second reviewer for higher tiers, so that no single person decides alone. Define what each reviewer examines, such as validity and reliability testing, safety results, or evaluation of harmful bias. Record the assignments in the assessment plan. The output is an assessment plan that names who reviews what, with the testing results each reviewer receives. The step is done when each scheduled assessment has at least one independent reviewer assigned.
Schedule consultation with domain experts, users and communities
Identify the domain experts, end users, external AI actors and affected communities whose input the risk tolerance requires for each system tier. Set a consultation schedule and agree the questions each group will be asked, such as whether outputs are accurate in their setting, where harm may occur, and what they need to know about the system. Confirm how each group can raise concerns, and test that the feedback channel works before relying on it. The Playbook suggests evaluating the effectiveness of external feedback mechanisms, so record how you will judge that. The output is a consultation calendar and a question set for each group. The step is done when each high-tier system has a scheduled consultation.
Record consultation and review outcomes
Log every consultation session and every review finding in a consultation log and a findings register. For each entry, capture the date, the participants and their roles, the questions asked, the concerns raised, and how the concern was resolved or passed to risk owners. Mark which findings came from independent reviewers and which came from external groups. Where a participant raised an issue that the team did not act on, record the reason. The step is done when each consultation and review has a dated entry with a named owner for every open concern.
Close the loop with a dated review record
Summarize each review cycle in a dated record that links the tier, the reviewers, their declarations, the consultations and the decisions taken. Send the record to the risk owner and the teams that build and operate the system, and feed open items into MEASURE 1.2 control reviews and MANAGE 1 risk decisions. Set the next review date and confirm who is responsible for it. The output is a review record that an auditor, customer or regulator can trace from a finding to a decision. Confirm with your own governance policy how often this cycle must run, because the NIST materials do not set a fixed interval.

Evidence that satisfies MEASURE 1.3 and how to keep it current

MEASURE 1.3 asks for evidence that the right people reviewed the system and that their input was recorded and acted on. The six artifacts below connect those reviews to named people, documented independence and a traceable decision trail. Refresh them on a schedule and after each material model change, not only before an audit.

Reviewer register

A register listing every internal reviewer and external assessor, with their role, reporting line and the AI systems they reviewed. For MEASURE 1.3, the register should show which people were not front-line developers of the system under review. Check that each entry names a reporting line that does not run through the development team, and that the register records who was added or removed since the last review.

Conflict-of-interest and independence declarations

Signed declarations from each reviewer stating any role in building, labeling, tuning or selling the system, and any relationship that could bias a result. AI 100-1 notes that independent review can mitigate internal biases and conflicts of interest, so these declarations are the proof of that claim. Confirm each declaration is dated, renewed for each assessment cycle, and updated when a reviewer changes teams.

Assessment plan with internal and external split

A plan that lists each planned review, its owner, and whether it is internal or independent. It should show which testing, evaluation, verification and validation (TEVV) activities the separate testing teams carry out and which go to an outside assessor. Assess whether the independence, stature and resources of each oversight actor are adequate, and record that assessment in the plan.

Consultation logs with dates and outcomes

Logs of each consultation with domain experts, users, external AI actors and affected communities. Each entry should record the date, the group consulted, the questions asked, and the outcome. Where a consultation was not run because risk tolerance did not require it, record that decision and the reason, so the absence is documented rather than silent.

Assessor report or review memo

The written output of each review, whether an internal memo or an independent assessor's report. Confirm the report states its scope, the methods used, the findings and the limits of the review. Only the independent auditor or assessor issues reports on its own work. Your team prepares the inputs and the system evidence the assessor examines.

Feedback-to-decision records

Records that show how each review or consultation finding changed the system, its controls or the risk decision. Link each finding to a change ticket, a model update, a revised risk acceptance or a documented decision not to act with its rationale. This is the artifact that proves expert input reached the outcome, so audit it for gaps where a finding has no recorded disposition.

What meeting MEASURE 1.3 costs in money and effort

This table lists the cost components of running MEASURE 1.3 reviews, who is paid for each, whether it is one-time or recurring, and what drives it up or down. Use it to structure a quote request, since no supplied figures exist for these costs and DeployOpen does not publish prices. The main takeaway is that open-source tooling removes license spend but not the labor of reviewers, assessors and stakeholder engagement.

Cost component	One-time or recurring	What drives it and how to keep it down
Internal reviewer time (staff who were not front-line developers)	Recurring	Number of systems in scope and how often reviews run. Keep it down by tiering systems by risk so lower-risk systems get lighter review cycles.
Independent assessor fees	Recurring, with a one-time setup for scope	Scope breadth, evidence quality and whether the assessor must re-sample. Keep it down by delivering a clean evidence package and a written scope before fieldwork starts.
Consultation and stakeholder engagement effort	Recurring	Number of domain experts, users and affected communities consulted and the depth of each session. Keep it down by reusing structured feedback channels and defining which groups the risk tolerance requires.
Testing and logging tooling and licenses	Mostly one-time setup, then recurring hosting and upkeep	Open-source tools such as garak (Apache-2.0) and Langfuse (MIT) carry no license fee, but hosting, storage and integration still cost money. Confirm license terms for any commercial add-on.
Evidence upkeep (records of reviews, findings and decisions)	Recurring	Volume of traces, test reports and review minutes, and how long they must be retained. Keep it down by defining evidence templates and automating exports from logging tools.
Retesting after model, data or configuration changes	Recurring, triggered by change	Frequency of model updates, vendor changes and new use cases. Keep it down by setting change triggers in advance so only affected tests rerun.
Implementation partner preparation work	One-time, plus optional retainer	Size of the existing evidence base, the number of systems and the depth of testing and documentation. Request a fixed-scope written quote, since DeployOpen does not publish prices and quotes follow from written scope.

Open-source tools that produce evidence for independent testing

Independent assessors and internal reviewers need records they can inspect, not summaries. The three tools below each produce a different kind of record, and none of them replaces the people who review it. Each card names the evidence it supports and the license terms to check before deployment.

garak model probing reports

garak probes language models for hallucination, data leakage, prompt injection, misinformation, toxicity generation and jailbreaks. It writes a JSONL report of every attempt and evaluation, plus a hit log of attempts that found a vulnerability. For MEASURE 1.3, the full report lets an independent reviewer check which probes ran and what was excluded, not only the headline result. Confirm the Apache-2.0 license terms with your legal team, and confirm that your probe selection matches the risks your organization mapped. A reviewer still has to judge whether the findings are material.

Langfuse traces for review

Langfuse records each trace of a request, including LLM calls, retrieval steps, tool executions and custom logic, with timing, inputs, outputs, token usage and metadata. Assessors can use these traces to reconstruct how a system behaved in production and to check whether outputs match the documented intended use. Ask who holds access to the traces, how long they are retained, and whether personal data in inputs is masked before a reviewer sees it. Langfuse is MIT licensed, which simplifies redistribution of the tooling itself, but it does not decide whether the recorded behavior is acceptable.

CISO Assistant assessment records

CISO Assistant ships more than 200 frameworks, including ISO 27001, NIST CSF, SOC 2, PCI DSS, NIS2, DORA and GDPR. It runs audits and compliance assessments against them with automatic mapping between frameworks. For MEASURE 1.3, this gives an independent reviewer one place to see which assessments were performed, by whom, and against which framework version. Confirm the AGPL-3.0-only terms, because network-facing modifications may trigger source disclosure duties. Automatic mapping is a starting point that a qualified person must validate before anyone relies on it for an assessment.

Who does what for NIST AI RMF MEASURE 1.3

Your organization owns the outcome MEASURE 1.3 describes. Someone accountable must appoint the internal reviewers who were not front-line developers of the system, decide how much consultation the risk tolerance requires, and accept or reject findings. These are decisions an implementation partner or assessor cannot make for you, and an assessor will expect to see them documented in your own governance records.

Appointing reviewers: Name internal experts who did not build the system, with written terms, time allocated and a reporting line that does not run through the development lead.
Owning risk tolerance: Set the thresholds that decide which domain experts, users, external AI actors and affected communities must be consulted, and record why.
Running consultation: Schedule and conduct the consultations, and keep the notes that show who was asked and what changed as a result.
Deciding on findings: Review each finding from internal and external reviewers, decide the response and record any residual risk you accept.
Funding the review: Budget for reviewer time, consultation and any external assessment, since these costs recur with each review cycle.

Before engaging anyone, confirm that your named reviewers and decision owners are documented in the GOVERN 2 accountability records.

Common gaps in independent review and how to close them

The gaps below recur across organizations that run AI assessments. Most come from who reviews, when experts are consulted, and whether the results are recorded and fed back. Each fix names the evidence that shows the gap is closed.

Reviewers who report to builders

When the people who assess a system report to the team that built it, their findings tend to bend toward delivery dates. MEASURE 1.3 asks for internal experts who were not front-line developers of the system, or independent assessors, or both. Check each reviewer's reporting line and whether they can hold a release. Record the independence decision in writing, including who approved it and when it was last reviewed. AI 100-1 notes that independent review can mitigate internal biases and conflicts of interest.

Consultation only at launch

Many programs consult domain experts, users and affected communities once, during design, then never again. MEASURE 1.3 expects these groups to be consulted as organizational risk tolerance requires, which implies repeated consultation as the system and its context change. Set consultation triggers, such as a new data source, a new user group or a material model change, and keep a dated log of each consultation and the decisions it informed.

No documented risk tolerance trigger

Without a written risk tolerance, nobody can say when a consultation or independent review is required, so the decision is made case by case and often skipped. Link the tolerance to specific thresholds, such as the impact categories or affected populations that require outside review. Confirm that the tolerance has been approved by the accountable owner, and keep the mapping from each tolerance level to the required review type in your AI policy records.

Feedback collected but not adjudicated

Teams often gather complaints, test results and stakeholder comments but never record how each item was judged or whether it changed the system. MEASURE 1.3 asks that feedback from consulted groups informs assessment and updates. For each item, keep a record that shows the decision, the reviewer who made it, the design change or the reason no change was made, and the date. Evaluate whether the feedback channels themselves work, using response times and resolved-item counts.

No record of human feedback exercises

Red teaming, annotation and structured human feedback exercises often run without a record of who participated, what they were told, or whether they helped build the model. For generative AI, the Generative AI Profile suggests verifying that the people running structured human feedback are not directly involved in developing the same model. Keep a roster of participants, their roles in development, the instructions used and the findings. Store this with the test records so an assessor can trace each result.

Unclear reviewer independence for third-party AI

When a model, dataset or tooling comes from a vendor, it is unclear whether the organization's testers are independent of that vendor, or whether the vendor's own evaluations are being accepted as independent. Define who counts as independent for each component, and state whether the vendor's test results were reproduced by a separate team. Assess the independence, stature and resources of each testing and oversight actor, and record the result before relying on it.

Questions buyers ask about MEASURE 1.3 and independent review

Can an organization get certified for MEASURE 1.3?

No. MEASURE 1.3 is a subcategory of the NIST AI Risk Management Framework, and the framework has no certification, accreditation or conformity assessment program. NIST does not validate or endorse any organization's use of the AI RMF. What a customer or regulator can reasonably ask for is evidence that your organization involves independent reviewers in regular assessments and consults the right people as your risk tolerance requires. Only an independent auditor or assessor can issue a report on that evidence, and no body issues an AI RMF certificate.

Can an internal reviewer count as independent under MEASURE 1.3?

Possibly, if the reviewer was not a front-line developer of the system being assessed. MEASURE 1.3 accepts internal experts who were not front-line developers, independent assessors, or both. NIST's guidance says independent review can mitigate internal biases and potential conflicts of interest, so the question is whether the reviewer's reporting line, incentives and access to the system allow an unbiased view. Document who reviewed what, their relationship to the build team, and who can overrule their findings. Assess the independence, stature and resources of the people doing testing and oversight, as the Playbook suggests.

When is an external assessor worth paying for?

An external assessor adds the most value when the system carries high stakes, when internal reviewers share incentives with the build team, or when a customer or regulator asks for independent evidence. Without a specific trigger, an internal reviewer with a documented separation from development may satisfy the outcome. For generative AI, NIST AI 600-1 suggests verifying that people running structured human feedback exercises are not directly involved in developing the same model. Ask any external firm to show how its work will be independent of the build, what it will test, and what it will hand over.

How much effort does expert consultation take, and what drives it?

Effort depends on the number of AI systems in scope, how many stakeholder groups must be consulted, and how much risk tolerance your organization has set in its governance documents. Costs are mainly staff time for internal experts, fees for external assessors or consultants, and the time domain experts, end users and affected communities spend giving input. Consultation is partly one-time, such as the first design review, and partly recurring, such as periodic reviews after material changes. Ask any quote to separate the scoping work, the number of review cycles, and the feedback channels it will set up.

What does the work need from our own team?

Your team owns the decisions, the risk tolerance and the evidence. You will need named owners for each AI system, access to model documentation, test results and logs, and a person with authority to act on reviewer findings. Domain experts must be identified, and end user and affected community feedback channels must exist and be monitored. An implementation partner can prepare systems, test environments and evidence folders, but it cannot make the risk acceptance decisions or serve as the independent reviewer of its own work. Confirm these responsibilities in writing before work starts.

Can a small team run MEASURE 1.3 in-house?

Yes, a small team can meet the outcome in-house if it can separate reviewers from developers and document the process. A practical setup is one reviewer who did not build the system, a written list of the experts and stakeholders to consult, and a recurring review calendar. The work becomes harder when a single person both builds and checks the system, or when the AI systems are customer-facing and affect many people. In that case, scope a limited external review of the highest-impact system. Platform work such as logging and test tooling can be set up with open-source components, but confirm each component's capabilities before relying on it.

How should we choose between an implementation partner and an auditor?

Choose an implementation partner to build, configure and document the systems and evidence, and choose an independent auditor or assessor to test and report on them. DeployOpen plans, implements, hardens and documents self-hosted open-source platforms inside your infrastructure, and runs security testing, including AI and agent security. DeployOpen prepares systems, controls and evidence for compliance assessments but does not issue audit reports or certifications. Ask each party for a written scope, deliverables and timelines, and avoid any arrangement where the party that built the system also signs the independent review.

What changes if NIST revises the AI RMF?

NIST says AI RMF 1.0 is being revised as part of the White House AI Action Plan and has not yet published a revised version, so confirm the current status on the NIST site before finalizing a program. The July 2025 AI Action Plan recommends removing references to misinformation, diversity, equity and inclusion, and climate change. NIST also expects a formal review with input from the AI community no later than 2028. Keep your MEASURE 1.3 evidence tied to your own risk tolerance and named reviewers, so that a change in wording requires an update to documentation rather than a rebuild of the process.

How DeployOpen scopes independent review and prepares its evidence

DeployOpen helps organizations set up the internal review and independent assessment arrangements that MEASURE 1.3 describes, and prepares the systems and evidence those reviewers need. Our role is to prepare, not to judge: the organization owns its decisions, and independent reviewers and assessors give their own conclusions.

How we scope

The written scope names which systems, AI use cases and assessment activities fall under MEASURE 1.3 review.
We agree which internal reviewers, testing teams and independent assessors are in scope before work starts.
Engagements are fixed-scope, with deliverables and timelines confirmed in writing; ongoing support is available as a retainer.

How we build

We help separate testing and oversight roles from the teams that develop the system, so reviewers are not grading their own work.
We set up self-hosted tooling for test runs, such as garak probe reports, and trace logs from Langfuse, that reviewers can inspect.
We document the independence, stature and resources of testing and oversight roles so the organization can review them over time.

How we hand over

You receive a documented review calendar, named reviewer roles and a log of which stakeholders were consulted and why.
Evidence packs map each review activity to its outputs, including test reports, findings, and records of feedback from domain experts and users.
Handover covers the records an independent assessor will ask to see, so the organization can share them directly with that assessor.

How we support

Retainers can cover recurring test cycles, updates to review procedures and refreshed evidence when systems or risk tolerances change.
We can help you confirm the scope of an assessor's engagement and prepare the systems and records they will test.
Any assurance report or conclusion comes from the independent assessor you engage, not from DeployOpen.