What MEASURE 3.2 asks when AI risks cannot be measured directly
MEASURE 3.2 expects an organization to set up tracking approaches for AI risks it cannot yet measure, not to claim a measurement it does not have. In NIST AI 100-1, this outcome sits under MEASURE 3 (tracking AI risks over time) in Table 3 of the Measure function, and it applies to settings where current measurement techniques cannot assess a risk well or where no accepted metric exists yet. Confabulation in a generative system and harmful interaction effects between components are typical examples. Picture a team that has deployed an assistant whose hallucinated answers are hard to score automatically. The question for that team is not which number to report, but how it will notice the risk, who will look at what it finds, and what it will do when the picture is unclear. Three conditions make these risks hard to handle. First, there is no accepted metric, so dashboards can show a reassuring figure that says little about the harm. Second, ownership is often unclear: the AI team builds the system, the business owns the outcome, and no one is named for emergent risks. Third, feedback loops are weak, because users may not know how to report a bad output and incident channels were designed for software bugs. The Playbook suggests addressing these through recourse mechanisms, bug bounties, user research, and documenting the rate and severity of hard-to-measure risks when making go or no-go decisions. Confirm with your own risk owners which of these channels already exist before designing new ones. NIST AI RMF is voluntary. NIST does not require organizations to use it, and there is no AI RMF certification, accreditation or conformity assessment program. NIST also does not validate or endorse any organization's use of the framework. If a customer or regulator asks for evidence, what you can offer is your own documented tracking approach, its named owners and the records it produces, which an independent assessor can then review against the organization's stated commitments.
Which AI systems, teams and feedback channels fall under MEASURE 3.2
MEASURE 3.2 applies wherever an AI risk cannot be assessed with current techniques or has no agreed metric. The decisions below determine which deployed systems, which people and which external inputs carry that tracking duty, so the scope is set before any monitoring plan is written.
Deployed systems with unvalidated metrics
Start with each deployed system and ask whether its most significant risks have a metric that TEVV has validated for the deployed context. Generative systems, agents and decision-support tools often fall into this gap, because pre-deployment test results may not predict behavior with real users. Record each system where no validated metric exists and treat it as in scope. Confirm the list against the AI inventory kept under GOVERN 1 so no system is missed.
TEVV owners for emergent risk
The Playbook tags MEASURE 3.2 to TEVV staff, and the tracking duty needs a named owner inside that group. Decide who reviews emerging failure patterns, who escalates them and who documents the rate and severity of each hard-to-measure risk. Without a named owner, signals from monitoring tend to stay in engineering queues. Ask the lead to show the role assignment in writing.
Domain experts who judge harm
Some harms, such as clinical misdirection or unfair treatment of applicants, can only be recognized by people who understand the domain. Decide which domain experts review sampled outputs and incident reports, and how often. Their judgment is often the only available measure for a risk that no automated metric captures. Record their qualifications and the decisions they are authorized to make.
Impact assessment reviewers
The AI Impact Assessment role is tagged to MEASURE 3.2 because impacts on affected groups often emerge after launch. Decide who reruns or updates impact assessments when usage changes, a new population is served or a complaint pattern appears. This keeps the assessment tied to what happens in production instead of the original design. Confirm that the trigger conditions for reassessment are written down.
Operations teams watching live behavior
Operations and monitoring staff see drift, odd usage and user workarounds first, so they hold the earliest view of emergent risk. Decide which operational signals they report to TEVV, which thresholds prompt escalation and how their observations are logged. Their reports become evidence only when they are timestamped and linked to a named system owner. Ask for the escalation runbook before approving the scope.
Third-party models and data sources
Pre-trained models and external data can add risks the organization cannot measure directly, such as undisclosed training data or silent model updates by a vendor. Decide which third-party components feed systems in scope and what the contract lets you observe about them. Where the vendor shares no measurable signals, record the gap as a hard-to-measure risk. Confirm the update notification terms with each supplier.
How an assessor tests MEASURE 3.2 and what passes
NIST does not publish a conformity assessment procedure for the AI RMF, and it does not certify organizations against it. An assessor, whether an internal audit function, a customer's third-party reviewer or an independent party engaged by the organization, therefore starts from MEASURE 3.2 itself: the outcome says risk tracking approaches should be considered where AI risks are hard to assess with current measurement techniques or where metrics do not yet exist. The assessor then checks whether the organization made that consideration explicitly and can show how it was done.
- Which risks are hard to measure: Can the organization name the specific risks it treats as hard to measure, and explain why current techniques fall short for each?
- Who tracks them: Are the AI actors responsible for tracking emergent risks named, with their roles written down?
- What approach is used: Which tracking methods were chosen, such as recourse mechanisms, user research or structured feedback, and why were they chosen for these risks?
- How decisions use the results: Does the organization document the rate and severity of hard-to-measure risks when it makes go or no-go decisions?
- Who was consulted: For generative AI, does the organization show it consulted external AI actors when identifying emergent risks?
Confirm with your assessor which of these questions they treat as mandatory for your scope, since the Playbook suggestions are optional.
Because no fixed evidence list exists, assessors generally accept evidence that shows a documented, repeatable process and proof that it ran. A single slide deck describing good intentions rarely passes. The strongest packages connect each hard-to-measure risk to a named owner, a tracking method, a record of what was collected and a decision that followed.
- A risk register for hard-to-measure risks: Entries that state the risk, why it resists measurement, the proxy indicators used and the owner.
- Named role assignments: A RACI or equivalent showing which AI actors track emergent risks, aligned to the AI actor categories the Playbook tags for MEASURE 3.2.
- Tracking records: Dated logs from recourse channels, user research sessions, bug reports or participatory engagement, with the triage outcome for each item.
- Decision records: Go or no-go or change decisions that cite the rate and severity observations, signed off by the accountable role.
- Review cadence evidence: Meeting notes or change records showing the register was revisited after deployment, not only at launch.
Ask what format the assessor prefers for logs, and whether they will sample tickets or interview the owners.
Most gaps in MEASURE 3.2 are not missing policies. They are mismatches between what the organization says it tracks and what the records show. Findings tend to cluster around a few patterns, and each one is easier to fix before the assessment than during it.
- Unnamed risks: The organization says it tracks emergent risks but cannot list which risks it considers hard to measure.
- Owners without activity: Roles are named in a document, yet no tracking records exist for the period under review.
- Feedback collected but not adjudicated: Reports arrive in a helpdesk or inbox, yet no one records how they informed a decision.
- Severity not documented: Decisions are made without recording the rate or severity of the hard-to-measure risk involved.
- Generative AI gaps: Organizations using generative AI rely on general monitoring and do not show a process for identifying emergent generative AI risks or consulting external actors.
Before testing, sample your own records against each finding above and fix the gaps you can prove.
Implementing risk tracking for hard-to-measure AI risks
This sequence turns MEASURE 3.2 into a working practice: named owners, a register of risks without usable metrics, recourse and feedback channels, a rate and severity record for go or no-go decisions, and a handoff into MANAGE. Each step produces a written artifact that an assessor or internal reviewer can inspect.
Name the AI actors who own emergent risk tracking
Assign named people to track emergent risks for each AI system in scope, drawing on the AI actor categories the Playbook tags to MEASURE 3.2: TEVV, domain experts, AI impact assessment, and operation and monitoring. Record each role's responsibilities, the escalation path to the accountable executive, and who may decide that a risk is emergent. Done when a roster exists, every role has a named person, and the owners have accepted their tasks in writing.
Register risks that lack usable metrics
Work with the domain experts and TEVV staff to list the risks for which current measurement techniques are weak or no metric exists, such as harms that are contextual, slow to appear, or hard to quantify. For each entry, record the system, the risk description, why it is hard to measure, the proxy indicators in use, and the owner. Done when every mapped risk is either tied to a metric or placed on this register with a stated reason.
Open recourse, feedback and participatory channels
Stand up the channels through which people outside the AI team can surface problems: a reporting route for users, a path for affected stakeholders, bug bounty or vulnerability intake where it fits, and user research or participatory sessions for systems with high impact. Confirm the intake tool and routing with the operations team, for example a ticketing system such as GLPI or Zammad, and document who triages each report. Done when each channel has an owner, a triage rule, and a tested route into the register.
Triage incoming signals into the risk register
Review each report, research finding and monitoring alert against the register. Classify it as a known hard-to-measure risk, a new emergent risk, or noise, and log the reasoning. Keep the original signal, the classification, and the date it was reviewed so the trail can be reconstructed later. Done when every signal in the period has a recorded disposition and emergent items have a named owner.
Record rate and severity for each hard-to-measure risk
For each risk on the register, document how often the signals occur and how serious the consequences are, using a scale the organization defines and writes down before analysis. Where a numeric measure is not valid, record a structured qualitative rating with the evidence behind it and the limits of that evidence. Done when each entry has a rate, a severity rating, a confidence note, and a review date set by the owner.
Use rate and severity in go or no-go decisions
Present the recorded rate and severity to the accountable decision maker before a system is deployed, expanded or kept running. Document the decision, the risks accepted, the conditions attached, and any system or use restrictions. Confirm that the record names the decision owner and that the rationale is retained. Done when each decision has a signed record that links back to the register entries it relied on.
Hand findings into MANAGE for response and communication
Route each risk that needs treatment to the MANAGE work: prioritized responses and residual risk records under MANAGE 1, responses to newly identified risks under MANAGE 2, and incident and error communication to relevant AI actors under MANAGE 4. Set a review cadence for the register with the owners. Done when every open emergent risk has a MANAGE ticket or a documented acceptance, and the register is reviewed on the agreed cadence.
How the Playbook and the Generative AI Profile shape the tracking program
The NIST AI RMF Playbook gives suggested actions for MEASURE 3.2, and the most useful ones for a hard-to-measure risk are the channels that bring signals in from outside the AI team: recourse mechanisms, bug bounties, human-centered design, user research and participatory engagement. Each of these is a way to notice harm that your metrics do not yet capture. The Playbook also asks you to name the AI actors responsible for tracking emergent risks and to document the rate and severity of hard-to-measure risks when making go or no-go decisions. Treat these as suggestions, not a checklist. NIST says the Playbook is neither a checklist nor a set of steps, so the job for your team is to choose the actions that fit the system, record why you chose them, and record why you left others out.
If your system is generative AI, NIST AI 600-1, the Generative AI Profile published July 26, 2024, adds one suggested action under MEASURE 3.2, labeled MS-3.2-001. It calls for processes that identify emergent generative AI risks, including consulting external AI actors, and it is tagged to the risks Confabulation and Human-AI Configuration. In practice, this means your intake channels should accept reports from people outside your organization, and someone should review those reports for signs of new failure modes, such as a model producing confident but false output in a setting where users act on it. An illustrative example: a customer support team notices a pattern of incorrect policy answers, routes it through the recourse channel, and the tracking owner adds it to the risk register with a documented severity rating.
Confirm the Playbook's status before you build the program around it. The Playbook's audit log shows its last content update in August 2023, and NIST's AI RMF Playbook page says the AIRC will update it after the AI RMF is revised. NIST also says AI RMF 1.0 is being revised as part of the White House AI Action Plan and has not yet published a revised version. Write your procedures so that the MEASURE 3.2 actions are tied to your own documented risk decisions, not to the current Playbook wording, and check the NIST AI RMF Playbook and the Generative AI Profile text again when a revision is published. Ask your assessor or internal auditor which version of each source they will test against.
Evidence that shows hard-to-measure risks are tracked
For MEASURE 3.2, the evidence is less about a metric dashboard and more about records showing who watches for emergent risks, how their findings were judged, and what decisions followed. The cards below cover the artifacts that connect those pieces. Each one needs a named owner and a review cadence set by the organization, because stale records are the most common gap.
Named owner register for emergent risks
A register listing each AI system, the person or team accountable for watching for emergent and hard-to-measure risks, and their backup. The NIST AI RMF Playbook suggests naming the AI actors responsible for this tracking, so the register is the first thing to show. Check that each owner still holds the role, that the Playbook's TEVV, Domain Experts, AI Impact Assessment and Operation and Monitoring roles are covered, and that the register records the date of its last review.
Emergent risk tracking log
A dated log of entries for each hard-to-measure risk, with the observation, its source (recourse request, bug bounty report, user research, expert review), the owner, the status and the next review date. A log with entries but no review dates shows activity without oversight. Keep the log in a ticketing or case system so entries cannot be silently edited, and confirm that closed items record what was done.
Rationale for proxy and qualitative indicators
Where no validated metric exists, a short written record explaining which proxy or qualitative indicator is used, why it was chosen, what it can and cannot show, and who approved it. Assessors look for this rationale because a proxy without stated limits can look like a measurement it is not. Revisit each rationale when the system changes, when a better metric appears, or when the indicator stops predicting the risk it was chosen for.
Go or no-go records with rate and severity
Decision records showing the rate and severity of hard-to-measure risks at the time of each go or no-go decision, the reasoning used when data was incomplete, and the approver. The Playbook suggests documenting these figures specifically, so a decision that ignored them will be questioned. Between audits, add a record whenever a release, scope change or new deployment context is approved, and note any risk accepted without a measurable basis.
Feedback and incident intake records
Records from every intake channel that can surface emergent risk, including helpdesk tickets, appeals, bug bounty submissions and incident reports, each linked to the AI system it concerns. Intake that is not connected to the risk log is a common gap. Check that each record shows triage, the owner who assessed it, and whether it changed a risk entry. Helpdesk platforms such as Zammad or GLPI can hold these records if the workflow is configured to tag AI systems.
Review meeting minutes
Minutes from periodic reviews where owners walk through the risk log, challenge proxy indicators, and decide whether to escalate, monitor or close items. Minutes show that tracking is discussed by people with authority, not only logged. Record attendees, the decisions taken, the open actions with owners and due dates, and the date the next review is scheduled. Keep minutes with the same retention as the risk log so the review history can be reconstructed.
What meeting MEASURE 3.2 costs in money and effort
This table breaks down the cost components of tracking hard-to-measure AI risks, who is paid for each, whether it is one-time or recurring, and what drives it up or down. Use it to build a written scope and to ask vendors and partners what their quotes include. No figures are given because none are supplied; request them in writing against your own systems.
| Cost component | Who is paid or who bears it | One-time or recurring | What drives it and what to ask for in a quote |
|---|---|---|---|
| Internal owner time | Named AI actors inside the organization, such as the product owner and risk lead | Recurring | Number of systems and the review cadence. Ask the partner to list which roles it expects you to staff and the hours each role needs. |
| Helpdesk or ticketing tooling | Software licensing or hosting, or internal platform staff if self-hosted | One-time setup, then recurring operation | Number of users and channels feeding reports. Ask whether the quote covers configuration, integration with email or chat intake, and ongoing upkeep. |
| User research and participatory engagement | Researchers, facilitators, and sometimes external panel or community organizers | Mostly one-time per study, with periodic repeats | Number of affected groups and whether engagement is in person or remote. Ask for the study design, participant recruitment method, and reporting format. |
| External red teaming or expert review | Independent testers or domain experts engaged under a written scope | Recurring per test cycle | Breadth of the system, number of hard-to-measure risks in scope, and whether testing covers new model versions. Ask for scope, rules of engagement, and retest terms. |
| Assessor time | Independent auditors or assessors, paid only for their own review | Recurring per assessment cycle | Depth of evidence required and how well records are organized before the review. Ask the assessor for a written scope and evidence list before fieldwork. |
| Risk documentation and go or no-go records | Internal governance and risk staff | Recurring with each release decision | Number of release decisions and documentation depth. Ask the partner what templates and records it will hand over and what remains your responsibility. |
| Setup of metric and measurement gaps | Internal data science or engineering staff, plus any external implementation partner | One-time, with ongoing revision | Number of risks with no accepted metric and how often new methods appear. Ask for the list of gaps covered and how proxy indicators will be reviewed. |
Who owns MEASURE 3.2 and what a partner can take on
Your organization owns the risk decisions. MEASURE 3.2 is about risks that current methods cannot measure well, so someone accountable must decide how those risks are tracked, which actors are responsible, and whether a system stays in service. The Playbook suggests naming the AI actors responsible for tracking emergent risks and documenting the rate and severity of hard-to-measure risks when making go or no-go decisions. Those are management acts, and they should be recorded under your own authority.
- Named owners: Assign specific roles, not teams alone, to track emergent risks and to escalate them.
- Risk acceptance: Record who accepts hard-to-measure risks and on what basis, including the rate and severity you observed.
- Tolerance settings: Define the thresholds that trigger review, pause or deactivation.
- Domain input: Involve domain experts and operators who see real use, since they often notice emergent risks first.
- Policy ownership: Keep the tracking procedure in your own governance documents.
Before engaging a partner, confirm that these decisions have named owners in writing, because a partner can build the workflow but cannot make the decision.
An implementation partner can prepare the systems and evidence that support tracking. DeployOpen plans, implements and documents self-hosted open-source platforms inside your own infrastructure, so intake channels, ticket workflows and evidence templates can run where your data already lives. The partner does not set risk tolerances or accept risk on your behalf.
- Intake systems: Configure channels for staff, users and affected parties to report suspected emergent harms, such as a helpdesk fed by email and web forms. Zammad, for example, consolidates email, chat, telephone and social media into one ticket system.
- Tracking workflows: Set up ticket states, categories and escalation paths in a service management tool such as GLPI, which provides ticketing, problem records and recurring-issue tracking under ITIL-style practices.
- Evidence templates: Build forms that capture the date, system version, observed behavior, severity and disposition of each report.
- Reporting views: Produce statistics on report volume, recurrence and time to close, to feed the go or no-go discussion.
- Documentation: Write runbooks that explain the workflow to operators.
Ask the partner which tools it will install, where they will run and who administers them after handover, and whether the scope includes testing the intake path itself.
An independent auditor, assessor or certification body tests your controls and issues any report or opinion. NIST does not validate or endorse any organization's use of the AI RMF, and no AI RMF certification exists, so any assurance statement about MEASURE 3.2 comes from a party you engage for that purpose, not from NIST or from DeployOpen.
- Test design: The assessor samples tracked risks and checks whether the workflow produced documented decisions.
- Evidence review: The assessor examines tickets, owner assignments and go or no-go records for consistency with your stated process.
- Independence: An assessor that helped build the workflow should not also be the party testing it, so role separation must be planned.
- Reporting: Only the assessor issues the report, with the scope and limits it chooses.
- Outcome ownership: Findings lead to corrective actions that your team owns.
Before engaging anyone, confirm which party will issue any written conclusion and whether that party is independent of the implementation work.
Organizations commonly outsource the build of intake and tracking systems, the drafting of evidence templates and the initial configuration of dashboards. They usually keep decisions, risk acceptance and ongoing review. Where the line falls depends on how much judgment the work involves and whether the same party would later evaluate its own output.
- Usually outsourced: Tool installation, workflow configuration, template drafting and integration with existing systems.
- Usually retained: Named actor assignments, severity judgments, go or no-go decisions and communication with affected users.
- Shared work: Testing of the intake path, which can be run by a security testing provider, while owners review the results.
- Independence line: Do not let the party that configured the tracking system certify that it works.
- Recurring effort: Ongoing administration, upgrades and retainer support recur after the initial build, so ask who carries them.
When requesting a quote, ask the partner to list each deliverable, the owner of each decision and the independent party, if any, that will review the outcome.
Where MEASURE 3.2 programs usually fall short
MEASURE 3.2 asks organizations to keep watching risks that current methods cannot measure well. The gaps below tend to appear once a system is live, and they share one root: nobody is clearly responsible for acting on signals that have no settled metric. Each card names the gap and the fix that closes it.
No named owner for emergent risks
Emergent risks often sit between product, data science, security and legal, so each function assumes another is watching. The Playbook suggests naming the AI actors responsible for tracking emergent risks. Write the owner, their backup and their reporting line into the risk register, and confirm that the owner has time and authority to escalate. Ask who would notice a new harm pattern on a Friday evening.
Reports that never reach decisions
Monitoring outputs are often sent to a dashboard or channel that no approver reads before a go or no-go decision. The Playbook suggests documenting the rate and severity of hard-to-measure risks when making those decisions. Put a standing agenda item in the AI governance forum, record the rate and severity of each open item, and show who signed off on any risk that stayed open.
Feedback channels nobody reviews
Teams often launch a feedback form, a bug bounty or a support queue and then let it fill up. Unreviewed input is not tracking, and it creates a record that shows the organization knew about a problem. Route reports into a ticketing system with an owner and a review cadence. Open GLPI or Zammad, for example, can hold these tickets, and the review should check how many reports stayed unanswered.
Qualitative risks logged without rationale
When a risk cannot be measured, teams often write a label such as low or medium with no explanation. An assessor cannot test that label, and neither can the next team that inherits the system. For each qualitative risk, record the evidence used, the people consulted, the assumptions made and the conditions that would change the rating. Review the rationale whenever the deployment context changes.
Emergent risks without external input
Internal teams rarely see harms that affect people outside the build group. For generative AI, NIST AI 600-1 suggests establishing processes for identifying emergent risks that include consulting external AI actors. Define who outside the organization can be consulted, how their input is recorded, and how it is weighed against internal findings before the next decision.
Tracking that ends at launch
Many programs run an intensive review before release and then stop tracking. Hard-to-measure risks tend to show up only after users change how they work with the system. Set a tracking schedule that continues after go-live, define triggers that reopen the risk assessment, and confirm that the schedule appears in the monitoring plan for the system.
Common questions about MEASURE 3.2 cost, effort and audit readiness
Can an assessor certify that we comply with MEASURE 3.2?
No. The NIST AI RMF has no certification, accreditation or conformity assessment program, and NIST does not validate or endorse any organization's use of the framework. An independent assessor can review your evidence and report on what it found, but no certificate exists for MEASURE 3.2 or for the AI RMF as a whole. When a customer or regulator asks for proof, ask the assessor for a written scope, the criteria they used, and the evidence they sampled. Your own documented tracking program, with named owners and dated records, is what you can show.
Can our in-house team implement MEASURE 3.2 without outside help?
Often yes, if the team already runs incident handling, product feedback and model monitoring. MEASURE 3.2 asks you to track risks that current methods cannot measure, name the AI actors responsible for that tracking, and record the rate and severity of hard-to-measure risks at go or no-go decisions. The Playbook tags this subcategory to TEVV, Domain Experts, AI Impact Assessment and Operation and Monitoring roles. Outside help is most useful when none of those roles exist in-house, or when someone must design the process and the evidence trail. Ask any partner to name which parts they would prepare and which parts your staff would own.
How much effort does MEASURE 3.2 preparation usually take?
The effort depends on how many AI systems are in scope, how many of them have risks that resist measurement, and how much tracking machinery already exists. NIST does not publish effort estimates for MEASURE 3.2, so treat any figure you hear as a claim to verify. The main drivers are the number of named AI actors, the channels already used for user reports and recourse, and whether hard-to-measure risks are recorded in a register. Ask for a written breakdown of effort by system, role and deliverable before you accept a quote.
What does the business need to provide for this work?
Preparation needs decisions and access more than it needs technical output. Leadership must agree who makes go or no-go calls when a risk cannot be measured, and what severity justifies stopping or pausing a system. Product, legal and support teams must share complaints, incident records and user research. Engineering must give access to logs, model release notes and monitoring data. Without these inputs, tracking records will be thin, and an assessor will see gaps between stated intent and practice.
How long does it take to get MEASURE 3.2 ready for review?
No researched timeline exists for this subcategory, so confirm durations with any provider in writing. Timing depends on whether the tracking process already exists, how many systems need it, and whether recourse and feedback channels must be built. Work usually moves faster when the AI inventory from GOVERN 1 is current and the owners from GOVERN 2 are named. Ask for milestones tied to concrete outputs, such as a named owner list, a tracking procedure and a sample register.
Which costs are one-time and which recur?
Setup costs cover defining the tracking process, naming owners, choosing a register or ticketing tool, and documenting how hard-to-measure risks feed go or no-go decisions. Recurring costs cover staff time for reviewing reports, updating the register after model changes, and periodic reviews of emerging risks. Tooling costs vary with licensing. Open-source helpdesk and ticketing platforms such as Zammad and GLPI can hold recourse requests and risk records, but their licensing terms and hosting costs should be checked against your own environment. Ask any quote to split one-time from recurring costs.
What does a DeployOpen engagement cover for MEASURE 3.2?
DeployOpen plans, implements and documents self-hosted open-source platforms inside your own infrastructure, and prepares systems and evidence for compliance assessments. For MEASURE 3.2, that can include setting up a ticketing workflow for user reports and recourse, configuring a register for hard-to-measure risks, and documenting the owners and decisions. DeployOpen does not own your controls, does not decide your risk tolerance, and does not issue audit reports or certificates. Engagements are fixed-scope, with scope and deliverables agreed in writing before work starts. A quote follows from that written scope.
How do we prove hard-to-measure risks are tracked when auditors ask?
Show a dated trail from risk identification to decision. Keep a register that names each hard-to-measure risk, the AI actor responsible for tracking it, the channels used (such as user reports, bug bounties or user research), and the rate and severity recorded for go or no-go decisions. Add meeting notes or decision records that show the review took place. The Playbook suggests documentation prompts for this subcategory, and the Generative AI Profile adds consultation with external AI actors for generative systems. Confirm with your assessor which evidence they expect before you build the trail.
How DeployOpen prepares MEASURE 3.2 tracking evidence
DeployOpen helps organizations set up the systems, channels and records that show how they consider and track AI risks that are hard to measure. The organization still owns its risk decisions and its AI risk management program, and DeployOpen prepares the operational evidence that supports them.
How we scope
- Engagements are fixed-scope, with the written scope naming which AI systems and emergent risk sources are covered.
- The scope lists the AI actors who own tracking, such as TEVV, domain experts and operations staff.
- The scope states which hard-to-measure risks are in view and how the go or no-go record will be kept.
How we build
- We configure feedback intake, recourse and incident channels in the client's own infrastructure, so records stay under its control.
- We set up ticketing workflows for emergent risk reports, with defined owners, triage steps and escalation paths.
- We document the rate and severity fields the organization will use to record hard-to-measure risks for decisions.
How we hand over
- Deliverables include configured systems, workflow documentation, role assignments and a register of tracked emergent risks.
- We provide documentation of each actor's tracking tasks, so reviewers can trace who acted on which signal and when.
- Handover covers operator training on the tools, and the organization retains administrative access to all records.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Periodic reviews can update tracking workflows when the Playbook or the planned AI RMF revision changes expectations.
- Independent auditors or assessors decide whether the evidence meets any requirement; DeployOpen does not issue reports or certificates.
