What MAP 3.1 asks for before an AI system goes live
MAP 3.1 is the NIST AI RMF outcome that the potential benefits of the intended AI functionality and performance are examined and documented. In practice, that means an organization can state what the system is expected to do better than the process it replaces, who benefits, and how it knows. Many teams can describe what their AI does, but they cannot point to a written record of the value they expect, the evidence behind that expectation, or the people who reviewed it. Benefits are often assumed because the business case was approved on enthusiasm, and nobody was asked to measure the claim before launch.
Benefits deserve the same rigor as risks for a practical reason: every risk decision depends on a benefit decision. If the expected gain is vague, a team cannot judge whether a tolerable error rate is worth accepting, whether a harm is proportionate, or whether a non-AI alternative would do the job. The framework also places MAP 3.1 alongside MAP 3.2 on the costs of errors, so the benefit side and the error side should be documented together. Good starting material includes the intended task, the baseline it is compared against, the performance results, and comments from end users and affected groups about whether the output is useful and understandable.
NIST AI 100-1 is voluntary, and there is no AI RMF certification. No external body will stamp MAP 3.1 as complete, so the evidence is whatever the organization can show its own governance committee, customers in a due diligence questionnaire, or a regulator who asks how a system was justified. Confirm with your legal and compliance leads which external obligations apply to your system, since some of them may name the framework directly. Then decide who signs off the benefit statement, where it is stored and how it is updated after performance changes. The NIST AI RMF Playbook lists suggested documentation for this subcategory, and a version-controlled wiki or document system can hold that record so changes and approvals stay traceable.
Which AI systems, teams and third parties MAP 3.1 covers
Before anyone can document potential benefits, the team has to decide which systems, people and outcomes the exercise covers. Each decision below changes how much evidence gathering, stakeholder engagement and benchmarking the organization must carry out, so settle them in writing before the benefit analysis starts.
Custom models built in house
Models the organization trains, fine-tunes or assembles itself carry the full burden of MAP 3.1. The team must state the intended functionality, define the performance measures that represent benefit, and keep the evidence that links those measures to the intended purpose. Confirm who owns the model card, evaluation datasets and benchmark results, because those records are the benefit claim an assessor will ask to see.
Vendor AI features inside existing tools
AI features switched on inside a platform the organization already runs, such as ticket triage or document summarization, still need a documented benefit case. The organization cannot inspect training data or model internals, so the work shifts toward recording the feature's intended use, measuring outcomes in its own workflows, and comparing them with the process it replaces. Ask the vendor which evaluation results it will share and under what terms.
Generative AI tools and assistants
Generative AI tools produce varied outputs, so a benefit claim for one use case may not hold for another. The scope should name each permitted task, such as drafting internal emails or code suggestions, and record the measures used for each. The Generative AI Profile adds no suggested actions for MAP 3.1, so the organization must set its own measures for each task rather than relying on profile guidance.
Affected groups whose outcomes count
A benefit to the deploying team can be a cost to customers, patients or staff elsewhere in the process. The Playbook suggests incorporating feedback from affected communities about perceived benefits, so the scope should list which groups are affected, how their outcomes will be measured, and who speaks for them. Expect more engagement work when the affected group is external to the organization.
Roles across development, deployment and impact
The Playbook tags MAP 3.1 with three AI actor categories: AI development, AI deployment and AI impact assessment. Each role owns a different part of the benefit record, from the performance evidence through to the assessment of who gains. Name one accountable person per role, and check that the impact assessment role is separate from the team that built the system, so benefit claims receive an independent review.
Third-party benchmarks behind vendor claims
When a vendor's benchmark supports the benefit case, the organization inherits that benchmark's assumptions without having run it. The team should request the test method, dataset, baseline and date, then decide whether the vendor's conditions match its own workflows. Where they differ, the organization must run its own comparison against a human baseline or a standard benchmark, which adds work that the vendor's marketing figures do not cover.
How an assessor tests MAP 3.1 and what passes
This table shows what an independent reviewer typically looks for when testing MAP 3.1 documentation, the evidence they ask for, and the gaps that come up most often. It describes review practice, not a pass guarantee, because the AI RMF is voluntary and has no certification. Read each row as a question the reviewer will ask of your records.
| Test approach | Evidence requested | Common findings |
|---|---|---|
| Review of the benefit statement against intended purpose | Documented benefit statement, system purpose from MAP 1, and the owner who approved the statement | Benefits are generic (faster, cheaper) with no link to the stated purpose or to measurable outcomes |
| Check that end users and affected communities were engaged | Engagement plan, participant groups, dates, and recorded input with how each item was adjudicated | Input came only from the build team, or feedback was collected but never recorded or traced to a decision |
| Review of baseline and benchmark comparisons | Named human baseline or benchmark, selection rationale, test data, and the comparison results | Baseline chosen after results were seen, unrepresentative workers used as the baseline, or no comparison performed |
| Check of benefit records for versioning and currency | Version history showing who changed each benefit claim, when, and why, plus the review date | Benefit claims copied from a pre-launch pitch and never updated after model, data or scope changes |
| Check that benefits link to MAP 3.2 and MAP 3.3 records | Cross-references showing the error costs and deployment scope that bound each claimed benefit | Benefits documented alone, with no statement of the errors that would erode them or the contexts where they hold |
| Check of interpretability and efficacy of outputs | Examples of outputs reviewed by users, interpretability notes, and efficacy results for the target population | Efficacy claimed from aggregate accuracy only, with no evidence for the groups most affected |
How to document AI benefits step by step
This sequence turns the intended purpose of an AI system into a documented benefit statement that names who gains, how the benefit was measured against a fair comparison, what users said about it, and where the claim stops applying.
Confirm the intended functionality with the system owner
Start by agreeing in writing with the system owner what the AI system is meant to do, which tasks it performs, and which outputs people will act on. Ask the owner to state the performance goal in plain terms, such as the decision or task it supports and the quality the business expects. The MAP 1 context record and the MAP 2 categorization are the natural inputs, so reuse them rather than rewriting the purpose. The step is done when the owner signs off a one-page functionality description that lists the intended tasks and excludes uses nobody has approved.
Identify users and affected communities
List the people who will use the output, including operators, reviewers and downstream teams, and the groups outside the organization who could be affected by it. The Playbook suggests engaging end users to document potential benefits, so make that engagement a named activity with a contact owner, typically the product owner or the AI deployment lead. Record each group, its role and how it will be reached. The step closes with a stakeholder register that separates direct users from affected communities, so later feedback can be traced to the right group.
Choose baselines and benchmarks with reasons
Select the comparison points the benefit claim will rest on. The Playbook points to human baselines, such as the performance of experienced staff doing the same task, and to standard benchmarks where a recognized one exists. For each choice, write down why it fits the intended use, what it does not cover and who approved it. Ask the system owner and a technical reviewer to challenge the choice before testing starts. The step is complete when a baseline register lists each comparison, its rationale and its known gaps.
Run the comparison with agreed measures
Test the AI system against each baseline using measures fixed in advance, such as accuracy, time to completion or rate of usable outputs, along with the efficacy and interpretability of outputs that the Playbook asks you to consider. Keep the test data, the evaluator names and the run settings in version-controlled records so the comparison can be repeated. The step produces a results note that reports each measure, its uncertainty and any cases where the system did worse than the baseline. Results that favor the system should be reported with the same detail as results that do not.
Collect end user feedback on perceived benefits
Ask the users and affected community representatives identified earlier how the system changes their work and whether the benefits they see match the test results. Use structured questions and record both benefits and concerns, including outputs that users found hard to interpret or trust. The Playbook also suggests incorporating feedback from affected communities about perceived benefits, so include them in this round where practical. The step closes with a feedback log that tags each comment to a user group and states whether it confirms, qualifies or contradicts the measured results.
Record the benefit statement with limits and review date
Write a benefit statement that names the intended users, the tasks covered, the baselines used, the measured results and the conditions under which the benefit holds. Add the known limits, the contexts where the system should not be used and the trigger for re-review, such as a model change, a new user group or a drift in measured performance. Store the statement in a versioned knowledge base with an approver and a next review date set by the organization's own policy. The step is done when the statement is approved, linked to the test records and the feedback log, and visible to the teams that rely on it.
Which evidence artifacts satisfy MAP 3.1
MAP 3.1 asks organizations to examine and document the potential benefits of an AI system's intended functionality and performance. The artifacts below are the records an assessor or internal reviewer is likely to ask for, and each one should be traceable to the same system version and intended purpose. Keep them under version control so the record of how the benefit case was built stays intact.
Benefit statement tied to intended purpose
A written statement of the benefits the system is expected to deliver, each linked to a named intended use, a user group and a measurable outcome. Vague claims such as improved efficiency do not satisfy MAP 3.1 on their own. Check that every benefit names the context it applies to and the limits outside that context. Review the statement whenever the intended purpose, the model or the underlying data changes, because a benefit proven for one task may not hold for another.
End user engagement record
A log of who was consulted about perceived benefits, when the sessions took place, what questions were asked and what the participants said about the efficacy and interpretability of outputs. The Playbook suggests engaging end users for exactly this purpose. Record the role of each participant, not only their names, so an assessor can judge whether the group represents the people who will use the system. Refresh the record when the user population or workflow changes.
Baseline comparison report with method
A report comparing system performance against a human baseline or a standard benchmark, including the dataset, the metrics, the test conditions and the reasons the comparison is fair. Without the method, the numbers cannot be reproduced or challenged. Record which benchmark version was used and confirm it still reflects the deployment context. Rerun the comparison after any material model, prompt or data change, and keep the earlier report for audit history.
Affected community feedback log
A record of feedback gathered from people outside the user group who may be affected by the system, including their views on perceived benefits and any concerns they raised. MAP 3.1 expects these perspectives to be documented alongside the views of the builders and direct users. Note how each group was reached, what was done with the input and where feedback changed the benefit statement. Review the log on a schedule and after incidents that involve affected communities.
Accountable owner sign-off
A dated approval from the person accountable for the AI system, confirming that the benefit statement, engagement record and benchmark report reflect the current system and were reviewed. This shows that the benefit case was accepted by someone with authority, not only prepared by the technical team. Confirm the owner is named in your governance records. Ask for a new sign-off whenever the system is materially changed or its owner changes.
Version history of each record
The change history for every artifact above, showing what changed, when and by whom, with the ability to compare versions and restore earlier ones. Assessors use this history to confirm that the benefit case was not rewritten after the fact to match results. Check that edit rights are restricted to the owners of each record and that the history cannot be silently cleared. Tie each version to the system version it describes.
Why the Generative AI Profile and Playbook leave MAP 3.1 to your own method
NIST AI 600-1 gives you no suggested actions for MAP 3.1, so a generative AI team must build its own evidence for documenting potential benefits. The Generative AI Profile, published July 26, 2024, ties its suggested actions to AI RMF subcategories, but it states that not every subcategory is included and that unaddressed ones may be added later. For a generative AI assistant, that means the benefit case, the comparison against human baselines and the feedback from end users have to be designed by your team and recorded in your own terms. Decide who owns that record (usually the product owner with the AI risk function) and agree the format before the first benefit claim is made to a customer or executive.
The Playbook does suggest actions for MAP 3.1, such as engaging end users, comparing performance with human baselines or standard benchmarks, and incorporating feedback from affected communities. Treat those suggestions as prompts, not requirements. NIST says the Playbook is neither a checklist nor a set of steps, and its audit log shows the last content update in August 2023. AIRC has said the Playbook will be updated after the AI RMF is revised. Before you quote Playbook wording in a policy, a contract or an assessment response, check the current guidance directly with NIST at the NIST AI RMF page, since the text you rely on may change.
NIST has also said that AI RMF 1.0 is being revised as part of the White House AI Action Plan and has not yet published a revised version. Confirm the status of that revision with NIST before you map your MAP 3.1 evidence to a version number, and record the version you used so that later reviewers can see which guidance shaped your benefit documentation. Do not assume the revision will leave MAP 3.1 unchanged or that the current wording will remain the reference point.
What documenting AI benefits costs in money and effort
This table lists the cost components of documenting the benefits of an AI system under MAP 3.1, who is typically paid for each, whether it is one-time or recurring, and what drives it up or down. Use it to structure a written scope and to check that a quote itemizes each component. The main takeaway is that the largest drivers are the number of user groups, whether a human baseline exists, and how often the system changes.
| Cost component | One-time or recurring | What drives it | How to keep it down |
|---|---|---|---|
| Engineering time to build comparison methods | One-time, with recurring rework after changes | Number of benchmarks and whether a human baseline can be measured in practice | Reuse one evaluation harness across use cases and limit comparisons to the benefits the business has named |
| Stakeholder time for end user engagement | One-time for initial sessions, recurring for periodic check-ins | Number of user groups and affected communities consulted | Group users by role, use structured questionnaires and sample representative users instead of surveying everyone |
| Benchmark or evaluation data preparation | One-time build, recurring refresh | Availability of labeled or representative data and the need for domain expert review | Start from existing validated datasets, document data provenance once and refresh only the segments that changed |
| Governance time to review and sign off | Recurring at each review cycle and each material change | Number of approvers, the risk tolerance set under MAP 1 and the depth of documented rationale | Agree a sign-off template and approval thresholds in advance so minor changes follow a lighter route |
| Evidence upkeep after each model or data change | Recurring | How often the model, prompts, data sources or intended use change | Tie benefit documentation to version control, record which version each result applies to and rerun only affected comparisons |
| External review by an independent assessor or consultant | Periodic, and at each assessment cycle | Scope of the review, the depth of evidence requested and whether the reviewer must reproduce results | Keep evidence organized by subcategory before the review starts and confirm in the scope which items the reviewer will test |
How XWiki and BookStack keep benefit records versioned and controlled
A MAP 3.1 benefit record is only useful if reviewers can see who wrote each claim, what changed and who was allowed to approve it. XWiki covers the history of the record, and BookStack covers who may edit or approve it.
XWiki keeps a traceable benefit history
XWiki keeps every change to a page under version control, recording what changed, when and by whom. For MAP 3.1, this means a statement such as an expected accuracy gain over the current process can be traced back to the draft that introduced it and the reviewer who edited it. Reviewers can compare any two versions side by side, and a team can roll back to an earlier version if a benefit claim was overstated. Confirm that the version history retention and export options meet your own record keeping rules before you rely on it.
XWiki comparison supports benefit reviews
Because any two versions can be compared, a review committee can see exactly which benefit figures, human baseline comparisons or end user feedback changed between one draft and the next. This matters when a benefit claim is revised after testing or after feedback from affected communities. The comparison shows the change itself, so reviewers do not need to rely on memory or a summary email. Ask your implementation partner to set a convention for change notes so each revision explains its reason.
BookStack limits edits through roles
BookStack gives each user abilities from the roles assigned to them. For a benefit record, this lets you separate the people who draft a benefit statement, such as product owners, from the people who approve it, such as a risk or AI governance lead. Restricting edit rights to a small set of roles reduces the chance that an unreviewed benefit claim is published as settled. Confirm that the role model fits your approval chain before you assign people to it.
BookStack overrides at each level
BookStack lets administrators override permissions per role at the shelf, book, chapter or page level. A benefit record can therefore be open to all AI team members at the shelf level, while a page holding approved benefit statements for a high-impact system is restricted to a named approver role. Narrow overrides keep the approved record protected without blocking drafting elsewhere. Map the override structure to your system inventory before you build the shelves, because changing it later is harder to audit.
Self-hosted deployment and license terms
Both tools can run on infrastructure you control, so benefit records stay inside your cloud accounts, data centers or air-gapped network. XWiki is licensed under LGPL-2.1-or-later and BookStack under MIT, and those license terms apply to whatever you deploy or modify. Your legal team should confirm that the obligations of each license fit your distribution and customization plans. Confirm also that each tool, version and hosting setup suits your own environment before you treat it as the system of record.
Who owns MAP 3.1 and where a partner helps
Your organization owns MAP 3.1. The accountable owner, usually a product or business leader with authority over the AI system, decides what benefits the system is intended to deliver and signs the benefit statement. The AI team runs the baselines, collects the performance data and records how outputs were judged. Governance functions review the record and decide whether the benefits claimed are supportable enough to show to customers, boards or regulators. NIST AI 100-1 places this work in the Map function, and the Playbook tags MAP 3.1 with AI Development, AI Deployment and AI Impact Assessment actors, so several internal roles must contribute.
- Benefit statement: the accountable owner writes the intended benefits in plain language, names the users who gain them and states what success looks like.
- Baselines: the AI team selects human baselines or standard benchmarks, runs them and keeps the results with the configuration used.
- Participatory input: product and operations staff who work with the outputs document efficacy and interpretability, and feedback from affected communities is recorded.
- Governance sign-off: a review body confirms the claims match the risk tolerance set under GOVERN 1 and GOVERN 2.
- Change control: any change to the model, data or intended use triggers a review of the benefit statement.
Before you engage anyone, confirm which person signs the benefit statement and which committee can reject it.
An implementation partner prepares the systems and evidence that your team uses for MAP 3.1. DeployOpen, for example, plans, implements and documents self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks, and prepares evidence for assessments. A partner does not decide the benefit statement, choose the business case or declare that the framework is met. Its work product is the structure, the tooling and the organized record that your team and your assessor can read.
- Documentation structure: the partner sets up page templates for the benefit statement, baseline method, participant list and sign-off record, mapped to MAP 3.1 and its Playbook suggestions.
- Comparison methods: the partner can help design and run comparisons against human baselines or benchmarks, and it records the procedure and raw results.
- Self-hosted wiki and permissions: the partner deploys a self-hosted wiki such as XWiki, which keeps version history, and configures roles in a tool such as BookStack so that only authorized reviewers can approve changes.
- Evidence packs: the partner assembles dated, version-controlled bundles that show what was claimed, when and by whom.
- Limits: the partner does not issue reports, attestations or certificates, and it should not write the conclusion that an assessor will test.
Ask the partner to state in writing which artifacts it produces and which decisions remain with your organization.
An independent auditor, assessor or certification body tests the record and issues any report or certificate. For the AI RMF, which NIST describes as voluntary and without a certification program, there is no official certifying body. Your organization may still face an internal audit, a customer assessment or a regulator review that asks how MAP 3.1 was carried out. Whoever performs that review should be independent of the team that wrote the benefit statement and of the partner that built the evidence packs.
- Independent testing: the assessor samples the benefit statement, traces the baseline results and checks that the participants and feedback records exist.
- Consistency checks: the assessor compares claimed benefits with the monitoring data and with the risk tolerance on file.
- Reporting: only the assessor issues the findings or report, and it states the scope it covered.
- Independence boundary: the assessor cannot design the benefit statement or the benchmarks it then tests.
- Scope agreement: the engagement letter defines which AI system, version and period are in scope.
Before hiring any assessor, confirm its independence rules, the scope it will test and who receives the findings.
Questions teams ask before documenting AI benefits
Is there a certification for MAP 3.1 under the NIST AI RMF?
No. The NIST AI RMF has no certification, accreditation or conformity assessment program, and NIST does not validate or endorse any individual organization's use of the framework. MAP 3.1 is an outcome your organization documents for its own governance. If a customer or regulator asks for evidence, you show the record of benefits, comparisons and stakeholder input you produced, and an independent assessor can test that record if the contract requires it. Ask any firm that claims otherwise to show the issuing body in writing.
Can our team document AI benefits without outside help?
Often yes, provided someone owns the work and the people who understand the business case are involved. The NIST AI RMF Playbook offers suggested actions, documentation prompts and references for the subcategory, but it is not a checklist, so your team still has to decide what applies to its system. Teams usually need outside help when they lack experience with TEVV methods, when no human baseline exists, or when they must connect the record to several frameworks at once. An implementation partner can prepare the documentation structure and evidence, while your organization keeps ownership of the decisions.
How much effort does MAP 3.1 take, and what drives it?
The effort depends mainly on how many AI functions are in scope, how many business owners and end users must be consulted, and whether performance comparisons already exist. A single internal assistant with clear users and an existing process baseline is far simpler than a customer-facing system that affects several communities. Your organization carries most of the recurring work, because benefits change as the system and its users change. Ask any partner to break its estimate into discovery, stakeholder engagement, baseline selection, documentation and review, and to state which of those steps it will perform.
What do business owners and end users need to contribute?
Business owners need to state the intended purpose, the value the system is expected to create and the outcomes that would count as success. End users need to describe where the output helps their work, where it fails them, and how interpretable the results are in practice. The Playbook suggests engaging end users to document potential benefits and the efficacy and interpretability of outputs, and to incorporate feedback from affected communities about perceived benefits. Schedule these sessions early and record who took part, what they said and how their input changed the documentation.
How do we choose a human baseline when none exists?
Start by defining the task the system performs and the current way people perform it, including time, accuracy and rework. If no measured process exists, document that gap and choose a benchmark that approximates the task, such as a sample of expert reviews or an established public dataset for the domain. The Playbook suggests comparing performance with human baselines or standard benchmarks, so record which comparison you chose, why it fits the intended use, and where it is weak. Note the limits openly, because an assessor will look for a justified choice rather than a perfect one.
How often should the benefits record be reviewed?
Review the record whenever the system, its data, its users or its intended scope changes, and at a regular interval set by your governance policy. NIST's guidance does not give a fixed period for MAP 3.1, so the interval is an internal decision that should be approved under your GOVERN policies. Tie reviews to monitoring results, because a drop in measured performance or new user feedback may show that a recorded benefit no longer holds. Keep prior versions so you can show what changed and when.
What should we ask a partner to include in a written scope?
Ask for the systems and AI functions covered, the stakeholders to be engaged, the baseline and benchmark methods to be used, and the documentation the partner will produce. Request a list of deliverables, the review cycle, the people responsible for each step, and what your organization must provide, such as access, data and business owner time. DeployOpen works on fixed scope agreed in writing before work starts, and it prepares systems and evidence without issuing audit reports or certificates. Confirm that the independent assessor, if one is required, is named separately.
How DeployOpen prepares MAP 3.1 evidence for your systems
DeployOpen plans, implements and documents the self-hosted open-source platforms that hold the records behind MAP 3.1 Potential benefits, inside your own infrastructure. We prepare the systems and evidence; your organization owns the benefit findings and the decisions that follow from them.
How we scope
- Agree in writing which AI functions, user groups and benefit claims the MAP 3.1 work covers.
- Identify which teams supply benefit evidence, such as end users, affected communities and AI development staff.
- Confirm which human baselines or benchmarks the organization will compare against before work starts.
How we build
- Configure a self-hosted wiki to hold benefit statements, comparison method notes and stakeholder feedback records.
- Set up page-level permissions so only assigned reviewers can approve or change benefit documentation.
- Connect review workflows so benefit findings move through named owners before they reach decision records.
How we hand over
- Deliver a documented structure mapping each MAP 3.1 evidence item to its owner and review cadence.
- Provide version history settings that record what changed, when and by whom for every benefit document.
- Train your documentation owners on how to export, compare and roll back records during an assessment.
How we support
- Offer ongoing support as a retainer to maintain benefit records as the AI system and its use change.
- Help update documentation when the NIST AI RMF Playbook or related guidance is revised, as your scope requires.
- Ask your independent assessor or auditor to confirm evidence sufficiency, since DeployOpen does not issue reports.
