NIST AI RMF documentation evidence: what to keep

NIST AI RMF documentation and evidence: what should you keep?

When a customer questionnaire, contract clause or regulator inquiry asks how you apply the NIST AI RMF, the honest answer is that the framework is voluntary and has no certification, so your organization must keep its own records that show it was used. Those records are the AI inventory, the documented risk decisions, testing and evaluation results, training data and model provenance, incident and error logs, and residual risk documentation. You own these records. DeployOpen can help prepare the systems and templates that produce them, but only an independent auditor, assessor or other qualified third party can report on them, and NIST does not validate or endorse any organization's use of the framework.

Start with the Playbook. Each of its 72 entries includes a Transparency and Documentation section listing what an organization can document, and seven subcategories carry the Documentation tag: GOVERN 1.4, GOVERN 1.6, MAP 1.6, MAP 2.2, MAP 3.1, MAP 3.3 and MEASURE 2.1. Treat these prompts as a starting list, not a checklist, and decide which ones your program will answer, who signs off on each answer and where the answer is stored. For generative AI, the Generative AI Profile adds specific expectations: inventory entries that record data provenance, known issues and incidents from sources such as the AI Incident Database, and human oversight roles, along with documentation of training data sources, potential biases and any fine-tuning or retrieval-augmented generation applied.

The list you need depends on whether you build or buy AI and which risks you face. A builder keeps architecture, training process and test results for its own models. A buyer relies more on vendor documentation, contract terms and its own deployment context, so it should confirm with each vendor what training data, performance metrics and known limitations can be shared. Under Texas Business and Commerce Code Section 552.103, as enacted by Texas HB 149, the attorney general may request an AI system's purpose, training data types, outputs, known limitations and monitoring records after a complaint, so confirm with counsel which of these you can produce on short notice.

Which records each AI RMF function produces

Govern is the cross-cutting function, so its records describe the rules the rest of the program follows. The Playbook's documentation prompts for GOVERN 1.1 ask the organization to document its regulatory environment, including minimum requirements set in laws and regulations, and to record whether the system has been reviewed for compliance with applicable laws, regulations, standards and guidance. Those two answers form the legal baseline that every later record refers back to.

Regulatory environment register: a dated record of the laws, regulations and standards that apply to each AI system, with the minimum requirements each one sets. Legal or compliance leads usually own it.
Compliance review records: the review of each system against those obligations, with reviewer, date, findings and open items.
Policies and risk tolerance statements: approved versions with approval dates and revision history, owned by executive sponsors.
AI inventory entries: the Generative AI Profile action GV-1.6-003 suggests recording data provenance, known issues, human oversight roles and intellectual property considerations for generative systems. The platform or AI owner maintains these.

Confirm with your legal function which regulations apply to your sector and jurisdictions before treating any register as complete.

What an AI inventory entry should contain under the Generative AI Profile

The AI inventory is the index that the other AI RMF records hang from: the Playbook documentation prompts, incident logs, training data records and residual risk notes all point back to a system entry. The Generative AI Profile suggests specific fields for generative AI entries, and the list below groups them by the questions a reader will ask. Confirm which of these fields your own tooling can hold before you design the register.

Data provenance fields

Record where each data input and model artifact came from, and how you can verify it. The Generative AI Profile's suggested action GV-1.6-003 names the source, signatures, versioning and watermarks as items to capture. Keep a version identifier for each model, prompt template and dataset so a later question about a specific output can be traced to the exact artifact that produced it. Check whether your model registry or repository can store signatures and watermark status, or whether those must live in a linked system.

Known issues from bug tracking

Link each system entry to the open and closed defects from internal bug tracking that affect its behavior or outputs. GV-1.6-003 also points to external sources, including the AI Incident Database, AVID, CVE and NVD entries, and the OECD AI incident monitor. Record which external entries were reviewed, when, and what your team decided. Confirm that your issue tracker can tag defects to a specific model version, since untagged defects are hard to reconstruct later.

Human oversight roles

Name the people or teams who review, override, approve or shut down each system, and the decisions each role owns. This field connects the inventory to the accountability and human oversight records held under GOVERN 2 and MAP 3. Include the escalation path for a serious output problem and the owner of any manual review queue. Confirm that the role names are kept current in your identity or HR system, because an out-of-date owner field is a common gap during an assessment.

Intellectual property considerations

Record the licenses, usage terms and ownership conditions for the models, datasets, code and generated content in each system. The Generative AI Profile treats intellectual property as a documented consideration, and MAP 4 covers the legal risk of third-party components. Note which license applies to each component, whether output may be reused commercially, and who approved the terms. Ask your legal team to confirm the list, since license terms change and an entry without a review date goes stale quickly.

Links to the other AI RMF records

Treat the inventory entry as the index for the rest of the program. Each entry should point to the Playbook documentation prompts completed for that system, its training data record, its incident and near-miss log, its change history and its residual risk note. Without these links, a customer or regulator question about one system requires a search across several folders. Confirm whether your tooling supports cross-references between records or whether you will maintain an identifier that all records share.

Training data, pre-trained model and fine-tuning records

The Generative AI Profile action MG-3.2-003 asks organizations to document the sources and types of training data and their origins, the potential biases in that data, and the architecture and training process of any pre-trained model, including any fine-tuning or retrieval-augmented generation applied on top of it. For an assessor or a customer's risk team, this record answers a basic question: where did the system's behavior come from? Start by building a data lineage record for each dataset used in training, tuning or retrieval, listing its source, the license or permission that covers its use, its collection dates, its known gaps and the bias checks run against it. Where a fine-tuning set or retrieval index was built in-house, keep the versions of the dataset, the filtering steps applied and the person who approved each release.

Most buyers of third-party models cannot produce the full picture on their own. The vendor holds the original training corpus, the architecture details and the pre-training run logs, so the buyer should request this material in writing before signing and again at each major model change. The request should cover the categories of training data, the known bias findings, the evaluation results the vendor used, and whether the vendor applied fine-tuning or retrieval on your behalf. Confirm with the vendor what it will disclose under confidentiality terms, what it will decline to share and how it will notify you of changes. Record each answer with its date, because a vendor statement that was accurate at procurement may not describe the model you are running six months later.

Keep the vendor responses alongside your own records in one file tied to the AI inventory entry for the system, so that a single lookup returns the model version, the vendor documentation received, your fine-tuning and retrieval configuration, and the open gaps you have accepted. Where the vendor cannot provide a field, write down the gap and the compensating control, such as additional output testing or restricted use, rather than leaving the field blank. This is the record a regulator or customer is most likely to ask for first, and a documented gap with a reasoned response usually serves the reader better than an empty field.

Which incident, error and change records should you maintain?

What does MANAGE 4.3 suggest you record about incidents and errors?

MANAGE 4.3 in the AI RMF Playbook suggests that organizations regularly share information about errors, incidents and negative impacts with stakeholders and affected parties, and maintain a database of reported errors, near misses and incidents. Each entry should carry dates, an impact assessment and the response taken. The Playbook presents these as suggested actions, not a checklist, so your organization decides which records fit its risk profile. Start by confirming which AI systems fall in scope and who owns the incident log for each one.

Who should receive information about AI errors and incidents?

The Playbook points to stakeholders and affected parties, which can include internal risk and legal teams, customers who deploy your system, and people affected by its outputs. MANAGE 4.1 and related subcategories also address communication with relevant AI actors. Decide the audience for each incident class in advance, and write down the trigger, the channel and the content. Confirm with your legal counsel which disclosures carry contractual or regulatory obligations before you agree to share anything externally.

What fields should an AI incident database include?

A useful record includes a unique ID, the date the error or near miss was detected, the system and version affected, a description of the event, and the impact assessment with the groups affected. It should also show the response, the owner, the closure date and any follow-up changes. The Generative AI Profile action GV-1.6-003 points to known issues drawn from internal bug trackers and external sources such as the AI Incident Database and the OECD AI incident monitor, so include a field for the external reference when one applies. Engineers can keep this in a ticketing system with tagged fields, and an auditor can then sample it.

How should you record system changes and version history?

Keep a change log that ties each change to a model, prompt, data source, configuration or deployment version. Each entry should name the requester, the approver, the test evidence and the rollback point. The Playbook suggests maintaining version history and a database of system changes, which lets you show what was running when an incident occurred. In a self-hosted environment, your version control system, container registry and infrastructure-as-code repository can produce much of this evidence, provided tags and approvals are enforced. Confirm with your platform team which changes must pass review before release.

How long should you keep AI incident, error and change records?

The AI RMF does not set a retention period, and the Playbook does not state one. Retention depends on contract terms with customers, sector rules, privacy obligations for any personal data in logs, and the limitation periods that apply to your business. Set a schedule per record type, then confirm retention rules with legal counsel before deleting anything. Also check whether a legal hold or regulator request could require you to keep records longer than your default schedule.

Who inside and outside the organization may read these records?

Access should follow the need to know: engineers and risk owners for operational work, compliance staff for audit sampling, and legal counsel for disputes. External readers such as customers, regulators or independent assessors should receive extracts under a defined process, not direct access to the full log. Incident records may contain personal data or security details, so restrict and log access. Confirm the access model with your legal and privacy teams, and state it in the policy that governs the records.

Can DeployOpen set up incident and change record systems for us?

DeployOpen plans, implements, integrates and documents self-hosted open-source platforms inside your own infrastructure, so it can configure ticketing, logging, and version control tooling in that environment and document how the records are produced. It does not own your incident process, set your retention schedule or issue audit reports. Your organization owns the controls and the decisions about disclosure. Engagements are fixed-scope, so the written scope should list which systems, record types and integrations are included before work starts.

How to document negative residual risk

MANAGE 1.4 in NIST AI 100-1 asks organizations to document negative residual risks, which the framework defines as the sum of all unmitigated risks, and to share that documentation with two audiences: downstream acquirers of the AI system and its end users. The point is that residual risk is not a single number to be filed away. It is the set of harms the organization has decided to live with after controls are applied, and the people who deploy or are affected by the system need to see it. For a program built on the Manage function, this record is the place where MAP and MEASURE findings turn into an explicit acceptance of what remains.

Residual risk documentation sits alongside the go or no-go determination that MANAGE 1 describes. A reader inspecting that record should be able to see, for each significant risk, the original finding from MAP or MEASURE, the treatment chosen, the control or limitation that remains in place, the residual risk that is accepted, and the named role that accepted it. The go or no-go decision itself should state the conditions attached to a go, such as restricted use cases, required human review or monitoring thresholds, and it should link to the version of the system being approved. Confirm with your own governance team which role holds that authority and whether your risk acceptance process already has a template you can extend.

Before you write the record, decide what "unmitigated" means in your environment and check the result with the people who will read it. Ask the vendor or internal model team for known limitations and failure modes that they have not fixed, and confirm whether those belong in the residual register or in customer-facing documentation. Then plan the handoff: an acquirer and an end user need different versions of the same facts, and both versions should trace back to one approved record. Also agree how the register is updated after changes, since the Playbook's MANAGE 4.3 prompts ask for system change history and version records that keep the residual view current.

What the Texas attorney general can demand after a complaint

This sequence shows how an organization turns its AI RMF records into answers an attorney general could request after a complaint under Texas Business and Commerce Code Section 552.103. Each step ends with a record that can be produced on demand, so gaps surface before a demand arrives.

Confirm whether the Texas law applies
Before building any records, have counsel confirm whether the organization's AI systems fall within the scope of Texas HB 149 (89th Legislature, Texas Responsible Artificial Intelligence Governance Act) and how Section 552.103 applies to them. Counsel should also confirm which deployments and complaint channels are relevant. The output is a short written applicability note listing the systems in scope, which the rest of this sequence then uses as its boundary. Confirm the statute text against the enrolled bill linked from this page's sources rather than relying on secondary summaries.
Document the system's purpose and deployment context
The product or risk owner writes down what the AI system is for, who uses it, where it is deployed, and what decisions or actions it supports. This maps to the context and intended-use material the AI RMF expects under the Map function. The record is complete when a person outside the project could explain the system's purpose and setting from the document alone. Keep it versioned so the description matches the system as it was deployed on any given date.
Record training data types and their origins
The data science or ML team lists the categories of training data, their sources, and how they were obtained, with any known bias concerns. For a pre-trained or fine-tuned model, it also records the architecture, the training process, and any fine-tuning or retrieval-augmented generation applied. The Generative AI Profile action MG-3.2-003 describes these elements. The record is ready when every training source traces to a documented origin and a named owner.
Catalog input data categories and outputs
The system owner documents the categories of data the system accepts as input, including any personal or sensitive categories, and the types of outputs it produces, such as text, scores, or recommendations. Where outputs feed a downstream decision, the record names that decision. The check that closes this step is a side-by-side table of inputs and outputs that the team has reviewed against the system's actual interfaces, so the description is accurate and not aspirational.
Assemble performance metrics and known limitations
The measurement team gathers the performance metrics used to evaluate the system, the test results behind them, and the conditions under which they were measured. It also records known limitations, including error modes, populations or inputs where performance is weaker, and residual risks documented under MANAGE 1.4. Results should be dated and tied to a system version. The step closes when every metric has a stated method, a result, and a limitation statement a reviewer could verify.
Retain post-deployment monitoring and incident records
The operations team produces the monitoring plan for the deployed system and the records it generates: monitoring results, a database of reported errors, near misses, and incidents with dates and responses, and the change history with version records. The Playbook suggestions under MANAGE 4.3 describe these records. The step is complete when the team can retrieve a dated incident entry and show the response and the system version it affected.
Describe user safeguards and human oversight
The product and compliance owners document the safeguards offered to users, including notices, human review or override paths, appeal or feedback channels, and any restrictions on use. This ties to the human oversight roles that the Generative AI Profile asks organizations to record. The output is a current safeguards document with an owner and a last-reviewed date, plus evidence that the safeguards are live in the deployed product.
Package the records for a possible demand
Compliance leads assemble the records from the previous steps into one index organized by the items named in Section 552.103, with a named custodian for each item and a note on where it is stored. They test retrieval by pulling two or three records at random and confirming they match the system version in production. The package is ready when any item can be located and produced without rebuilding it from memory, and counsel has reviewed what the package discloses and how.

How to check the evidence file before a customer or regulator asks

A NIST AI RMF evidence file is only useful if its records connect. The checks below test whether an inventory, risk decisions, test results, incidents, change history and third-party documentation can be followed from one to the next. Passing them prepares evidence for an assessor or a customer review but does not replace an independent assessment.

Inventory entries link to risk decisions

Each AI system in the inventory should point to the risk decision that governs it, including the go or no-go determination and the owner who signed it. Generative AI inventory entries should also carry data provenance, known issues, human oversight roles and intellectual property notes, as the Generative AI Profile suggests. Pick three systems at random and confirm you can move from the inventory line to the decision in under a minute. A broken link here usually means the decision was made informally and never recorded.

Risk decisions link to test results

Every documented risk decision should cite the measurements that informed it, such as TEVV reports, evaluation results and the metrics chosen under MEASURE 1. Check that each cited result names the system version it was run against. A decision that relies on a test run against an earlier model or prompt configuration does not show the current system was evaluated.

Incident records are dated and answered

The Playbook suggests keeping a database of reported errors, near misses and incidents, each with a date, an impact assessment and a response. Check that every entry has all three fields and that responses reference the change or control that followed. Entries with no response, or responses dated before the incident, will draw questions in any review.

Change history matches the deployed version

Confirm that the version history for each system lists every model, prompt, retrieval source and configuration change, and that the version in production matches the most recent entry. The Playbook suggests maintaining a database of system changes for this purpose. A mismatch between the logged version and the running version undermines every test result tied to it.

Third-party model documentation is on file

For any pre-trained or vendor-supplied model, keep the documentation the supplier provided and your own record of the source, architecture, training process and any fine-tuning or retrieval-augmented generation you applied. The Generative AI Profile suggests documenting training data sources, their origins and potential biases. Where the supplier withholds detail, record that gap as a residual risk rather than leaving the field blank.

The file has a named owner

Assign one accountable person to the evidence file as a whole, separate from the people who build and test each system, and record their name, role and review date. Ask whether that owner can produce the file within a set period when a customer or regulator asks. Without a named owner, the other five checks tend to decay between reviews.

Who owns the evidence, who prepares it and who assesses it

Your organization owns the AI RMF program and every record it produces. Accountability for the risk decisions, the go or no-go calls, the residual risk acceptances and the retention schedule sits with named people inside the company, and those people must be able to explain the records when a customer or regulator asks. Documentation prompts in the Playbook are questions to answer, not forms that someone else can complete on your behalf.

Decisions and sign-offs: Record who approved each deployment, each risk acceptance and each decision to keep a system running after an incident.
Inventory ownership: Keep the AI inventory current, including data provenance, known issues and human oversight roles for each system.
Retention rules: Set how long incident logs, change histories, training data records and residual risk documents are kept, and who may change or delete them.
Stakeholder communication: Decide what errors and incidents are shared with affected parties and when, and keep proof that the sharing happened.
Legal review: Confirm which laws, contracts and sector rules apply to each system, since no outside party can decide that for you.

Check that each record has a named owner, a storage location and a retention period that your legal and security teams have approved.

Questions buyers ask about AI RMF documentation

Does the NIST AI RMF require us to produce any documents?

No. The NIST AI RMF is a voluntary framework, and NIST's FAQ says neither private nor public sector organizations are required to use it. No regulator or standards body issues an AI RMF certificate, so no document is mandated by the framework itself. The documents you keep matter because a customer contract, a state law that names the framework, or a regulator's information demand may ask for evidence of how you governed and measured your AI. Decide which records you need from those external obligations, then build your documentation to match.

How much effort do AI RMF records take, and what drives it?

Effort depends mainly on how many AI systems you run, how many are generative, and how much of each system you build versus buy. The Playbook tags seven subcategories with the Documentation topic (GOVERN 1.4, GOVERN 1.6, MAP 1.6, MAP 2.2, MAP 3.1, MAP 3.3 and MEASURE 2.1), so those are a sensible starting set. Records for procured systems usually depend on what your vendors will disclose, which can slow collection. Ask for a quote built from a written list of systems, owners and the document types each one needs. Most of this work is recurring, because inventories, incident logs and change histories must stay current after launch.

Can our in-house team build these records without outside help?

Often yes, if people who own each system can write its records and a separate reviewer checks them. Teams commonly struggle with the inventory, the training data history and the residual risk write-up, because these need input from engineering, legal and product at the same time. An outside implementation partner can set up the templates, inventory fields and logging systems, and it can run a gap review against the Playbook. It cannot issue an attestation. Only an independent assessor can give an opinion on your program, so keep that role separate from the people who wrote the records.

What does a customer questionnaire usually ask for about AI documentation?

Customer questionnaires tend to ask for a short set of recurring items, mapped to the same records the framework describes:

A current inventory of AI systems in scope, with owners and purpose
Sources and types of training data, and known bias concerns
Known limitations, performance metrics and testing results
Incident and error history, and how incidents were communicated
Residual risks left after mitigation, and who accepted them
Change history and version records for the system

For generative AI, the Generative AI Profile's suggested actions add provenance details and third-party disclosures. Prepare these as standing packages so answers stay consistent across questionnaires.

Is the Playbook enough on its own to document our program?

No. The NIST AI RMF Playbook gives suggested actions and documentation prompts for each of the 72 subcategories, and NIST says it is neither a checklist nor a set of steps. It does not hold your evidence. You still need your own inventory, data records, test reports, incident database and decision records. The Playbook helps you decide what to write down for each subcategory. Its audit log shows the last content update in August 2023, so check whether its prompts still match your current needs and any later NIST guidance before you rely on them.

How will the planned revision of AI 100-1 affect our records?

NIST says AI RMF 1.0 is being revised as part of the White House AI Action Plan and has not yet published a revised version. The July 2025 Action Plan recommends removing references to misinformation, diversity, equity and inclusion, and climate change, so wording in some subcategories may change. Your underlying records, such as inventories, incident logs and test results, are still useful under any version. Keep each record tied to the framework version you used, and confirm with your customers or counsel which version a contract or state law names. Review your templates when NIST publishes the revision.

How DeployOpen prepares AI RMF evidence

DeployOpen helps organizations decide which records their NIST AI RMF program should keep, then sets up the self-hosted systems that produce them inside the client's own infrastructure. The organization owns its AI risk decisions and its evidence, and only an independent auditor or assessor issues any report.

How we scope

Engagements are fixed-scope, with the documentation records, AI systems and business units covered agreed in writing before work starts.
We map the Playbook documentation prompts for the agreed subcategories to specific records the client already keeps or must create.
The written scope names the evidence owners, storage locations and review cadence, so the client knows who maintains each record.

How we build

We configure self-hosted tooling in the client's cloud accounts, data centers or air-gapped networks to capture AI inventory fields, change history and incident logs.
Templates for training data provenance, known limitations and residual risk documentation are built to match the client's own review and approval workflow.
Access controls and retention settings are configured so that records stay complete, versioned and available to the client's reviewers.

How we hand over

Deliverables include the configured systems, documented procedures and a record of what each evidence item shows and where it lives.
Handover documentation explains how to add a new AI system to the inventory and how to close an incident record.
The client receives a written list of open items it must confirm with its own legal, risk and product teams before reliance.

How we support

Ongoing support, upgrades and continuous testing of the evidence systems are available as a retainer under a separate written scope.
When NIST revises AI RMF 1.0 or the Playbook, we can review which records need new fields, as agreed in scope.
Questions about what an independent assessor or regulator will request are routed to the client's own counsel or auditor, since DeployOpen does not issue attestations.

NIST AI RMF documentation and evidence: what to keep