MEASURE 2.8 Transparency and accountability: evidence guide

What MEASURE 2.8 asks for and why teams struggle with it

MEASURE 2.8 asks you to examine and document the transparency and accountability risks that you identified earlier in the MAP function. A customer, contract or regulator may ask how your AI system is overseen, and the honest answer is a body of records rather than a certificate. The AI RMF is voluntary, and NIST does not validate or endorse any organization's use of it. No certification or conformity assessment exists, so an outside party cannot issue you a MEASURE 2.8 result. What you can show is evidence: the risks you named, how you examined them and what you documented.

The hard part is producing traceable records. Teams can usually describe their oversight model, but struggle to show the audit logs and histories of system activity, the statistics on overrides, reported errors and complaints, the response times for escalations, the policy exceptions granted, and the go or no-go decisions made by the people accountable for them. Those records must exist before the question arrives, and they must tie back to a specific decision and a named owner. Confirm with your own engineering and operations teams which of these records the system already writes, and which depend on people remembering to file them.

Accountability also depends on transparency, but the two are not the same. AI 100-1 Section 3.4 says accountability presupposes transparency, and that a transparent system is not necessarily accurate, privacy-enhanced, secure or fair. A complete log of who approved a model's output does not show that the output was correct. Treat MEASURE 2.8 as the layer that proves who decided and what happened, and check accuracy, security and fairness under their own MEASURE 2 topics.

Which systems, teams and records fall under MEASURE 2.8

MEASURE 2.8 applies only to the systems whose transparency and accountability risks were identified in MAP, so the first scoping decision is which systems those are. Each system then brings its own records, and each record is owned by a team whose role in the Playbook actions differs.

Systems with risks identified in MAP

MEASURE 2.8 examines risks that MAP already identified, so the scope starts with the MAP 1 to MAP 5 outputs for each AI system. A system with no recorded transparency or accountability risk is not automatically out of scope, but the decision to exclude it should be documented and reviewed. Systems that affect people's rights or access to services usually need the most detailed records. Confirm that every production system in your AI inventory is either mapped to a risk entry or explicitly excluded with a reason.

Audit logs and decision histories

Audit logs and decision histories are the core records for this subcategory, since they show what the system did and who acted on it. Confirm which events are logged (model version changes, configuration edits, access grants and output decisions), where the logs are stored, and how long they are kept. Logs that exist only in an application database are harder to protect and to produce during review, so check whether they are copied to a central store.

Oversight, override and complaint queues

Oversight statistics, overrides, reported errors, complaints and response times live in ticketing, case management and review tools that often sit outside the AI platform. Scope these queues explicitly, because they hold the human side of the accountability record. Ask who can override an output, whether overrides are recorded with a reason, and whether complaints about the system are linked back to the model version involved.

Go or no-go and escalation records

Policy exceptions, escalations and go or no-go decisions made by accountable parties need a defined home, such as a signed approval record or a change ticket linked to the system. These records are often the thinnest part of the evidence, because approvals happen in meetings or chat threads. Identify who holds decision authority for each system, and confirm that each release or exception has a dated, attributable decision that can be retrieved.

Teams that own each record

The Playbook ties MEASURE 2.8 to TEVV, domain experts, operations and monitoring, AI impact assessment and deployment actors. Each team holds different records, so assign a named owner per record type rather than assigning the subcategory to one group. TEVV usually owns test explanations and instrumentation, operations owns override and error statistics, and impact assessment owns the link between complaints and affected groups. Writing this down is often the largest effort reduction in the scoping work.

Third-party models and vendor-held logs

When a system uses a third-party model or a hosted generative AI service, some logs, prompts and moderation records may sit with the vendor. Identify which records the vendor keeps, how long they are retained, and how you can obtain them. Request these terms before procurement closes and write log access, retention and export into the contract, since you cannot produce records you cannot reach. Confirm this with each vendor, because log availability varies by service and plan.

How an assessor tests MEASURE 2.8

This table walks through the steps an internal reviewer, customer auditor or independent third party would take to test MEASURE 2.8, with the evidence each step requests and the findings that commonly appear. No AI RMF certificate exists, so each reviewer reports its own conclusions, and the table helps you prepare records that survive that review.

Test step	Evidence requested	Common finding
Review MAP risks on transparency and accountability	The MAP risk register entries that name transparency, explanation or accountability concerns, with the owner and date of each entry	Transparency risks identified in MAP never appear in any monitored metric or MEASURE result
Sample audit logs and histories	Audit log excerpts for sampled periods, log retention settings, and the access list for who can read or change the logs	Logs are enabled and retained, but no one has reviewed them or documented what was checked
Read oversight and override statistics	Counts of human overrides, reported errors, complaints and response times over the review period, with the definitions used	Override counts exist but are not broken down by system, cause or outcome, so trends cannot be read
Test complaint and escalation outcomes	Complaint and escalation tickets sampled end to end, showing the decision taken and the date it was closed	Complaints are logged, but many close without a documented outcome or a link to a risk review
Trace go or no-go decisions to accountable parties	Signed go or no-go records, the named accountable owner for each decision, and any recorded policy exceptions	Decisions are recorded without a named accountable owner or with approval by a role that has no authority over the risk
Confirm explanations were tested with different audiences	Test records showing how explanations were checked with users, operators and reviewers, and the results	Explanations are documented for engineers only, with no evidence that operators or affected users could understand them
Check the link between findings and corrective action	Tickets or change records showing how transparency findings led to system changes, policy updates or retraining	Findings are closed without evidence of a fix, or the same finding recurs in later review cycles

How to implement MEASURE 2.8 in sequence

This sequence turns the transparency and accountability risks named in MAP into logged events, reviewable records and documented go or no-go decisions that an assessor can trace from start to finish. Each step produces an artifact that the next one depends on, so the chain of evidence holds together.

Confirm transparency and accountability risks from MAP
Start from the MAP outputs, especially the impact characterization and the risks recorded for the system's context, users and human oversight. Ask the TEVV lead, the domain experts and the people who will operate the system to confirm which transparency and accountability risks are in scope, such as users not knowing when they are interacting with AI, unclear ownership of decisions or missing records of who approved a change. The step is done when each risk has an identifier, a short statement and a link to the MAP entry that produced it. Keep this register as the baseline that every later step is measured against.
Define required events, owners and review cadence
For each confirmed risk, decide which events must be recorded, such as prompts and outputs, configuration changes, overrides, complaints, escalations and approvals. Name the accountable reviewer for each event type and state how often the records are reviewed. The Playbook suggests tracking these items, but it is not a checklist, so the event list must come from your own system and risk register. The output is an event specification that lists the event, its source, its retention owner and its reviewer. Confirm the specification with the accountable parties before any logging is built.
Instrument the system with histories and audit logs
Configure the platform and its surrounding services to write the events defined in the specification. For self-hosted open-source components, this may mean enabling audit output in the platform's own configuration, routing logs to a protected store and setting access controls on that store. Trace tooling can capture request-level detail for AI workloads, but confirm exactly which fields it records and whether sensitive values are masked before you rely on it. The output is a test run showing that each event type appears in the log with the expected fields, and that a reviewer can read it without changing it.
Set up oversight, override and complaint statistics
Build the counts and timings the Playbook points to: oversight actions, overrides, reported errors, complaints, escalations and response times. Define each metric in writing, including what counts as an override and when a clock starts for response time. Assign a data owner who produces the figures on the agreed cadence. The output is a metric definition sheet and a first report generated from real or test data, reviewed by the accountable owner. Do not publish targets for these statistics until the organization has set them through its own governance.
Test explanations with different audiences
Show the explanations and system disclosures to the audiences who rely on them, such as operators, affected end users, domain experts and assessors, and record whether each group can understand what the system did and why. Use structured interviews or walkthroughs and capture the questions and misreadings. Feed the results back into the disclosure wording and the operator guidance. The output is a dated test record per audience, listing what was tested, who took part, what failed and what changed as a result. Note that explainability methods themselves are covered under MEASURE 2.9.
Document go or no-go decisions by accountable parties
Record each go or no-go decision for a release, a material change or a return to service, together with the evidence reviewed and the named decision makers. Include any policy exceptions granted and any escalations that informed the decision. Ask the accountable executive to sign off where your governance requires it, and keep the signed record with the logs it relied on. The output is a decision register in which every decision links to its evidence, its approver and its date. This register is what a reviewer will sample first when testing whether accountability works in practice.
Confirm the evidence chain with an independent review
Ask a reviewer outside the delivery team to trace one sample decision backward through the register, the statistics, the logs and the MAP risk that started it. Note any gaps, such as missing events, unowned metrics or decisions without linked evidence, and assign fixes with owners. Only an independent auditor or assessor can give an assurance opinion, so treat this review as internal preparation. The output is a gap list with owners and a closed record showing each gap was addressed or formally accepted by the accountable party.

Which evidence shows MEASURE 2.8 is met

MEASURE 2.8 asks for risks tied to transparency and accountability to be examined and documented, so the proof is a set of records that show who decided what, when and on what basis. The cards below cover the core artifacts an assessor will ask to see, plus generative AI records from NIST AI 600-1, and how to keep each one current between formal reviews.

Audit log exports and retention

Audit logs show who accessed or changed the AI system, its data and its configuration, and when. For MEASURE 2.8, check that logging covers administrative actions, model and prompt changes, and access to training and production data, and that the retention period is written into policy. Schedule regular exports to a separate, access-controlled location so records survive changes to the source system. Confirm with each platform's administrator how logs are rotated and where they are stored. Review a sample export each quarter to confirm it still contains the events you need.

System histories tied to versions

The Playbook suggests instrumenting the system with histories so that each output can be traced to the model, prompt, configuration and data in use at the time. Assessors look for a version record that links each deployed release to its test results and approval. Keep a change register that references model versions, prompt templates and retrieval sources, and make sure each entry points to the TEVV evidence for that version. Review the register whenever a release ships and during scheduled governance meetings to catch changes made outside the process.

Oversight and override statistics

Oversight statistics show how often humans reviewed, corrected or overrode system outputs, and whether those interventions were effective. For MEASURE 2.8, track counts of overrides, reported errors, complaints and response times by system and by period. Define the metric dictionary in advance so that numbers from different quarters can be compared. Confirm with the operations team which events are logged automatically and which require manual entry, since gaps in manual logging are a common reason these figures fail review. Refresh the dashboard monthly and archive each snapshot.

Complaint and escalation registers

A complaint and escalation register records each concern raised about the system, who handled it, what was decided and how long the response took. Assessors use it to test whether the accountability chain works in practice and whether response times match what the organization documented. Each entry should carry a unique identifier, a severity, the accountable owner and a closure note. Reconcile the register against support tickets every month so that complaints raised through informal channels are captured. Include policy exceptions and escalations in the same review.

Go or no-go decision records

Go or no-go records document each decision to deploy, expand, pause or retire an AI system, with the evidence reviewed and the named approvers. MEASURE 2.8 specifically looks for the accountable party behind each call, so a decision without a named owner is a finding. Keep a template that lists the risks considered, the open issues accepted and the conditions attached to approval. Store signed records in the same repository as the TEVV results they rely on. Review open conditions at each release and at least once a year so that accepted risks do not quietly expire.

Test explanations by audience

The Playbook suggests testing explanations with different audiences, such as engineers, domain experts, affected users and auditors, and documenting what each group understood. Assessors look for evidence that the explanation was tested, not just written. Record the audience, the explanation method, the questions asked and the observed gaps, then update the explanation when gaps appear. Repeat the test when the model, its inputs or its user population changes, and file the results with the model documentation so that each version has a matching record.

Generative AI content provenance and annotator instructions

For generative AI systems, NIST AI 600-1 suggests recording each instance where content is generated, modified or shared, using digital content transparency solutions (MS-2.8-002). It also suggests documenting the instructions given to data annotators and AI red-teamers (MS-2.8-003). Keep provenance logs with timestamps and content identifiers, and version each annotation or red-team instruction set alongside the tests it produced. Review a sample of provenance records and instruction versions each quarter to confirm they still match what was deployed.

What meeting MEASURE 2.8 costs in money and effort

This table compares each cost component of meeting MEASURE 2.8, who is paid for it, whether it is one-time or recurring, and what pushes it up or down. Use it to structure a written quote, since most of the effort sits in staff time that license savings do not remove.

Cost component	Who is paid	One-time or recurring	What drives it up or down
Logging and audit tooling	Platform engineers and the vendor or open-source support contract, if any	Setup is one-time; storage and maintenance recur	Log volume, retention periods, number of AI systems feeding one log store, and whether logs must be tamper-evident
Engineering time to instrument systems	Internal engineers or an implementation partner under a fixed scope	One-time per system, plus changes whenever a model or workflow changes	Number of AI systems, number of integration points, and whether histories and audit logs are added to existing applications or new ones
Oversight and complaint process design	Compliance, product and operations staff, sometimes legal	One-time design, then recurring review	Number of accountable parties, number of decision types that need a go or no-go record, and how many channels users can complain through
Evidence upkeep	Operations and compliance staff who collect and file records	Recurring	Frequency of overrides, complaints and escalations, and how much of the record set is generated automatically versus compiled by hand
Generative AI exposure	Engineers and reviewers who record generated, modified or shared content and annotator or red-team instructions	Recurring, with setup for content transparency tooling	Volume of generated content, number of annotation or red-team exercises, and whether outputs leave the organization
External testing or review	Independent testers or reviewers, where the organization chooses to commission them	Usually per engagement, possibly repeated	Scope of the systems tested, whether explanations are tested with several audiences, and whether third-party models are included
Assessor sampling time	Internal staff who answer requests, and the independent auditor who samples records	Recurring at each assessment cycle	How well records are indexed, how many systems an assessor samples, and how quickly staff can produce each log extract
Third-party dependencies	Vendors whose logs, models or APIs must be integrated and monitored	Recurring	Number of vendors, whether vendor logs are accessible, and how contracts define access to audit records

Which open-source tools produce MEASURE 2.8 evidence

Self-hosted open-source tools can generate much of the raw record that MEASURE 2.8 asks organizations to examine: histories, audit logs, overrides, and traces of how an AI system and its surrounding services behaved. The tools produce records only. The organization still has to name the accountable owners, set the review routines, and write down the go or no-go decisions that turn those records into transparency and accountability. Before relying on any tool below, confirm its fit in your own environment, including retention, storage location, and what it does and does not capture by default.

For the platform layer, Nextcloud's admin_audit app writes additional event information to audit.log in the data directory, or to a path set with the logfile_audit setting, which is useful where generated documents, shared files and administrative changes must be traceable to a user. OpenBao's audit devices keep a detailed JSON log of every API request and response, including errors, with most sensitive values hashed with HMAC-SHA256. That lets a reviewer reconstruct who requested which secret or policy change and when, without the log itself holding plaintext secrets. Confirm that the audit device is enabled in every environment, that the log is shipped to storage the AI team cannot edit, and that the hashing setting matches your privacy review. OpenBao is released under MPL-2.0 and Nextcloud under AGPL-3.0-only, so confirm both license obligations with your legal team before redistribution.

For language model applications, Langfuse (MIT) records each trace with every operation in a request, including LLM calls, retrieval steps, tool executions and custom logic, along with timing, inputs, outputs, token usage and metadata. This supports the review of how an AI system reached an output and where a human overrode or corrected it. The garak scanner (Apache-2.0) probes language models for issues such as prompt injection, data leakage and toxicity, and writes a JSONL report of every attempt and evaluation plus a hit log of attempts that found a vulnerability. Those files can feed the error and complaint records that the Playbook suggests tracking. Decide who owns each log, how often it is reviewed, and what triggers an escalation, and record that decision alongside the evidence.

Who does what: your team, a partner and the assessor

Your organization owns the transparency and accountability outcome. MEASURE 2.8 asks that risks tied to transparency and accountability are examined and documented, but the people accountable for the AI system must decide what those risks are, who answers for them and when a system may go live. These decisions cannot be delegated to a vendor or a consultant without leaving a gap that an assessor will notice.

Accountability structure: Name the owner of each AI system, the escalation path and the people who can approve overrides or exceptions.
Go or no-go decisions: Record who decided, on what evidence, and which risks were accepted, along with the date and any conditions.
Complaint and escalation process: Define intake channels, response targets and who reviews complaints about AI outputs.
Oversight statistics: Review override counts, reported errors and response times on a schedule, and act on the trends.
Policy exceptions: Approve and track each exception to policy, with an expiry and a named owner.

Confirm that these roles and records exist before any partner starts configuring tooling, because logs can only prove decisions that were made and written down.

Questions buyers ask about MEASURE 2.8 cost, effort and outsourcing

Can we meet MEASURE 2.8 without an outside partner?

Yes, the organization can do this work with its own staff, because the AI RMF is voluntary and no outside body must be engaged to use it. MEASURE 2.8 asks the organization to examine and document transparency and accountability risks, and the people accountable for the system have to own those records. An outside partner is useful when internal teams lack time, have no experience with TEVV documentation, or need help instrumenting logs across several platforms. Confirm internally who will own each evidence item before deciding whether to bring in help.

How much extra work does MEASURE 2.8 add if we already keep logs?

Usually less than building logging from scratch, but the gap is in content, not collection. Existing logs show what happened, while MEASURE 2.8 also asks for records of overrides, complaints, escalations, response times and go or no-go decisions by accountable parties. Teams often find these records scattered across tickets, chat threads and meeting notes. The added work is defining what counts as an override or escalation, agreeing on retention, and checking that logs cover the AI components in scope. Ask your team to inventory current logs against the Playbook's suggested actions before estimating anything.

What does MEASURE 2.8 need from our internal staff?

It needs named accountable owners, reviewers who can judge oversight records, and engineering time to instrument systems. The NIST AI RMF Playbook tags MEASURE 2.8 to several AI actor categories, including TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment and AI Deployment, so more than one team is involved. Typical internal roles are:

An accountable executive who signs off go or no-go decisions
A TEVV lead who defines metrics and test documentation
Operations staff who record overrides, complaints and response times
Engineers who configure audit trails and dashboards

The tasks of each actor category are described in Appendix A of NIST AI 100-1.

How long does it take to produce a first MEASURE 2.8 evidence set?

No duration is published for this work, so the time depends on your own environment and should be estimated from a written scope. The main drivers are how many AI systems are in scope, how much of the required logging already exists, whether accountable parties have defined go or no-go procedures, and how quickly reviewers can work through backlogs of complaints and overrides. Teams with mature ticketing and audit logging generally move faster than teams that must first agree on definitions. Ask any partner to break the schedule into inventory, instrumentation, documentation and review phases.

Can we get a certificate proving MEASURE 2.8 is met?

No. The AI RMF has no certification, accreditation or conformity assessment program, and NIST does not validate or endorse any organization's use of the framework. Any claim of AI RMF certification should be treated with caution. What you can produce is an evidence set showing how your organization examines and documents transparency and accountability risks, which a customer, auditor or regulator can review. If a contract or regulator asks for independent assurance, ask which assessor would issue it, since only an independent auditor or certification body can issue a report or certificate against a specific standard.

What should we ask a partner for in a fixed-scope written quote?

Ask for a quote built from a written scope that names the AI systems, the MEASURE 2.8 suggested actions covered, the deliverables and the timeline. A useful quote separates one-time work, such as instrumenting logs and writing procedures, from recurring work, such as reviewing oversight statistics and maintaining records. It should state who owns each deliverable, what the customer's staff must provide, which tools will be deployed, and what evidence will be handed over. DeployOpen works from fixed-scope written agreements and does not publish prices, so each quote follows from the scope you define. Confirm the handover format and who keeps the records after the engagement ends.

Which open-source tools can produce MEASURE 2.8 records?

Several self-hosted tools can capture parts of the evidence, though none satisfies MEASURE 2.8 alone. Nextcloud's admin_audit app logs event details to a file, OpenBao's audit devices keep detailed JSON logs of API requests and responses, and Langfuse traces capture inputs, outputs, timing and metadata for LLM applications. For generative AI testing, garak writes a JSONL report of every attempt and a hit log of vulnerabilities found. Confirm each tool's current behavior in your version, and check that logs cover the oversight, override and escalation records that the Playbook suggests.

How DeployOpen scopes and prepares MEASURE 2.8 evidence

DeployOpen helps organizations set up the logs, audit trails, oversight records and decision records that MEASURE 2.8 asks them to examine and document. The organization keeps ownership of its risks, accountable roles and go or no-go decisions, while DeployOpen prepares the self-hosted systems and evidence those decisions depend on.

How we scope

Agree in writing which AI systems, components and risks from MAP fall inside MEASURE 2.8 scope.
Confirm which accountable parties, oversight roles and decision records the organization already has.
List the transparency and accountability evidence to produce, with owners and agreed deliverables before work starts.

How we build

Configure audit logging on self-hosted platforms, such as Nextcloud's admin_audit app, inside the client's own infrastructure.
Set up trace and request logging for LLM applications, including inputs, outputs, tool calls and token usage, where the scope requires it.
Test that logs capture overrides, escalations and policy exceptions, and confirm that retention and access controls on those logs work as designed.

How we hand over

Deliver written documentation of each log source, what it records, where it is stored and who reviews it.
Provide templates for oversight statistics, complaint and error tracking, and go or no-go decision records for the organization to maintain.
Walk the organization's teams through the evidence set so they can produce it again without outside help.

How we support

Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
Ongoing support, upgrades and continuous testing are available as a retainer for logs and records that must keep working after handover.
Confirm with your auditor or assessor which evidence they will accept, since only independent assessors issue reports or certificates.

MEASURE 2.8 Transparency and accountability