MANAGE 4.2 Continual improvement activities

MANAGE 4.2 only works if every update to an AI system is measured before and after the change. The outcome asks for measurable continual improvement built into system updates, with regular engagement with interested parties, and that measurement depends on MEASURE. The MEASURE functions supply the metrics and production monitoring that give an update its baseline. Without a baseline recorded before a change, an organization cannot show whether the change improved trustworthiness or only moved the risk somewhere else. Within the loop, MANAGE 4.1 supplies the monitoring plan, user input, override and change management records that feed improvement work, and MANAGE 4.3 governs how incidents and errors are communicated and documented once they surface. A practical check is to ask, for each planned release, which metric it is expected to move, what value it had before the change, and where the post-change result will be recorded.

The Playbook suggestions for MANAGE 4.2 give the working steps. Organizations should integrate the trustworthiness characteristics into their continual improvement protocols and metrics, so that a change to a model, prompt, retrieval source or interface is judged against characteristics such as validity, safety, privacy and fairness rather than only against uptime or throughput. Feedback should be evaluated before it is integrated. Someone with the authority to decide should record why a piece of feedback was accepted, deferred or rejected. Proposed improvements should also be checked against applicable legal frameworks and against context-specific values, which means the review involves legal and domain owners, not only the engineering team. Where an improvement trades one characteristic against another, such as accuracy against explainability, the basis for that tradeoff decision should be documented so that a later reviewer can reconstruct it.

Interested parties and relevant AI actors are named in the Playbook for this subcategory: TEVV, AI design, AI development, AI deployment, operation and monitoring, end users, and affected individuals and communities. Appendix A of NIST AI 100-1 describes the tasks of each AI actor category, and it is a useful starting point for assigning who collects feedback, who adjudicates it and who signs off on changes. Confirm with your own governance owners how often these engagements occur and which forums carry them, because the framework describes the outcome but does not set a schedule. For generative AI, the Generative AI Profile adds actions under MANAGE 4.2 that call for regular monitoring, published reports on performance, feedback and improvements, and post-mortem reviews after harmful content incidents. Tools such as GLPI can hold change and problem records, Zammad can hold feedback tickets from users, and Langfuse traces can supply before and after performance data, but each must be configured to record the measurement itself.

MANAGE 4.2 asks whether improvement is measured and built into how AI systems change, so the evidence an assessor expects is a set of linked records rather than a policy statement. The core artifacts are metric definitions with baselines, an improvement backlog where each item traces back to a feedback source, release records showing the metric before and after each change, stakeholder engagement logs, tradeoff decision records, and periodic performance reports. Assessors typically sample a release and ask whether the team can show what was measured, what changed, who was consulted and why the tradeoff was accepted. The NIST AI RMF is voluntary and no one certifies against it, so the assessor here is usually an internal audit function, a customer's security team or a regulator's examiner who will judge the evidence on its own merits. Confirm with that reader which artifacts they want in what format before you build the set.

The weakest point is usually the link between a release and its metric change. Tie each release record to the metric it was meant to move, the baseline value, the post-release value and the date both were measured, and reference the backlog item and feedback record that prompted the change. Record tradeoff decisions in the same place, noting which trustworthiness characteristic gained and which lost, and the legal and context-specific values checked. Engagement logs should capture who was contacted, which AI actor category they belong to, what they reported and what happened to it, including feedback the team rejected. For generative AI systems, the Generative AI Profile describes monitoring reports on performance, feedback and improvements, so a periodic report template can carry that structure. If your tooling stores these records, confirm with the vendor that it can export history with timestamps, since an assessor will ask how you know a record was not edited afterward.

Keeping this current between audits is mostly a scheduling discipline. Set a recurring review date for each metric definition and for the backlog, and record who reviews it and what they changed, so the file shows the loop running over time. A single well-documented release tells an assessor little; a sequence of releases, each with its own before and after result, shows that improvement is an operating habit. Plan for the cost side as well: the people hours for metric reviews, the time to write release records, and any tooling licenses or hosting are recurring, while setting up metric definitions and the first evidence template is largely one-time. An implementation partner can design the record structure, wire the systems that generate it and document the review cycle, but the organization owns the decisions and the records. Ask any partner to show the evidence set for one sample release before agreeing to a fixed scope.