What MANAGE 2.3 requires when an AI system surprises you
When a deployed AI system shows a risk nobody mapped, MANAGE 2.3 expects your organization to follow procedures that respond to the risk and recover from it once it has been identified. The subcategory is part of the Manage function in NIST AI 100-1, the *Artificial Intelligence Risk Management Framework (AI RMF 1.0)*. The framework is voluntary, and NIST does not certify or accredit any organization's use of it, so no outside body will hand you a badge for meeting MANAGE 2.3. What an assessor, customer or regulator will look for is evidence that the procedures existed, were followed and produced a result. The outcome itself is simple to state. The difficulty lies in building the machinery before the surprise arrives.
The work is hard for two reasons. First, an unknown risk cannot be pre-scripted, so a playbook written today can only cover the risks you can already imagine. The most useful preparation is therefore a generic response process: who declares an incident, who can pause or disengage the system, who communicates with affected parties and how the case is closed. Second, you cannot respond to what you cannot see. Monitoring for output quality, drift and changes in how users actually employ the system has to be running before the unexpected event, with logs and metrics retained long enough to reconstruct what happened. The Playbook offers suggested actions for this subcategory, including keeping protocols and metrics for continual monitoring, checking components for drift and verifying that mission-critical systems can be deactivated. These are suggestions, not required steps, so your organization should decide which ones fit its risk tolerance.
Before you commit budget, confirm three things with your team. Check whether your monitoring already records enough inputs, outputs and metadata to reconstruct an incident. Confirm who holds authority to deactivate the system and whether that authority has been tested. Ask whether the response plan has been rehearsed against a scenario nobody has written down yet. The Playbook is the voluntary companion to AI 100-1, and its audit log shows the last content update in August 2023, so read its suggestions as a starting point and check current versions with NIST. Open-source tooling such as DFIR-IRIS can help hold case records and evidence, and Langfuse can hold trace data, but whether either fits your environment is a decision to confirm during scoping.
Which AI systems, teams and third parties fall under MANAGE 2.3
MANAGE 2.3 applies once a system is live and a new risk surfaces, so the scoping question is which deployed systems, components and parties must be ready to respond. The cards below cover the systems that warrant monitoring plans, how generative AI widens the picture, and how third-party models and vendors change who acts.
Deployed systems that need response plans
Not every deployed model needs the same monitoring and response depth. Start with systems whose outputs affect people, money, safety or regulated decisions, and systems whose failure would be hard to detect or reverse. For each, the Playbook suggests keeping protocols and metrics for continual monitoring and reviewing treatment and response plans for incidents. Check that every in-scope system has a named owner, a documented risk tolerance and a plan that can be exercised before an incident occurs.
Components that drift unnoticed
Monitoring should cover the parts of the system that change without a code release: input data pipelines, retrieval sources, prompts, business rules and upstream model versions. The Playbook suggests monitoring components for drift and decontextualization, meaning a system still runs but is applied to conditions it was not built for. Confirm that each component has an owner, a baseline and an alert threshold, and that changes to any component reopen the scope decision.
Generative AI and value chain integration
The Generative AI Profile (NIST AI 600-1) adds one suggested action for MANAGE 2.3, MG-2.3-001, tagged to the risk of value chain and component integration. It asks organizations to develop and update incident response and recovery plans, including for newly encountered uses. For a system built on a foundation model, this means the plan must cover the model provider, the orchestration layer and any tools the model can call, not only the application the business owns. Check whether the plan names who responds when a failure starts upstream.
Third-party models and vendors
When a vendor supplies a model, API or hosted component, the organization still owns the risk response for its own deployment, but the vendor controls the model's behavior and release schedule. MANAGE 2.3 planning should define who is notified of a vendor incident, who can suspend the integration, and how the organization would switch or disable the dependency. Ask vendors for their incident notification commitments and change logs in writing, and test whether internal staff can act on them without vendor help.
Teams with authority to act
A response plan fails when the people who can pause a system are not the people who detect the problem. Identify the operations, security, product and legal roles that share responsibility for each in-scope system, including who may deactivate it. Illustrative example: a monitoring alert fires at night, and the plan names an on-call engineer who can disable one tool integration without waiting for a business sign-off. Write down the escalation path and rehearse it.
How an assessor tests whether response procedures actually work
An independent assessor tests MANAGE 2.3 by asking whether response procedures exist and whether they have been used. NIST does not certify organizations against the AI RMF, so any review of MANAGE 2.3 is performed by a reviewer the customer or regulator selects and pays for, not by NIST. That reviewer, which may be an internal audit function, a consultant or an external assurance firm, will work from the organization's own records and the suggested actions in the NIST AI RMF Playbook, which the organization can adopt, adapt or set aside. The reviewer's first questions are practical: which monitoring protocols and metrics are in place for each deployed system, who owns them, and when they were last reviewed.
The assessor then looks for evidence that the response and treatment plans were tested in practice. A plan that has never been exercised is weak evidence, so the reviewer will ask for incident records, post-event reviews and tabletop or drill reports, and will check whether the plans were revised afterward to cover newly encountered uses. For generative AI systems, the reviewer may also check whether the plans account for the newly encountered uses named in NIST AI 600-1. Confirm with your reviewer which evidence they accept before you build the file, since expectations vary by engagement.
Finally, the assessor checks whether contingency and deactivation processes actually work. Expect requests to demonstrate that a mission-critical system can be switched off or rolled back, who is authorized to do it, and when that authority was last tested. Drift and decontextualization monitoring should show alerts that reached a named owner and led to a documented decision, including a decision to decommission a system that exceeded its risk tolerance. Ask your reviewer in advance whether they will request a live test or accept a recorded one, and plan the test so it does not disrupt production.
What assessors request, how they test it and common findings
This table compares six areas of MANAGE 2.3 by how an assessor would test them, what evidence they would ask for and where programs commonly fall short. Read each row as a question the assessor will put to the organization, with the evidence needed to answer it. Because NIST AI RMF is voluntary and has no certification, these tests are internal or independent reviews, and the organization owns the conclusions.
| Test area | Test approach and evidence requested | Common findings |
|---|---|---|
| Monitoring protocols | Assessor samples deployed AI systems and asks for the monitoring protocol, defined metrics, alert thresholds and records showing alerts were reviewed. Evidence includes monitoring configuration, dashboards and alert tickets with dates. | Monitoring is defined for launch but not refreshed after changes, or alerts exist without a named owner who reviews them. |
| Drift and decontextualization checks | Assessor asks how input data, model behavior and use context are compared against the baseline set at deployment. Evidence includes drift reports, comparison methods, and records of decisions taken when drift was found. | Drift is measured on model outputs but not on the operating context, such as a new user population or a new business process the system was not designed for. |
| Treatment and response plans | Assessor reviews written plans for previously unknown risks and checks whether they cover identification, triage, containment, recovery and communication. Evidence includes approved plans, version history and a tabletop or post-incident review record. | Plans exist for known incident types only, were never exercised, or do not cover generative AI uses that appeared after deployment. |
| Contingency process verification | Assessor asks whether fallback processes work when an AI component fails, and requests test records or drill results. Evidence includes runbooks, the date of the last test, and the outcome and follow-up actions of each test. | Contingency processes are documented but untested, or depend on a third-party AI service with no tested alternative. |
| Ability to deactivate mission-critical systems | Assessor asks who can disable the system, how quickly that can be done, and when it was last demonstrated. Evidence includes access records for the kill switch, the documented procedure and a recent test or change record. | Deactivation requires a vendor or a single engineer, or the procedure has never been run against the production configuration. |
| Decommissioning decisions | Assessor checks whether systems that exceed risk tolerances were retired or restricted, and whether the decision was approved by an accountable owner. Evidence includes decision records, risk tolerance thresholds, retirement plans and preserved logs or forensic material. | No decision record exists for systems kept in service after a threshold was breached, or retired systems lose their logs before review. |
How to build a response procedure before the next unknown risk
This sequence turns MANAGE 2.3 into a working procedure: triggers and severity criteria are agreed first, monitoring and treatment plans are built on them, and authority to contain or deactivate a system is assigned and rehearsed before an incident occurs. Each step ends with a record that an assessor can inspect.
Define triggers and severity criteria
Start by writing down what counts as a previously unknown risk for each AI system in scope, such as a new failure mode, an unexpected output pattern, a new misuse path, or an incident reported by users or monitoring. Agree severity levels and the thresholds that move a finding into the response procedure, with the risk owner and the system owner signing off. Done means a trigger table exists for each system, reviewed against the risk tolerances set under GOVERN 1 and MANAGE 1. Output: a severity and trigger matrix with named owners.
Set up continual monitoring and drift detection
Instrument each deployed system so the triggers can actually fire. Define the metrics, logging scope and review cadence that will reveal performance change, data drift and decontextualization, which the Playbook suggests monitoring under MANAGE 2.3. Tracing tools such as Langfuse can record inputs, outputs, tool calls and token usage per request, which gives responders the trail they need. Confirm the logging meets your privacy and retention obligations before turning it on. Done means alerts route to a named queue and a test event has been seen end to end. Output: a monitoring plan and a metrics register.
Write treatment and response plans
For each severity level, document the treatment options available, such as restricting inputs, adding a human review step, rolling back a model or prompt version, or switching to a non-AI method. Describe the response steps, the communication path to affected parties, and the criteria for returning to normal operation. Generative AI systems should also include plans for newly encountered uses, as NIST AI 600-1 suggests under MG-2.3-001. Done means each plan names its trigger, its actions and its approver. Output: a set of treatment and response plans, version controlled.
Define contingency and deactivation authority
Name who can pause, roll back, disengage or decommission each mission-critical system, and confirm that the technical path to do so works. Verify that the contingency process can run without the system, including manual or prior-version fallbacks, and that decommissioning follows your risk tolerances. Record who holds this authority when the usual owner is unavailable. Ask your platform team to demonstrate the disable action in a non-production environment. Done means a signed authority matrix exists and the disable path has been tested. Output: a contingency and deactivation runbook with an authority matrix.
Plan evidence preservation and case management
Decide in advance how incident material will be kept. A case management platform such as DFIR-IRIS can hold assets, indicators, tasks, evidence and a full audit trail in one record, with SHA-256 hashes and chain of custody for logged evidence. Preserve model versions, prompts, logs and configuration snapshots before any rollback changes them. Confirm retention periods with legal counsel, since the appropriate period depends on your sector and jurisdiction. Done means every response case has a template and a storage location. Output: an evidence handling procedure and case template.
Rehearse with tabletop exercises
Run a tabletop exercise against a realistic but invented scenario, such as an unexpected generative output reaching customers, and walk through triggers, severity calls, communication and deactivation decisions. Record the time each decision took, where people were unsure who owned it, and which plan step was missing. Treat the exercise as illustrative testing of the procedure, not as proof of control performance. Done means a written after-action report lists each gap and its owner. Output: an exercise report and a corrections log.
Record outputs, owners and review dates
Consolidate the artifacts into one register that links each system to its triggers, monitoring metrics, treatment plans, authority holders, evidence location and last exercise. Assign an owner and review date to every item, and set the procedure to be revisited after each real event and after any change to the system. Because NIST has said AI RMF 1.0 is being revised, check the register against the current NIST publications before each review. Done means the register is complete for all in-scope systems and an assessor could trace one trigger to one decision. Output: a response procedure register with owners and review dates.
What meeting MANAGE 2.3 costs in money and effort
MANAGE 2.3 has no certification fee because the AI RMF is voluntary and no body issues a certificate for it. The money goes to four parties: the organization, which owns its controls and carries staff time; software vendors or open-source projects, which may charge for licenses or support; an implementation partner, which prepares monitoring, runbooks and evidence under a written scope; and any independent assessor or external tester that the organization chooses to bring in. Each line item in a quote should name the payee and the deliverable.
- Organization staff: Time from security, platform, data science and product owners to define drift thresholds, approve treatment plans and run rollback drills.
- Tooling and licenses: Costs for incident case management, LLM observability and model test harnesses, which may be open source with no license fee but still need hosting and operations.
- Implementation partner work: Fixed-scope work to configure monitoring, write procedures and connect evidence stores, priced from the written scope.
- External testing: Red teaming or application security testing, paid to the tester and usually scoped separately.
- Assessor sampling: Fees an independent assessor charges if the organization commissions a formal audit against another framework that references this practice.
Ask each bidder to show which of these five payees their price covers and which it leaves to you.
Most MANAGE 2.3 spending is front-loaded in design and then becomes a running cost. Setup work produces the procedures, thresholds, dashboards and first contingency test. After that, the program consumes time through monitoring reviews, drift investigations, incident exercises and evidence refreshes. Separating these two streams in a budget prevents a launch project from looking cheaper than the program will be to operate.
- One-time: Mapping monitored components, defining drift and decontextualization thresholds, writing response and deactivation runbooks, and installing initial tooling.
- One-time: The first full contingency test, including a deactivation drill for each mission-critical system.
- Recurring: Monitoring triage, metric review and tuning of alert thresholds as the system and its data change.
- Recurring: Exercises, plan reviews and updates after any incident or new generative AI use case, which NIST AI 600-1 expects to be reflected in response plans.
- Recurring: Evidence retention, license renewals and hosting for logs and case records.
Ask for the recurring cost per year in the quote, not only the setup price.
The total cost of MANAGE 2.3 scales with how much AI is running and how much of it depends on other parties. Three drivers matter most. The number of deployed systems multiplies monitoring, runbooks and drills. Generative AI use adds prompt, output and model-behavior monitoring, and NIST AI 600-1 ties its MANAGE 2.3 action to value chain and component integration risk. Third-party dependence adds contract review, change notices and fallback planning for models or services the organization does not control.
- System count: Each mission-critical system needs its own thresholds, owner, runbook and deactivation test.
- Generative AI use: Prompt and output logging, jailbreak and data leakage testing, and more frequent review of incident plans for new uses.
- Third-party dependence: Fallback arrangements for pre-trained models and hosted services, plus monitoring of vendor changes.
- Regulatory scope: Sector rules or state laws that name the framework may require documentation depth that adds effort.
- Automation level: Mature telemetry and case tooling reduce manual triage but add engineering time at setup.
In a quote, ask which of these drivers the price assumes and what changes the price if a system is added.
Effort for MANAGE 2.3 is spread across roles rather than concentrated in one team. Platform engineers carry most of the rollback and drift work. Security staff own incident response and evidence. Application owners decide whether a system should be superseded or disengaged. No duration can be stated here because it depends on the number and complexity of systems in scope, which only a written scope can establish. Organizations should estimate effort from their own inventory.
- Platform engineering: Building drift signals, rollback paths and the ability to deactivate systems without breaking dependent services.
- Security operations: Triage of alerts, case handling and forensic preservation when a system is deactivated.
- Application owners: Approving thresholds and risk tolerance decisions tied to MANAGE 2.3 outcomes.
- Governance and legal: Reviewing treatment plans and communications, often on an approval cycle.
- Trainers and drill leads: Running contingency exercises that prove the procedures work.
Ask each partner to show the roles and hours assumed in the scope, and confirm which of those roles you must staff yourselves.
NIST does not validate or endorse any organization's use of the AI RMF, so there is no assessor who tests MANAGE 2.3 against a NIST standard. Costs arise when an organization asks an independent auditor to examine AI controls under another framework, or when a customer requires third-party review. Auditors then sample records of monitoring alerts, drift decisions, incident cases and deactivation tests. Evidence upkeep is the recurring work that makes sampling affordable. Confirm the auditor's scope and sample method before the engagement starts, since those choices set the fee.
- Sampling scope: The number of systems, alerts and incidents an auditor selects to examine.
- Evidence quality: Case records with timestamps, approvals and audit trails that need little manual reconstruction.
- Evidence retention: Storage and access controls for logs and case files over the retention period.
- Remediation cycles: Extra work and re-testing when a sample finds a gap.
- Independent review: Fees paid to the auditor, which DeployOpen does not issue or influence.
Ask the auditor how many items they expect to sample and whether they accept evidence produced by your existing tools.
Which self-hosted tools produce MANAGE 2.3 evidence
This table compares three self-hosted open-source tools by the MANAGE 2.3 evidence each can generate: incident case records, adversarial probe reports, and production traces. Read each row as a question to confirm in your environment, since a tool supports evidence but does not make a response process compliant on its own. The main takeaway is that the tools cover different parts of the monitoring and response cycle, so most programs will need more than one.
| Tool | Evidence it can produce for MANAGE 2.3 | Monitoring or response role | What to confirm before relying on it |
|---|---|---|---|
| DFIR-IRIS (LGPL-3.0-or-later) | One case record linking assets, IOCs, tasks and evidence, with a full audit trail | Response and recovery: records how a newly identified risk was triaged, investigated and closed | Whether alert intake from your SIEM, EDR or REST sources matches your incident workflow |
| DFIR-IRIS (LGPL-3.0-or-later) | Evidence logs with SHA-256 verification and chain of custody | Preserves material from a risk event so later review or deactivation decisions can be traced | Who holds administrative rights and how case exports are protected |
| DFIR-IRIS (LGPL-3.0-or-later) | Consolidated investigation timeline and generated reports from case data | Supports documented review of response plans after an incident | Whether generated report templates satisfy your internal review requirements |
| garak (Apache-2.0) | JSONL report of every probe attempt and evaluation | Continual monitoring for generative AI systems against probed failure modes such as prompt injection and data leakage | Which probes match the risks in your own risk register, since coverage is not complete by default |
| garak (Apache-2.0) | Hit log of attempts that found a vulnerability | Feeds treatment plans and contingency reviews when a previously unknown weakness appears | How hits are routed to owners and whether re-testing confirms a fix |
| Langfuse (MIT) | Traces with timing, inputs, outputs, token usage and metadata for each operation | Drift and anomaly detection in production, including retrieval, tool execution and LLM calls | Data retention settings and redaction of personal data captured in inputs and outputs |
Who owns response procedures, who prepares evidence and who audits
Your organization owns the response procedures, the risk decisions and the authority to stop a system. NIST's AI RMF is voluntary and there is no certification, so no outside party can make these decisions for you. An assessor will look for named owners who can act on a previously unknown risk, including the power to deactivate a system without waiting for a committee to meet.
- Procedure ownership: A named owner maintains the incident response and recovery plans, including procedures for uses that were not anticipated when the system was approved.
- Decision rights: Documented thresholds state when a drift finding or new risk triggers treatment, escalation or deactivation, and who signs off.
- Deactivation authority: Specific roles can disengage a mission-critical system, and the fallback process has been confirmed to work.
- Risk tolerance and residual risk: Leadership accepts or rejects residual risk in writing, based on the monitoring results.
- Lessons learned: The organization records what happened, updates the procedures and tracks the changes to completion.
Confirm that each role named in the procedures has actually been briefed and that the authority described on paper matches what your change and access systems permit.
An implementation partner such as DeployOpen can prepare the systems and evidence that support MANAGE 2.3. The partner builds and configures the tooling, drafts runbooks for your review and runs exercises, but it does not own the risk decisions and does not issue any assessment result. Scope and deliverables are agreed in writing before work starts, so you can see what is included and what remains your responsibility.
- Monitoring and drift checks: Setup of protocols and metrics for continual monitoring, including drift and decontextualization checks, with dashboards your team reviews.
- Case and evidence tooling: Self-hosted tools such as DFIR-IRIS can keep incident records, timelines and chain-of-custody logs in one place, with the audit trail intact.
- Runbook drafts: Written procedures for treatment, contingency and deactivation, delivered for your approval and maintenance.
- Exercises: Tabletop or technical exercises that test whether the contingency and deactivation steps work in practice, with a written record of results.
- Evidence packages: Organized artifacts (configurations, logs, exercise reports) mapped to the MANAGE 2.3 suggested actions for the assessor.
For each deliverable, ask who approves it, who operates it after handover and whether ongoing support is needed under a separate retainer.
An independent auditor or assessor tests the controls and issues any report or attestation. Because the AI RMF has no certification program, NIST does not validate or endorse any individual organization's use of the framework. Any formal opinion on your MANAGE 2.3 practices therefore comes from an auditor chosen under a separate engagement, typically one that also covers a standard your customers or regulators require.
- Independence: The auditor cannot design the procedures it later tests, and should not be the party that ran the exercises it evaluates.
- Testing: The auditor samples incidents, drift alerts, deactivation records and exercise results, and checks whether the procedures were followed.
- Reporting: Only the auditor issues the report. Your implementation partner's documents are inputs to the audit, not conclusions.
- Scope agreement: Confirm in writing which framework and which version the auditor will assess against, since the AI RMF is being revised.
- Timing: Ask the auditor how it handles evidence gathered before the audit period begins.
Confirm what the auditor will accept as evidence and whether it has reviewed the tooling in your environment before the engagement starts.
Where MANAGE 2.3 programs usually fall short
MANAGE 2.3 asks whether an organization can respond to and recover from a risk nobody anticipated. The gaps below tend to appear together: monitoring that never triggers action, plans that exist only on paper, and records that cannot reconstruct events. Each card names the gap and the check that closes it.
Monitoring with no defined triggers
Teams often collect logs, dashboards and metrics for a deployed model but never define the thresholds that start a response. The Playbook suggests keeping protocols and metrics for continual monitoring, so the check is whether each metric has a written threshold, an named owner and a documented action when it is crossed. Ask for a sample alert and trace who received it, what they decided and when.
Drift detection that goes unreviewed
Drift and decontextualization, where a model is used outside the context it was built for, are easy to miss when no one compares live behavior with the assumptions made at deployment. Confirm that inputs, outputs and usage patterns are compared against a documented baseline on a set schedule. Record who reviews the comparison and what happens when the system moves outside its intended use.
Response plans never rehearsed
An incident response plan that has never been run usually fails on its first real use because roles, contact lists and access are out of date. Use tabletop exercises that include a novel AI failure, such as a use the system was not designed for. Check that each exercise produces a written list of gaps and that the plan is updated afterward. NIST AI 600-1 asks generative AI programs to account for newly encountered uses in their response plans.
No owner with authority to deactivate
Teams often discover that the person who can detect a problem cannot switch the system off, and the person who can switch it off is not notified. Assign a named owner with written authority to disengage each mission-critical system, and verify that the deactivation path works in production. Confirm the owner can act without waiting for approval from the teams that built the system.
Third-party models without contingency
Many deployments depend on a hosted model or vendor API that the organization cannot inspect or restart. When the provider changes behavior, degrades or withdraws the service, the business has no fallback. Require a documented contingency process for each third-party model, including a non-AI alternative or prior version that can run, and test that the switch works. Ask vendors how they notify customers of model changes.
Records that cannot show what happened
After an incident, investigators need a consistent timeline of prompts, outputs, tool calls, configuration changes and human decisions. Gaps appear when logs are kept in separate systems, traces omit tool executions, or evidence is not preserved in a verifiable form. Check that each case is stored as one record with a timeline, that evidence has integrity checks and chain of custody, and that traces capture every operation in a request.
Questions teams ask about MANAGE 2.3 cost, effort and evidence
Can we meet MANAGE 2.3 in-house without outside help?
Yes, in most cases the organization can do the work itself, because the AI RMF is voluntary and MANAGE 2.3 asks for procedures you run, not a product you buy. The work is writing and testing the response, monitoring, drift and deactivation procedures, then showing they operate. In-house teams usually struggle with the monitoring instrumentation, incident tabletop exercises and keeping evidence consistent across systems. An implementation partner can take on those pieces under a written scope, while the organization keeps ownership of the decisions, risk tolerances and accountability. Confirm internally who owns each procedure before deciding what to outsource.
How long does MANAGE 2.3 take and what drives the effort?
No supplied source gives a duration for MANAGE 2.3, so the timeline depends on your environment. Effort rises with the number of AI systems in production, how many of them are generative AI systems that need the GAI-specific action MG-2.3-001 addressed, how mature your existing incident response and monitoring are, and how many teams must approve plans. Systems with mission-critical roles need tested contingency and deactivation paths, which take longer to verify than a monitoring dashboard. Ask any provider for a phased plan that lists the systems in scope and the procedures delivered for each.
What does MANAGE 2.3 require from our staff?
It requires named owners across operations, security, data science and the business owner of each system. Deployment and operation staff run monitoring and drift checks, security staff handle incident response and recovery, and the business owner decides when to deactivate a system or accept a residual risk. The Playbook tags MANAGE 2.3 with the AI actor categories AI Deployment and Operation and Monitoring, and AI 100-1 Appendix A describes their tasks, so use those categories to map roles. Plan time for staff to rehearse response steps, not only to write them.
Is there a certificate for MANAGE 2.3 compliance?
No. The NIST AI RMF has no certification, accreditation or conformity assessment program, and NIST does not validate or endorse any organization's use of the framework. Any claim of MANAGE 2.3 certification should be treated with caution. Evidence for the subcategory is what a reviewer, customer or auditor examines: documented procedures, monitoring records, tested contingency plans and deactivation records. Independent auditors or assessors can examine that evidence against your own stated approach, but they do not issue a NIST certificate. Ask any firm what their report actually states.
What evidence should we keep for MANAGE 2.3?
Keep records that show the procedure ran, not only that it exists. Useful evidence includes monitoring protocols and metric definitions, drift and decontextualization review logs, incident response and recovery plan versions with review dates, exercise reports, and records of any deactivation or decommissioning decision. For generative AI, keep documentation of plans updated for newly encountered uses. Tools such as DFIR-IRIS can hold incident cases with an audit trail and chain of custody for evidence, and Langfuse traces can record operations for review, subject to your own configuration and data handling rules.
What should we request in a quote for MANAGE 2.3 work?
Ask for a written scope that lists the AI systems covered, the procedures to be delivered, the systems and cloud accounts touched, and the evidence each deliverable will produce. Request the roles of your staff and the provider's staff, what is one-time work such as drafting plans and configuring monitoring, and what recurs such as reviews, exercises and retainers. Ask what is excluded, including legal advice, audit fees and changes to models. DeployOpen works from a written scope and provides no price list, so a quote follows from that scope. Confirm acceptance criteria before work starts.
Which parts of MANAGE 2.3 do organizations usually outsource?
Organizations most often outsource the technical build: wiring monitoring and drift detection into pipelines, configuring incident case tooling, setting up logging, and running tabletop exercises or security testing of response paths. They usually keep the decisions: approving treatment plans, accepting residual risk, and choosing to deactivate a system. Where an external auditor will later review the work, keep that reviewer independent of the team that built the controls. Confirm this split with your own risk and legal teams before agreeing a scope.
Evidence that passes for MANAGE 2.3 and how to keep it current
An assessor testing MANAGE 2.3 looks for proof that the organization detected new risks, responded to them and can still switch a system off. The cards below cover the artifacts that usually carry that proof and the routine each one needs to stay current. The organization owns these records; an implementation partner can help build them, but only an independent assessor can judge whether they are sufficient.
Monitoring protocols and metric records
The monitoring protocol names what is watched, how often, by whom and against which thresholds, and the metric records show those checks actually ran. Assessors look for dated entries that tie back to specific risks identified in MAP and MEASURE. Check that every metric in the protocol has a recent record, and that a metric with no entries is either retired or fixed. Review the protocol whenever the system's inputs, users or business context change.
Drift and decontextualization reports
Drift reports record changes in input data, model behavior or the context of use compared with the baseline set at deployment. The Playbook asks organizations to monitor components for drift and decontextualization, which means checking whether a system is still used for the purpose it was validated for. Keep each report with the baseline, the measured change and the decision taken. Confirm the reporting cadence with the system owner, since the right frequency depends on how quickly the data changes.
Incident response and treatment plans
Response and treatment plans set out who declares a previously unknown risk, how it is triaged, what containment steps apply and how recovery is confirmed. NIST AI 600-1 asks generative AI programs to develop and update response and recovery plans, including for newly encountered uses. Review each plan after every real incident or near miss, and record the version history so an assessor can see when the plan changed and why.
Exercise and tabletop records
Exercise records show that the response plan was tested before it was needed. A tabletop or technical exercise should record the scenario, the participants, the time each decision took, the gaps found and the corrective actions assigned. These records are often the weakest part of an AI program, so schedule exercises that include a previously unseen failure mode, not only the scenarios the team already knows. Keep the lessons learned with the plan they changed.
Contingency and deactivation test results
The Playbook asks organizations to verify contingency processes and the ability to deactivate mission-critical systems. A deactivation test records whether the system can be switched off or bypassed, how long the fallback process took and whether the manual or non-AI alternative produced acceptable outcomes. Run these tests in a non-production environment where possible, and record the failed attempts as well as the passed ones. Ask the platform team to confirm the test covers the actual production path, not a simplified copy.
Decommissioning decisions
A decommissioning decision records why a system was retired because it exceeded its risk tolerance, who approved the call and what happened to its data, logs and models. Keep the decision with the risk assessment that triggered it and the evidence that the system was disconnected. Assessors commonly ask whether retired systems are still reachable, so confirm that access, credentials and scheduled jobs were removed and that the record of that removal is retained according to the organization's retention policy.
How DeployOpen prepares MANAGE 2.3 response systems and evidence
DeployOpen plans, implements and documents the self-hosted open-source tooling an organization uses to detect, respond to and recover from a previously unknown AI risk, inside the organization's own infrastructure. The organization owns its response procedures and decisions, and an independent auditor or assessor reviews the evidence, since NIST AI RMF has no certification.
How we scope
- We agree a written scope listing the AI systems, incident channels and monitoring sources that the response work covers.
- The scope names which MANAGE 2.3 suggested actions the organization wants prepared, including drift monitoring, contingency and deactivation checks.
- Engagements are fixed-scope, so deliverables and timelines are agreed in writing before any work starts, and a quote follows from that scope.
How we build
- We deploy incident case management such as DFIR-IRIS, keeping each case as one record of assets, indicators, tasks and evidence with an audit trail.
- We configure trace and observability tooling such as Langfuse so that model calls, retrieval steps and tool executions can be reviewed after an event.
- We set up scheduled adversarial probing with garak, with JSONL reports and hit logs kept as dated evidence of testing that was performed.
How we hand over
- We document the response and recovery procedures, the owners of each step and the escalation paths that the organization approves.
- We deliver runbooks for deactivating mission-critical AI systems and records showing how contingency processes were verified before handover.
- We provide an evidence index that maps each artifact to the MANAGE 2.3 subcategory it supports, for review by the organization and its assessor.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers after the initial handover.
- Retainer work can cover rerunning monitoring checks, reviewing alert triage quality and updating response plans when new AI uses appear.
- Before engaging, ask us to confirm which parts of the response program your own staff will run and which parts we will support.

