What the NIST AI RMF go or no-go determination requires
MANAGE 1.1 asks the organization to decide whether an AI system achieves its intended purposes and stated objectives, and whether its development or deployment should proceed. It sits in the Manage function (NIST AI 100-1, Table 4) and is the point where the work done in MAP and MEASURE has to become a decision. Someone must answer two questions in writing: does the system do what it was built and approved to do, and do the remaining risks justify putting it into use. Nothing in the framework makes this decision for you, and no outside body will certify the answer.
The decision has to rest on MAP and MEASURE outputs, not on a launch date or on the enthusiasm of the delivery team. MAP establishes the intended purpose, the context, the affected people and the identified risks. MEASURE produces the test, evaluation and monitoring results that show how the system performs against each trustworthiness characteristic. The Playbook suggests using these TEVV outputs when weighing risk treatment, comparing negative risks and benefits, and testing tradeoffs against real-world use and impacts. The hard part is that the evidence is rarely clean: a system can be accurate overall and still fail for one affected group, and the tradeoffs between characteristics such as safety, privacy and explainability often pull in opposite directions. Write down which results were weighed, which were not available and who accepted the remaining gap.
The framework is voluntary, and the NIST AI RMF has no certification, accreditation or conformity assessment program, so no one can issue a go or no-go certificate. Your organization owns the decision, and the named accountable executive should sign it. An implementation partner can help assemble the MAP and MEASURE records, set up test evidence and draft the decision memo, but the partner does not decide or attest. Confirm with your legal and risk teams whether any law or contract that cites the framework expects a specific decision record. Before the meeting, ask who can stop deployment, what conditions would trigger a no-go, and where the decision will be stored for later review.
Which AI systems and decisions fall under a go or no-go review
MANAGE 1.1 asks whether a system meets its intended purpose and whether it should proceed, so the review has to reach every system, stage and change that could alter that answer. The cards below cover the triggers, the components and the teams that bring work into scope, along with the scoping decisions that change how much work the review involves.
Systems that reach people or decisions
Start with the systems whose outputs affect people, money, access or safety, including internal tools that feed hiring, lending, clinical or security decisions. A system can look low-risk because it is a small model or a back-office utility, yet still change outcomes for customers or staff. Record the intended purpose and the decisions it supports, then check whether a failure would reach someone outside the building. Systems that cannot state an intended purpose cannot be given a go decision at all.
Development and pre-deployment gates
The first determination usually comes before a system is built further or released, when MAP and MEASURE outputs are available for the first time. Treat the gate as a required step in the delivery process, with named approvers and a written record of the decision. Ask whether test results cover deployment-like conditions, not only lab data. A pilot that reaches real users counts as deployment for this purpose.
Major changes to an approved system
A new model version, a change in training data, a new user group, a new tool connection or a shift in the decisions the output supports can each make an earlier go decision stale. Define in advance which changes re-open the determination and which are routine updates. Confirm that the re-test covers the changed component, since a prompt or retrieval change can alter behavior as much as a new model does. Keep a list of approved configurations so reviewers can see what was actually approved.
Pause and retirement decisions
The same determination applies when an operating system should be paused or withdrawn, for example after monitoring shows harms that current controls cannot manage. NIST's guidance on unacceptable negative risk says development and deployment should cease in a safe manner until risks can be sufficiently managed. Agree in advance who can order a pause, how users are told, and how data and dependent processes are handled. A retirement plan that exists only in a meeting note is hard to act on under pressure.
Third-party models as components
A pre-trained model, hosted API, embedded vendor feature or open-weight checkpoint is usually part of the system under review, because it shapes outputs even when the organization did not build it. The scoping choice is whether the review covers the component's behavior in your use case or only the contract. Ask vendors for evaluation results, change notice practices and data handling terms, and confirm what you can test yourself. Components with no evidence available still belong in the record as open risks.
Data sources and the teams that approve
Training, fine-tuning, retrieval and evaluation data each carry their own permissions, quality limits and consent conditions, and any of them can change the answer. The determination also needs a defined set of approvers, normally drawn from the business owner, the technical owner, risk or compliance, and people who can speak for affected users. Confirm that the approvers have authority to stop a release, not only to comment on it. Where those roles are unclear, the determination is unlikely to hold up under review.
How an assessor tests the go or no-go decision
This table sets out what an assessor checks for MANAGE 1.1, step by step, from whether a decision exists to whether trade-offs against the trustworthiness characteristics were weighed. Read each row as a test question, the evidence the assessor asks for, and the failure that most often leads to a finding. The main takeaway is that a decision without a recorded rationale, prior MAP and MEASURE inputs, and a clear owner is hard to defend, even when the system is ultimately sound.
| Test question | Evidence the assessor requests | Common finding |
|---|---|---|
| Does a documented go or no-go decision exist? | Signed decision record for each AI system in scope, with date, system version and decision outcome (go, conditional go or no-go) | No decision record, or only informal approval in chat or meeting notes |
| Who made the decision and did they hold the authority? | Approval matrix or RACI showing the accountable executive or governance body, and delegation records | Decision made by a team lead with no delegated authority, or accountability unclear |
| Was the decision made after testing finished? | Timeline showing MEASURE test completion dates before the decision date, and any release gates in the change record | Decision recorded before TEVV results were available, or testing still open at go-live |
| Which MAP outputs informed the decision? | Intended purpose, context, risk categorization and impact characterization documents referenced in the decision record | Decision references a use case that differs from the one documented in MAP |
| Which MEASURE results informed the decision? | TEVV reports, metric values against stated thresholds, and any deployment-like or monitoring results cited | Metrics measured but not linked to the decision, or results omitted without explanation |
| Were negative risks weighed against benefits and the trustworthiness characteristics? | Trade-off analysis covering validity, safety, security, transparency, explainability, privacy and fairness, with the reasoning for each | Only accuracy or cost discussed, with no record of the other characteristics |
| Was the decision tied to real-world use cases and impacts? | Impact assessment or stakeholder input relevant to the intended deployment setting | Evaluation performed only on benchmark data without deployment context |
| Was a no-go or conditional outcome acted on? | Evidence that blocked release was held, conditions were met before go-live, or the system was withdrawn | Conditions recorded but never verified, or a no-go overridden without documented approval |
How to set up a repeatable go or no-go process
This sequence turns MAP and MEASURE outputs into a documented decision on whether an AI system meets its intended purpose and should be developed or deployed, with named approvers and clear conditions for stopping. Each step ends with an artifact that a reviewer or assessor can inspect.
Fix the intended purpose and objectives
The product owner and risk lead restate the intended purpose, stated objectives, intended users and out-of-scope uses using the MAP 1 context record. Legal, business and affected-function representatives confirm the wording before any evaluation results are reviewed, so the decision is tested against the purpose the system was approved to serve. Done when the purpose statement is versioned, signed off, and linked from the go or no-go file.
Assemble TEVV results from MAP and MEASURE
The AI risk function gathers the test, evaluation, verification and validation outputs produced under MAP 2 and MEASURE, including metric results, known limitations, unresolved findings and monitoring baselines. Each result is tagged to the risk it addresses and to the trustworthiness characteristic it measures. Done when every high-priority risk from MAP has a matching result or a documented gap that the reviewers must accept or close.
Weigh negative risks against benefits
The risk committee sets negative risks beside the expected benefits and reviews each one against the seven trustworthiness characteristics: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. Reviewers note where a characteristic is weak and whether the weakness can be contained. Done when the weighing is written per characteristic and each weak characteristic has a stated treatment or an explicit acceptance.
Test tradeoffs against real-world use
The team compares the results with how the system will actually be used, including operator skill, workflow pressure, affected populations and foreseeable misuse, drawing on the MAP 3 and MAP 5 impact characterization. Where pilot or staging evidence exists, it is used in place of assumptions about use. Done when the tradeoff analysis names the real-world scenarios tested, the impacts found and the people consulted, and the record shows where results are still based on assumptions.
Record the decision with named approvers
The accountable executive records one of three outcomes: go, go with conditions, or no-go. The record names each approver, their role and their decision date, and it links to the TEVV evidence and the weighing from earlier steps. Under the Govern accountability structures, the approver must hold authority over the risk being accepted. Done when the signed decision record is stored with the AI inventory entry for the system.
Define conditions for continued operation
For a conditional go, the team writes each condition as a measurable check, with an owner, the evidence required and the trigger that would reopen the decision, such as a metric crossing a threshold or a new incident category. The conditions feed the MANAGE 2 monitoring plan and the MEASURE 3 tracking. Done when every condition has a named owner and a monitoring source, and the operations team confirms it can observe each trigger.
Set the cease path before deployment
The team documents how development or deployment would stop in a safe manner if an unacceptable negative risk appears, including the criteria, the person authorized to halt the system, the rollback or fallback process and the notification list. The AI 100-1 guidance on ceasing until risks can be managed is the reference point for the criteria. Done when the cease procedure has been walked through with operators and the halt authority is recorded in the decision file.
Re-run the decision on material change
The go or no-go determination is repeated when the model, data, intended purpose, user population or key third-party component changes materially, and the approvers confirm whether the earlier decision still holds. Teams should confirm with their own change management policy which changes count as material. Done when each re-run produces a new dated decision record that references the prior one, so the history of approvals can be audited.
When the NIST AI RMF says development should stop
The framework's clearest position on stopping comes from Section 1.2.3 of NIST AI 100-1, not from MANAGE 1.1 itself. It says that where an AI system presents unacceptable negative risk, such as imminent significant negative impacts, severe harms already occurring or catastrophic risks, development and deployment should cease in a safe manner until those risks can be sufficiently managed. The word "should" matters. NIST describes this as the expected response to an unacceptable risk, but it does not define a threshold, a sign-off role or a shutdown procedure. Your organization has to set those itself, in its risk tolerance and go or no-go criteria, before a system reaches a decision point.
In practice, a no-go outcome needs a written record that names the system, the intended purpose and stated objectives it failed to meet or the unacceptable risk that triggered the stop, the MAP and MEASURE evidence relied on, the accountable decision owner and the date. Confirm with your own governance team who holds the authority to halt a launch or pull a live system, and whether that authority sits with the product owner, a risk committee or an executive. The record should also state what must change before the question is reopened. Under MANAGE 1.1, the Playbook's suggested actions point to TEVV outputs and tradeoff analysis as the inputs, so the no-go file should show which results were weighed and why they were judged insufficient.
Operationally, a stop is only real if the system can actually be stopped. Check that deployment pipelines, model endpoints, feature flags and third-party integrations can be disabled or rolled back in a controlled way, and that downstream users and affected parties are told through the channel your incident process defines. Preserve logs, evaluation reports and model versions so the decision can be reviewed later. For generative or agent systems, confirm which tool permissions and data connections would need to be revoked. The framework does not prescribe these mechanics, so treat them as design requirements your team tests in advance rather than as steps the framework supplies.
Evidence that shows a go or no-go decision was made
A go or no-go decision under MANAGE 1.1 is only as credible as the record that shows who decided, on what evidence, and under which conditions. The cards below cover the artifacts an assessor or internal reviewer will ask for, and the habits that keep them current between formal audits.
Decision record
The decision record states the outcome for each AI system at a defined point: go, conditional go, or no-go. It names the release or change being decided, the date, the people who reached the decision, and the MAP and MEASURE inputs they relied on. Check that every deployment or significant change has its own record, and that a no-go is recorded with the same care as a go so that the rejected path is visible later.
Intended-purpose statement
The intended-purpose statement describes what the system is meant to do, for which users, in which setting, and what it must not be used for. MANAGE 1.1 asks whether the system achieves its intended purposes and stated objectives, so the decision can only be tested against this statement. Confirm that the version attached to a decision matches the version actually released, and that out-of-scope uses are listed explicitly.
TEVV summary referenced
The decision should point to a dated summary of test, evaluation, verification and validation results, with the measured outcomes for each trustworthiness characteristic that matters to the use case. The summary should identify the test set, the system build it covered, and any results that failed or were only partly met. Check that the reference points to the results that were current when the decision was made, not to a newer or older run.
Risk-benefit tradeoff memo
The tradeoff memo sets out the negative risks identified in MAP and MEASURE against the expected benefits, and explains why the balance supports the decision. The NIST AI RMF Playbook suggests weighing these against real-world use cases and impacts, so the memo should describe the affected people and the likely harms in plain terms. Ask whether the memo records the alternatives considered, including a non-AI option, and why they were rejected.
Approver sign-off
Sign-off shows that an accountable person with authority over the risk accepted the decision. The approver should be named in the record, hold a role documented under GOVERN 2, and be able to supersede the system if needed. Check that approvers are not the same people who built the system when the organization requires separation, and that sign-off is captured in a system that keeps its own timestamp and history.
Conditions attached to a conditional go
A conditional go lists the conditions that must be met before or after release, such as a limit on user groups, a human review step, or a monitoring threshold. Each condition needs an owner, a due date, and a check that closes it. An assessor will look for conditions that were never verified, or that quietly expired. Link every open condition to the ticket or change request that tracks it.
Review trail and change linkage
The review trail connects each decision to the release, configuration change, model update or data change it governs, and records when the decision was revisited. Keeping it current between audits means tying the decision identifier to the change management process, so a new model version cannot ship without a fresh determination. Confirm that the trail is kept in version control or an audited system and that its retention period is set by policy.
What a go or no-go process costs in money and effort
This table breaks the go or no-go process into cost components, showing who is paid for each, whether it is one-time or recurring, and what drives it up or down. NIST does not publish fees or durations for this work, so use the drivers and quote questions to compare offers on the same written scope.
| Cost component | Who is paid or who spends time | One-time or recurring | What drives it and how to keep it down |
|---|---|---|---|
| Internal review time | Product, risk, legal and engineering staff who read MAP and MEASURE outputs and sign the decision | Recurring with each decision point | Number of reviewers and review rounds. Agree decision criteria and a single decision owner before the first review. |
| Producing MAP and MEASURE inputs | Engineers and evaluators who run TEVV work and write it up for the decision | Largely one-time per release, then recurring | Scope of the system and number of trustworthiness characteristics tested. Reuse existing test suites and templates where the system has not changed. |
| Logging and evaluation tooling | Platform or security teams who run logging, tracing and evaluation software, or a vendor or partner who configures it | Setup is one-time; hosting and maintenance recur | Volume of traces, number of integrations and retention needs. Confirm licensing terms, support and hosting costs for each tool before selecting it. |
| Documentation of the decision | Compliance or risk staff who record the determination, evidence references and conditions | Recurring with each decision | Quality of templates. A standard decision record lowers time per review and makes later audits easier. |
| External review, if used | An independent assessor, consultant or implementation partner, paid under a written scope | Usually one-time per engagement, with optional follow-up | Depth of review and how many systems are covered. Ask for fixed-scope deliverables, named outputs and what is excluded before agreeing. |
| Re-decisions after change | Same reviewers as the initial decision, plus engineers who assess the change | Recurring, triggered by model, data, use or context changes | Frequency of changes and whether change criteria are defined. Written triggers reduce unplanned reviews. |
| Stopping or pausing a system | Engineering and operations staff who shut down or roll back, and business owners who absorb lost value | One-time per event | How quickly the system can be deactivated. Confirm rollback and deactivation paths during design, not after a no-go. |
Which open-source tools can support the evidence
garak is an open-source scanner that probes language models for weaknesses such as hallucination, data leakage, prompt injection, misinformation, toxicity generation and jailbreaks. For MANAGE 1.1, its output is useful because it gives the go or no-go review a documented record of what was tested, what was attempted and what succeeded before deployment. Its license is Apache-2.0, which your legal team should confirm against its intended use.
- Attempt report: garak writes a JSONL report covering every probe attempt and its evaluation, so reviewers can see the full test population, not only the failures.
- Hit log: A separate log lists attempts that found a vulnerability, which gives the decision owner a short list of open issues to weigh against intended use.
- Scope limits: Probes cover the risks garak implements. Gaps in coverage for your use case must be recorded as untested, not as passed.
- Repeatability: Keep the probe configuration and model version with each report so a later review can reproduce the run.
- Human review: Someone accountable must judge whether the hits are acceptable for the intended purpose.
Before relying on these reports, confirm the probe set, configuration and version you ran, and whether the results match the risks named in your MAP outputs.
Langfuse is an MIT-licensed observability platform that records traces of AI application requests. Each trace captures every operation in a request, including LLM calls, retrieval steps, tool executions and custom logic, along with timing, inputs, outputs, token usage and metadata. For MANAGE 1.1, these traces serve as evidence from TEVV and monitoring, showing how the system behaved in conditions close to real use before and after a decision.
- Call-level records: Each LLM call, retrieval step and tool execution is logged with inputs, outputs and timing, which lets reviewers trace a specific failure to a specific step.
- Usage data: Token usage and metadata help teams compare cost and behavior across model versions considered in the decision.
- Pre-release testing: Traces from staging or pilot use can show whether measured behavior matches the intended purpose.
- Monitoring input: Production traces feed the continuing review that the Manage function expects after deployment.
- Data handling: Traces can contain personal or sensitive data, so confirm retention, access and redaction settings before enabling them broadly.
Confirm that your deployment captures the operations relevant to your risks, and that the trace retention period matches the evidence you need to keep.
CISO Assistant is an AGPL-3.0-only platform for governance, risk and compliance work. It ships with more than 200 frameworks, including ISO 27001, NIST CSF, SOC 2, PCI DSS, NIS2, DORA and GDPR, and runs audits and compliance assessments against them with automatic mapping between frameworks. For MANAGE 1.1, it is most useful as the place where the go or no-go record, its evidence links and its approvals are tracked alongside the rest of the risk program.
- Decision record: Store the determination, its date, the approver and the evidence it relied on in one controlled register.
- Evidence links: Attach garak reports, Langfuse trace summaries and MAP outputs to the decision so reviewers can follow the chain.
- Assessment tracking: Run audits and assessments against the frameworks your organization is subject to.
- Framework mapping: Its automatic mappings are a starting point; verify any mapping against the source framework before relying on it.
- Licensing: AGPL-3.0-only has obligations if you modify and distribute the software or offer it over a network, so review them with counsel.
Check the framework content and mappings for the versions you use, and confirm who owns the register and who may approve a no-go or a conditional go.
Where go or no-go decisions usually fail
A go or no-go decision under MANAGE 1.1 is only as sound as the inputs, owners and follow-up behind it. The gaps below are the ones that most often leave a deployment running on a decision nobody can later reconstruct or defend.
Delivery team decides alone
When the team that built the system also approves its launch, the determination rests on the people with the strongest incentive to proceed. MANAGE 1.1 asks whether development or deployment should proceed, so the decision needs owners outside delivery, including risk, legal and the business owner of the use case. Check that the approval record names who signed, what they reviewed and who had authority to stop the launch.
No link to MEASURE results
A go decision that does not cite test results, evaluation metrics or monitoring baselines cannot show that the system achieves its objectives. The Playbook suggests using TEVV outputs from MAP and MEASURE when weighing risk treatment. Check that every go decision references the specific evaluation runs, thresholds and open findings it relied on, with dates.
Missing intended-purpose statement
Without a written intended purpose, nobody can say whether the system is achieving it. Vague statements such as improving efficiency leave no standard for a go or no-go call. Check that the statement names the users, the decisions the system supports, the populations affected and the uses explicitly excluded, and that the decision is tested against that text.
Conditions not tracked after launch
Teams often approve a system on conditions, such as a human review step, a limited user group or a required accuracy threshold, and then nobody monitors them. The approval loses its basis once a condition lapses. Check that each condition has a named owner, a monitoring signal and a review date, and that the production dashboards report on them.
No defined path to stop
Many programs approve launches without deciding in advance what evidence would trigger a pause, who could order it and how it would be carried out. AI 100-1 Section 1.2.3 says that where a system presents unacceptable negative risk, development and deployment should cease in a safe manner until the risks can be managed. Check that the runbook names the stop criteria, the authorized person, the rollback method and how users are informed.
Questions buyers ask about the MANAGE 1.1 go or no-go decision
Can the NIST AI RMF be certified?
No. The NIST AI RMF has no certification, accreditation or conformity assessment program, and NIST does not validate or endorse any organization's use of it. An organization can say it uses the framework and can show its MANAGE 1.1 decision record, but no body issues an AI RMF certificate. If a customer asks for proof, ask what evidence they expect: the documented go or no-go decision, the MAP and MEASURE outputs it relied on, and the approvers. A contract may ask for an independent assessment, which is a different thing from a certificate, and an independent auditor or assessor issues any report. DeployOpen prepares systems and evidence but does not issue reports or certificates.
Is MANAGE 1.1 mandatory under the NIST AI RMF?
The framework itself is voluntary. NIST's FAQ says neither private nor public sector organizations are required to use it. The obligation can still reach you indirectly. Customers may write the framework into procurement requirements, and some US state laws name the NIST AI RMF or its Generative AI Profile. Check each law and contract for the exact wording, because a reference may ask you to use the framework, consider it, or show alignment, and each of those carries different duties. Also confirm whether any sector regulator or your own board has adopted the framework as policy. Those sources, not the voluntary status of the framework, will decide what you must document.
How long does a MANAGE 1.1 go or no-go decision take to prepare?
No duration is fixed by the framework, and the time depends on the system's scope and how much MAP and MEASURE work already exists. Before the decision can be made, the team needs documented intended purposes and stated objectives, the TEVV results for the trustworthiness characteristics, an analysis of real-world use cases and impacts, and an agreed view of negative risks against benefits. Ask for a plan that lists each input, its owner and its current state. If those inputs are missing, most of the effort goes into producing them, not into the decision itself. A fixed-scope engagement should state its deliverables and timeline in writing before work starts.
Which people need to take part in the go or no-go decision?
The decision needs the people the Playbook ties to MANAGE 1.1: those responsible for AI deployment, operation and monitoring, and those who perform AI impact assessment. In practice this usually means the system owner, the technical lead who ran the TEVV work, a risk or compliance owner, and an executive with authority to stop or proceed. AI 100-1 Appendix A describes the tasks of each AI actor category, which helps you assign roles. Record who was consulted, who decided and what they relied on. Without a named decision maker, a go decision has no owner when a later incident raises questions.
Can we make the MANAGE 1.1 decision in-house?
Yes, if your organization has the people and the authority to do it. The framework does not require outside parties, and an internal team can document a decision that reflects its own context. The effort is usually in the evidence rather than the judgment: collecting TEVV outputs, reconciling them, and keeping a record that an assessor or customer can follow. Check that the people deciding are independent enough from the team that built the system, and that the decision can actually stop deployment. If you are unsure about either point, an outside review can help, though it does not replace the decision owner.
When should we bring in a consultant for the go or no-go decision?
Bring in outside help when the internal inputs are incomplete, when the decision will be scrutinized by a customer or regulator, or when the team lacks security testing or measurement skills. An implementation partner can prepare the systems, logs and evidence that MANAGE 1.1 depends on, and it can run security testing that feeds the MEASURE results. It should not make the decision for you or sign off on it. Ask any consultant to state in writing what they will deliver, what you own, and what an independent auditor would still need to check. DeployOpen plans and implements these inputs under a fixed scope agreed before work starts, and a quote follows from that written scope.
What evidence supports a go or no-go decision in practice?
The evidence should show what the system is for, how it performed against that purpose, and what risks remained when the decision was made. Useful records include the intended-purpose statement, the TEVV reports from MEASURE, a list of prioritized risks from MAP, and the reasoning that weighed negative risks against benefits. Self-hosted tools can generate parts of this evidence. garak writes a JSONL report of every probe attempt and evaluation, plus a hit log of attempts that found a vulnerability, which helps document language model testing. Langfuse captures traces with timing, inputs, outputs and token usage, which supports operational monitoring after a go decision. Confirm each tool's current capabilities in your own environment before relying on it.
What does a no-go decision require under the framework?
AI 100-1 Section 1.2.3 says that where a system presents unacceptable negative risk, such as imminent significant negative impacts, severe harms already occurring or catastrophic risks, development and deployment should cease in a safe manner until the risks can be sufficiently managed. A no-go decision therefore needs a stop plan, not just a refusal. Document what triggers a stop, who can invoke it, how users are protected during the pause, and what must change before the system can return. Confirm the stop path with your engineering and operations teams before you need it, since a decision that cannot be executed does not reduce the risk.
How DeployOpen prepares go or no-go evidence
DeployOpen works on MANAGE 1.1 by preparing the systems, test records and documentation that the organization's decision makers need to reach a go or no-go determination. The organization owns the decision and its controls, while DeployOpen implements the self-hosted open-source tooling that produces the evidence inside the client's own infrastructure. Independent assessors, not DeployOpen, issue any audit reports or certifications.
How we scope
- Each engagement starts with a written scope that names the AI systems, decision owners and deployment environments covered by the go or no-go review.
- We agree which MAP and MEASURE outputs feed the determination, such as TEVV results, identified risks and trustworthiness evaluations for each system.
- The written scope lists deliverables, timelines and responsibilities before work starts, and engagements are fixed-scope, so the client can confirm what is included.
How we build
- We deploy self-hosted tooling such as CISO Assistant to record risk treatment decisions, framework mappings and assessment results in one place.
- We configure garak to run hallucination, data leakage, prompt injection and jailbreak probes, with JSONL reports and hit logs kept as deployment evidence.
- We set up Langfuse tracing so retrieval steps, tool executions, token usage and outputs can be reviewed when deciding whether a system should proceed.
How we hand over
- We document the go or no-go evidence pack, mapping each test result and MAP or MEASURE output to the decision criteria the client's owners approved.
- We include the test scripts, configuration files and access records so the client's team and any independent assessor can reproduce the evaluations.
- We walk the client's decision owners through the evidence and record the open risks, conditions and limitations attached to each determination.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after the initial determination is documented.
- A retainer can rerun the probes and tracing checks when a model, prompt, dataset or deployment environment changes.
- Ask us to confirm in writing which evidence will be refreshed, how often, and who in your organization approves a renewed go or no-go decision.
Sources and further reading
- NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023 ↗
- NIST AIRC: AI RMF Playbook, Manage function ↗
- NIST AI RMF Playbook data (JSON download, NIST AIRC) ↗
- NIST AI 600-1, AI RMF: Generative Artificial Intelligence Profile, July 2024 ↗
- NIST: AI Risk Management Framework ↗

