How do you apply the NIST AI RMF when you buy AI instead of building it?
When a customer questionnaire or contract clause names the NIST AI RMF, the first thing to establish is that the framework is voluntary and has no certification. Your organization owns its controls, so any assessment is an internal review or an independent review you commission, never an AI RMF certificate. Nobody can issue a conformance badge for the framework, and any vendor or consultant who offers one should be questioned closely. Your job is to show how you govern, map and monitor the AI you use, and to produce evidence a reviewer can test. For a buyer, the work shifts toward GOVERN 6 (third-party software, data and supply chain), MAP 4 (risks of each component, including third-party models and data) and MANAGE 3 (monitoring third-party AI and its controls). Most Measure outcomes assume your team can run testing, evaluation and validation (TEVV) on the model itself, and you usually cannot. Practically, that means writing procurement due diligence that asks the developer for documentation on training data, testing, limitations and usage instructions, and recording what you could not verify. The NIST AI 600-1 profile lists requests such as software bills of materials, service level agreements and attestation reports as ways to apply existing controls to third-party generative AI. An implementation partner can help you define these procurement requirements, assemble the evidence files and test your own monitoring and incident handling. The partner prepares systems and evidence but does not issue reports. Only an independent auditor or assessor can issue a report or opinion, so confirm who will perform that review, its scope and its timing before you sign. Ask the partner to show which evidence supports each claim, and ask your auditor which of those items it will accept.
Which Measure outcomes assume model development, and how buyers adapt them
Many Measure subcategories assume the organization trained the model and can inspect its data, code and test history. Buyers can still meet the intent of these outcomes by obtaining what the vendor will share, testing the rest in their own environment, and writing down what they asked for and what they received.
Obtain TEVV documentation from the vendor
Measure 2 expects test, evaluation, verification and validation records for each trustworthiness characteristic. A buyer usually cannot see the developer's internal test suites, so the practical step is to request the vendor's TEVV summary, known limitations, test populations and version history in writing. Check whether the summary covers the model version you will run, not a later release. Record any gaps as residual risk for your own decision.
Test deployment-like conditions yourself
Measure 2 also asks for testing under conditions that resemble real use. Vendor benchmarks rarely reflect your users, data quality or workflows, so the buyer should run its own evaluation with representative inputs before go-live and after each material update. Define pass criteria in advance, including performance for affected groups. Keep the test set and results as evidence, since a vendor cannot produce them for you.
Select metrics for your mapped risks
Measure 1 asks the organization to choose metrics for its most significant risks and to review whether those metrics still work. Developers often report metrics suited to their design goals, and those may not match the harms you mapped. Ask the vendor which metrics it tracks and why, then add the metrics that reflect your context. Document who approved each metric and when it will be reviewed.
Monitor production behavior after deployment
Measure 3 covers tracking risks after deployment, including emergent risks and end user problem reporting. The buyer owns this monitoring because the vendor sees only aggregated or contractual signals. Set thresholds, log outputs where your policy allows, and define how users report errors and appeal decisions. Confirm in the contract what telemetry the vendor will provide, how long it is kept, and who may access it.
Document scientific integrity from the vendor
Map 2 asks the organization to understand the scientific basis of a system, its knowledge limits and the TEVV considerations behind it. For procured systems, request the vendor's description of training approach, data sources, known failure modes and evaluation design. Compare these answers with your intended use and state clearly where the system is outside its validated scope. Treat unanswered questions as a procurement risk, not a reason to skip the Map step.
How to run procurement due diligence against the AI RMF
This sequence turns a vendor proposal into a documented go or no-go decision, with each step producing a record that shows how the buying organization assessed the AI system, its third-party risks and the residual risks it inherits. It follows the AI RMF's Govern, Map, Measure and Manage logic without assuming the buyer can test the model itself.
Confirm governance policy covers third-party AI
Before any proposal arrives, the AI governance owner confirms that the organization's policies cover procured AI, not only systems it builds. Under GOVERN 6.1, the policy should set expectations for transparency into training data, algorithms and limitations, thorough testing of third-party AI, clear usage instructions, and the legal and ethical issues of buying third-party software, hardware and data. Done means a written, approved policy exists and procurement staff can point to it. Ask counsel and the risk function to confirm the policy matches the organization's risk tolerance.
Intake and initial risk analysis of each proposal
Procurement routes every AI-related proposal through the required review and records the intended use, the business owner, the data the system will touch and the people it will affect. The reviewer makes an initial risk analysis and assigns a provisional risk tier. The output is an intake record that names the accountable owner and the risk tier. San Jose's model of a required procurement review for all technology purchases, with an initial risk analysis of each proposal, is a useful pattern to adapt.
Complete impact assessment for mid and high-risk proposals
For proposals in the mid or high tier, the business owner and risk function complete an impact assessment that maps the system's context, its likely impacts on people and the likelihood and magnitude of each. This is where MAP 4 and MAP 5 questions about third-party components, data and intellectual property risks get answered for the vendor's stack. The output is a signed assessment listing unresolved questions that must be answered before any decision.
Request vendor transparency and supporting artifacts
The buyer sends a written request for information covering training data sources, model limitations, testing performed by the developer, usage instructions and, for generative AI, the controls listed in the Generative AI Profile. The Profile's suggested action GV-6.1-004 points toward contracts and service level agreements that address content ownership, usage rights, quality standards, security requirements and provenance. Requests may also include software bills of materials and SSAE attestation reports, where the vendor holds them. The output is a vendor response file. Where the vendor declines to disclose, the gap goes into the risk record, because AI 100-1 notes that developers may not be transparent about their risk metrics.
Test the third-party system in the deployment context
The deploying team runs its own testing on the system as it will be used, since the vendor's results may not match the organization's data, users or tolerances. This step draws on MEASURE 2 and checks the vendor's claims against the buyer's use case, including limitations the vendor disclosed. The output is a test report with results, the test conditions and any failures. If the buyer lacks testing capacity, an independent tester can run this work, but the buyer keeps ownership of the findings.
Document negative residual risks for the decision
The reviewer consolidates unresolved risks from the assessment, the vendor response and testing into a residual risk register. MANAGE 1.4 calls for documenting negative residual risks, defined as the sum of all unmitigated risks, so the register must be complete enough to share with downstream acquirers and end users. The output is a residual risk statement that names each open item, its owner and the compensating controls, if any.
Make the go or no-go determination
The accountable executive, using the assessment, test results and residual risk statement, decides whether to proceed under MANAGE 1. The decision record states the conditions attached to a go, such as usage limits, human oversight requirements or contract terms, and the criteria that would trigger a no-go or a pause. The output is a signed determination with a named approver. Ask the executive to confirm the decision falls within the organization's documented risk tolerance.
Hand over conditions to contract and monitoring
Once a go decision is recorded, procurement writes the conditions into the contract and the monitoring plan. Contract terms should reflect the transparency, testing and usage commitments the vendor made during review. The deploying team then schedules monitoring of the third-party system under MANAGE 3. The output is an executed contract with the conditions attached and a monitoring plan tied to the residual risk register. The file is complete when the owner can trace each open risk to a contract term, a control or a review date.
Which contract terms and SBOM requests the Generative AI Profile supports
The Generative AI Profile's action GV-6.1-004 suggests that organizations draft and maintain contracts and service level agreements for generative AI systems. The action names five areas: content ownership, usage rights, quality standards, security requirements and content provenance expectations. For a buyer, these map to the questions a contract should answer before signing, such as who owns outputs, what the buyer may do with them, how quality is measured, what security the vendor must maintain, and how the origin of generated content is recorded.
- Content ownership: State who owns prompts, outputs, fine-tuning artifacts and logs, and whether the vendor may reuse buyer data.
- Usage rights: Define permitted uses, restricted uses and any limits on downstream sharing.
- Quality standards: Specify measurable performance expectations and how the vendor reports them.
- Security requirements: Name the controls the vendor must maintain and the notice period for changes.
- Content provenance: Ask how generated content is labeled or traced, and what records the vendor keeps.
Confirm with counsel how each term is enforceable under your jurisdiction and governing law, and whether any state law you are subject to imposes its own disclosure requirements.
AI 600-1 Appendix A.1.3 lists requests for software bills of materials (SBOMs) as one of the existing risk controls that also apply to proprietary, open-source and third-party generative AI. An SBOM is an inventory of the software components in a product. For AI systems, buyers often also want to know about model dependencies, libraries and data sources, but the profile does not define the content an SBOM must contain, so the buyer should specify it in the request.
- Component list: Ask for the names, versions and suppliers of software components, including open-source packages and base models.
- Model and data lineage: Request the source of pre-trained models and whether training data sources are disclosed.
- Update cadence: Ask how often the SBOM is refreshed and how changes are communicated.
- Vulnerability linkage: Ask how the vendor maps known vulnerabilities to listed components.
- Disclosure limits: Ask which items the vendor withholds and why.
SBOM content depends on what the vendor can disclose. Proprietary models may only be described at a high level, so record gaps as residual risk. Confirm with counsel what the contract allows you to request and share.
AI 600-1 Appendix A.1.3 also names SSAE attestation reports as a control a buyer can request. An SSAE report is issued by an independent certified public accountant (CPA) firm after examining a service organization's controls. The report covers the vendor's controls, not the buyer's. It tells you what the vendor's auditor tested and what it found, and it does not show that your own use of the service is controlled.
- Independent issuer: The CPA firm issues the report. DeployOpen and the buyer do not issue or sign it.
- Scope: Check which services, locations and time period the report covers.
- Complement controls: Reports usually list controls the customer must maintain. Map those to your own program.
- Exceptions: Read the auditor's noted exceptions and the vendor's responses.
- Currency: Confirm the report period is recent enough for your review cycle.
Ask the vendor which report type it holds and whether it covers the AI service you buy, not only the wider platform. Confirm with counsel what reliance the contract places on the report.
The Playbook's suggestions for GOVERN 6.1 include transparency into training data, algorithms and limitations, thorough testing of third-party AI and clear usage instructions. AI 100-1 notes that a developer's risk metrics may not match those of the deploying organization and that the developer may not be transparent about them. Procurement due diligence is how a buyer obtains enough information to decide whether the system is acceptable for its use.
- Transparency requests: Ask for documentation of training data sources, known limitations and intended uses.
- Testing evidence: Request the vendor's testing results and whether you may run your own tests.
- Usage instructions: Obtain the operating guidance the buyer needs to use the system safely.
- Legal and ethical review: Document the legal and ethical issues of acquiring the system.
- Residual risk notice: Under MANAGE 1.4, document negative residual risks and pass them to downstream acquirers and end users.
Where the vendor refuses a request, record the refusal and the risk it creates. Confirm with counsel which due diligence questions must be answered in writing.
Passing residual risk down the chain
MANAGE 1.4 in NIST AI 100-1 calls for documenting negative residual risks, which the framework defines as the sum of all unmitigated risks, and sharing that record with downstream acquirers of the AI system and with end users. For a buyer, this is the point where procurement stops being a vendor selection exercise and becomes a risk record the organization owns. The vendor will have mitigated some risks in the model, data and product. What remains unmitigated still lands on the deployer, so the deployer needs its own list of what the vendor has not addressed, what the deployer has added through configuration or use, and what is left over after both.
In practice, the buyer should build this register from the vendor's documentation, contract terms and its own testing. Useful inputs include the vendor's stated limitations, known failure modes, data handling terms, and any SBOM or attestation the vendor provides under the contract. Each residual risk should name the affected use case, the likelihood and magnitude estimate the organization used, who accepted it and when it will be reviewed. Confirm with the vendor, in writing, which limitations it has documented and whether it will update them when the model or product changes. Vendors often change underlying models without a version notice, so the register needs a review trigger tied to vendor updates.
The register feeds MANAGE 4 communication. When monitoring finds an incident or a materially changed risk, the buyer has to decide who learns about it and how, including affected users and, where the use case touches individuals, the people the system affects. Illustratively, a hiring tool bought from a vendor might carry a documented residual risk of uneven error rates across groups. If monitoring shows that risk is growing, the buyer's communication plan should say how candidates are told, who can request a human review, and which contract clause obliges the vendor to supply the data needed for that notice. Map these obligations before signing, because a notice clock that starts in the contract can be shorter than the internal review cycle.
How the City of San Jose handled procured systems
The City of San Jose, California, is the clearest published example of an AI RMF program built around procurement. NIST's AIRC use case describes governance the city already had before adopting the framework: a required procurement review for all technology purchases, an initial risk analysis of each proposal, and impact assessments for proposals rated mid or high risk. The city also keeps a public algorithm register and monitors its high-risk systems on an ongoing basis. Buyers should read this as NIST's description of one city's practice, not as an endorsement, and should confirm the current details directly in the NIST AIRC use case for the City of San Jose.
The use case also reports a finding that matters for any buyer. San Jose's Measure score was low, and the city attributed this partly to subcategories written around the technical development of AI models. Most of the AI systems the city runs are procured rather than built in-house, so the measurement subcategories assumed access to model internals that the city does not have. The lesson is not that Measure does not apply to buyers. It is that buyers must translate Measure into what they can actually test: vendor documentation, acceptance testing in their own environment, and monitoring of outputs in production. Ask which Measure subcategories a procured system can meet with evidence the vendor supplies, and record where that evidence stops.
Other buyers can copy the structure more easily than the scores. Add an AI question to the existing procurement gate rather than creating a separate track, set impact assessment thresholds that trigger on the system's use, and keep a register that the public or auditors can read. Then decide which vendor disclosures you will require before signing, such as the training data and limitations information described in the Playbook's GOVERN 6.1 actions. Document each gap you accept, because the city's experience shows that unmeasured risk is easiest to overlook when the procurement file says nothing about it.
Who owns what when AI is procured
The buying organization owns its risk decisions. NIST's Procurement role in AI 100-1 covers AI actors with financial, legal or policy authority to acquire AI models, products or services from a third-party developer, vendor or contractor. That authority carries the duty to decide whether the system fits the organization's risk tolerance, intended use and legal obligations, even when the organization never sees the model.
- Go or no-go decisions: Approve or reject the system against documented risk tolerance and intended purpose.
- Vendor due diligence: Request transparency about training data, testing, limitations and usage instructions.
- Contract terms: Set security requirements, service levels, content provenance and exit rights.
- Residual risk records: Document the risks the vendor and buyer have not mitigated, for downstream users.
- Ongoing monitoring: Track the system's behavior after deployment and retain authority to suspend it.
Ask whether the procurement review covers AI systems, not only software, and who signs the go or no-go decision.
The vendor builds, trains, tests and documents the AI system it sells. AI 100-1 notes that the developer's risk metrics may not align with those of the deploying organization, and that the developer may not be transparent about them. The vendor therefore controls the evidence the buyer can see, which makes disclosure terms a contract question rather than a courtesy.
- Model documentation: Provide descriptions of training data sources, evaluation methods and known limitations.
- Testing evidence: Share results from pre-release testing and any independent evaluations it has commissioned.
- Security artifacts: Supply software bills of materials and vulnerability handling commitments where the contract requires them.
- Assurance reports: Provide SSAE attestation reports or equivalent, if it has them, covering the service.
- Change notices: Announce model updates that alter behavior, so the buyer can re-assess.
Ask the vendor which of these it provides as standard and which require a negotiated addendum, and what it will not disclose and why.
An implementation partner prepares the buyer's systems, integrations and evidence. It does not own the risk decision and does not issue assessment results. DeployOpen plans, implements, integrates, hardens and documents self-hosted open-source platforms inside the client's own infrastructure, and it prepares systems and evidence for compliance assessments. It does not issue audit reports, attestations or certifications.
- Inventory and mapping support: Record each AI system, its components and its third-party dependencies in a form the buyer can maintain.
- Technical controls: Configure logging, access control, monitoring and isolation for systems the buyer runs.
- Testing separated from delivery: Run security and AI testing that is documented apart from the team that built the integration.
- Evidence packages: Organize records for the buyer's governance, legal and assessment teams.
- Documentation handover: Deliver runbooks and records the buyer's staff can operate after the engagement.
Ask for the written scope, the named deliverables, and a statement of which decisions remain with the buyer.
An independent auditor, assessor or certification body tests the program and issues any report or certificate. NIST's AI RMF use cases page states that NIST does not validate or endorse any individual organization or its approach, and NIST describes no certification, accreditation or conformity assessment program for the framework. Any assessor must therefore be chosen on its own qualifications and independence, and its report describes what it tested, not what NIST certified.
- Independence: The assessor should not design or operate the controls it tests.
- Scope agreement: Define which AI systems, vendors and Measure outcomes are in the test.
- Procedures: Ask how the assessor samples evidence and tests vendor claims that it cannot see directly.
- Reporting: Confirm the report format, who receives it, and whether it covers vendor-supplied systems.
- Qualifications: Check the assessor's experience with AI systems and third-party risk, since NIST publishes no approved assessor list.
Confirm the assessor's independence from both the buyer's implementation work and the vendor before engaging it.
Questions buyers ask about the AI RMF and procured AI
Is the NIST AI RMF required for AI vendors or the organizations that buy from them?
No. NIST states that neither private nor public sector organizations are required to use the AI RMF, which it designed as a voluntary framework. A customer may still ask a vendor to show alignment with it, and a regulator or state law may name it, so the obligation usually comes from a contract or a specific law rather than from the framework itself. Check the exact wording of any law or contract that cites the AI RMF, including which version it names, because NIST has said AI RMF 1.0 is being revised and has not yet published the revised version.
Can a vendor be certified against the NIST AI RMF?
No. There is no AI RMF certification, accreditation or conformity assessment program. NIST does not validate or endorse any organization's use of the framework. A vendor can describe how its program maps to the Core functions and subcategories, and an independent assessor can test those claims, but no body issues an AI RMF certificate. If a customer asks for a certificate, the closest option is ISO/IEC 42001, which is a certifiable management system standard, so ask the vendor which standard its claims rest on and what evidence supports them.
What should we do when a vendor refuses to provide an SBOM?
Treat the refusal as a risk finding and document it. The Generative AI Profile says organizations can apply existing controls to third-party providers, including procurement due diligence, requests for software bills of materials (SBOMs), service level agreements and SSAE attestation reports. Ask in writing for the components, versions and licenses of the software and models in the delivered product, and for a statement of what the vendor will not disclose. If the vendor still declines, the contract can require notice of changes, the risk can be recorded for the go or no-go decision, and the system can be limited to lower-impact uses. Ask counsel whether existing contract terms already cover this request.
How much internal effort does procurement due diligence take?
The effort is mostly internal, and it depends on how many AI purchases you review, how complex each product is and how many teams must sign off. The main internal roles are the procurement lead, a security reviewer, a privacy or legal reviewer, the business owner of the use case and someone who can read a model card or technical documentation. Your costs are staff time for each review, plus recurring time for monitoring after purchase. Outside help, such as an implementation partner, can prepare the review templates, evidence requests and records, but the accept or reject decision stays with your organization. Ask any partner for a written scope before work starts.
Can we do Measure work for a model we cannot access?
Yes, in part. Measure outcomes assume testing, but a deployer without weights or training data can still test the system as deployed, through its interface or API, against the risks it mapped. Build a test set from your own use cases, record inputs and outputs, check results for accuracy, safety, bias and security, and monitor performance in production. Ask the vendor for its own test results, known limitations and change notices. Where the vendor refuses access to documentation, record the gap as a residual risk, since MANAGE 1.4 calls for documenting negative residual risks to downstream acquirers and end users.
When does the AI RMF Playbook help a buyer?
The Playbook helps when you need concrete questions for each subcategory. It offers suggested actions, documentation prompts and references, and for GOVERN 6.1 it suggests policies for third-party AI covering transparency into training data, algorithms and limitations, testing of third-party AI, usage instructions, and the legal and ethical issues of procuring software, hardware and data. NIST says the Playbook is neither a checklist nor a set of steps, so use it to draft your procurement questionnaire and evidence list, not as a pass or fail test. Its last content update was in August 2023, so check newer guidance such as the Generative AI Profile for generative systems.
How DeployOpen prepares procured AI systems for review
DeployOpen plans, implements, integrates, documents and supports self-hosted open-source platforms inside the client's own infrastructure, and runs security testing that includes AI systems and agents. For organizations that buy AI rather than build it, that work prepares the systems, controls and evidence a deployer needs for a NIST AI RMF review. DeployOpen does not issue assessments or certifications, since the framework has none.
How we scope
- Agree a written scope listing which procured AI systems, deployment environments and AI RMF categories are covered before work starts
- Inventory each procured system with its vendor, intended purpose and the data it touches, for confirmation by your team
- Identify which Govern 6 and Manage 3 outcomes you already meet and which the vendor must supply
How we build
- Set up self-hosted open-source platforms in your own cloud accounts, data centers or air-gapped networks, with logging that records how procured AI is used
- Prepare a third-party AI request set covering training data transparency, known limitations, testing results and usage instructions
- Configure monitoring of high-risk procured systems and document the testing performed on each one
How we hand over
- Deliver the documentation your procurement, legal and risk teams need, including Playbook-style records for each in-scope subcategory
- Record residual risks from vendor systems in a form you can share with downstream users, as MANAGE 1.4 describes
- Provide a handover session so your staff can maintain the records and answer questions from customers or auditors
How we support
- Offer ongoing support, upgrades and continuous testing of the platforms and evidence under a retainer
- Review vendor contract and service level agreement terms against your written requirements when you ask for a second reading
- Confirm with your independent assessor or counsel which evidence they need, since DeployOpen prepares material but does not issue reports
