How the NIST AI RMF applies to a RAG knowledge assistant
When a team deploys an internal document assistant and asks what NIST expects, the answer has two parts. The AI RMF is voluntary and has no certification, so no body can certify a deployment against it. The Generative AI Profile, NIST AI 600-1, is the lens that matters most here, because it names the generative AI risks that a retrieval assistant faces, including *confabulation* and *information integrity*. The AI RMF Core has four functions: Govern, Map, Measure and Manage. Govern is cross-cutting and should have its outcomes in place first. NIST says most users then start with Map and continue to Measure or Manage, and it describes the actions as not being a checklist. Teams can choose which categories and subcategories fit their assistant.
This page focuses on two outcomes. MAP 2.2 asks that the system's knowledge limits and the way humans use and oversee its output be documented in enough detail to support decisions by the relevant AI actors. For a RAG assistant, that means writing down what the corpus does and does not cover, which document sets are excluded, which upstream sources and connectors feed the index, and how staff are expected to check a cited answer before acting on it. The AI 600-1 suggested actions for this outcome list End Users as the responsible actor, so the people who ask questions belong in the documentation, not only the platform team.
MEASURE 2.5 asks that the system be shown to be valid and reliable, and that the limits of its generalizability beyond development conditions be documented. In practice, the team should test retrieval and answer quality on a question set drawn from the real corpus, record the operating conditions and limits, and document what happens when a question falls outside the indexed documents. Confirm with your implementation partner how they will evaluate answers and retrieval before release, and ask which evaluation tooling, such as self-hosted Langfuse or Phoenix, will record results. Treat the evaluation record as evidence for MAP and MEASURE, not as a one-time test.
Which AI RMF outcomes a RAG assistant must meet and how to configure them
This table maps the AI RMF outcomes most relevant to a RAG assistant to concrete configuration and documentation actions, with the evidence an internal reviewer would expect. Read each row as an outcome, the action that meets it on a self-hosted stack, and where teams usually fall short. The main takeaway is that confabulation and information integrity are controlled through retrieval design, citation display, refusal behavior and a maintained test set, not through the model alone.
| Outcome | What to configure | Evidence to keep |
|---|---|---|
| MAP 2.2: knowledge limits and human oversight (AI actor: End Users) | Document what the assistant covers, which collections it searches and what it must decline. Show users how to verify answers against the cited source before acting. | Scope statement, collection list, user guidance, sample of answers where the assistant declined |
| MAP 2.2: settings outside intended use | Set out-of-scope rules in the system prompt and collection access, such as no legal advice, no personal case data and no answers from unapproved folders. | Intended-use statement, out-of-scope test cases and their results |
| MAP 2.2: dependencies on upstream data and models | Record each source connector, the embedding model, the chat model and the vector store, and when each is refreshed or replaced. | Data flow diagram, dependency register, change log for models and indexes |
| MEASURE 2.5: validity and reliability | Build a labeled question set from real internal questions, score retrieval (right document found) and answer faithfulness (answer supported by retrieved text) separately. | Test set version, scores by release, reviewer sign-off on sampled answers |
| MEASURE 2.5: generalizability limits | Test on questions outside the indexed corpus and on documents added after indexing. Document the conditions under which quality was measured, such as collection, language and document type. | Documented limits, out-of-corpus test results, list of document types not validated |
| Confabulation and information integrity | Show the cited passage beside each answer, require a citation for factual claims, and configure refusal when retrieval scores fall below a threshold you set and test. | Refusal rate and threshold settings, citation coverage checks, periodic answer audits |
| Data privacy and information security (GAI risks) | Enforce source permissions at retrieval time, keep the vector store and logs on your infrastructure, and restrict admin access to the assistant and its keys. | Permission test results, access reviews, retention settings for prompts and logs |
| Value chain and component integration (GAI risk) | Pin component versions, review license terms of the platform, and test upgrades against the evaluation set before rollout. Confirm the Open WebUI branding terms or Dify multi-tenant limits before deployment. | Version register, license review record, regression results per upgrade |
| GOVERN cross-cutting outcomes | Assign an owner for answer quality, a process for reporting wrong answers, and a review cycle for the test set. NIST describes Govern as cross-cutting, so apply it before Map and Measure work. | Named roles, incident and feedback records, review minutes |
Self-hosted platforms that fit an NIST-aligned RAG assistant
Four open-source platforms cover most RAG assistant designs, and each one carries a different license, identity model and operating burden. The choice shapes which NIST AI RMF evidence you can produce without custom work. Confirm each platform against your own requirements before you commit, including who the users are and whether the assistant is internal or customer-facing.
Dify for workflow-heavy assistants
Dify combines a visual RAG pipeline, prompt tooling and LLMOps monitoring. The Docker Compose deployment starts the API, worker, web, plugin daemon, PostgreSQL, Redis, Weaviate, Nginx, a code sandbox and an SSRF proxy. The SSRF proxy blocks outbound requests to internal and private IP ranges, and the environment reference covers workspace encryption keys, which can be stored locally or in Azure Key Vault. Its license forbids operating a multi-tenant service from the source code without written authorization, so a customer-facing assistant needs a license review first. Confirm which edition covers SSO and multi-workspace needs.
Open WebUI for staff access controls
Open WebUI provides role-based access control with groups and per-resource permissions, SSO through OIDC and LDAP, SCIM 2.0 provisioning and authenticator-app MFA. Its retrieval features support hybrid BM25 and vector search with reranking across 13 vector databases. Multi-replica deployments need PostgreSQL, Redis and shared file storage. Its license clause protects Open WebUI branding and applies to deployments above 50 users in any rolling 30-day period unless you hold an enterprise license or written permission. Check your user count against that limit before rollout.
RAGFlow for document parsing
RAGFlow is an Apache-2.0 retrieval-augmented generation engine focused on document understanding and grounded answers with citations. Parsing quality matters for MAP 2.2 and MEASURE 2.5 because scanned, tabular and layered documents determine what the model can know and where it can be wrong. Test parsing on a sample of your real corpus, check that citations point to the exact passage, and confirm how it handles permissions from your source systems.
AnythingLLM for small, contained deployments
AnythingLLM is MIT licensed and packages document ingestion, workspaces and chat in one application, which suits a contained pilot or a single department. Its simplicity limits how far you can extend access control, audit trails and evaluation without additional engineering. Confirm workspace-level permissions, SSO support and log retention against your NIST-aligned requirements, and plan how logs will leave the application for review.
LiteLLM and Langfuse for logging and evaluation
Platform choice does not supply the evidence on its own. LiteLLM sits between the assistant and model providers and records cost, guardrail and request data, while self-hosted Langfuse traces prompts, retrieved passages, outputs and evaluation scores. Together they support MEASURE 2.5 validity checks and the event records that the EU AI Act requires for high-risk systems. Confirm which events each tool captures, where they are stored and who can read them.
How MAP and MEASURE work in practice for a document assistant
MAP 2.2 asks the team to write down what the assistant knows, what it does not know and who relies on its output. For a RAG assistant, that record covers the corpus boundaries (which repositories, date ranges and document types are indexed and which are excluded), the upstream dependencies (the document stores, connectors, embedding model and any external model API the pipeline calls), and the external network paths the system can reach. The NIST AI RMF Playbook suggests documenting settings outside the intended use and testing the human-AI configuration under close to real-world conditions, so the record should name the staff who check answers, the citation format they see and the cases where they must go back to the source file. The AI 600-1 task for this outcome lists End Users, so the owners of the assistant should confirm that the user guidance states plainly that answers are generated from retrieved text and can be wrong.
MEASURE 2.5 turns those limits into a test plan with stated operating conditions. Define the conditions first: which collections are live, which languages and file types are supported, and how old an indexed document may be before it is flagged. Then build a labeled question set drawn from real staff questions, with reference answers and the source passages that should be cited, and score retrieval (did the right passage appear) separately from generation (did the answer follow from it). Where the outputs are scored by a judge model or by reviewers, use confidence intervals or repeated runs to show whether a change in score is meaningful, and use cross-validation style splits so the question set is not tuned on the same questions used to report results. Document where the scores hold and where they do not, including questions from document types or departments the test set did not cover. That written boundary is the generalizability limit MEASURE 2.5 asks for.
Treat the Playbook as a menu of suggested actions, not a checklist. NIST says the Playbook is neither a checklist nor a set of steps, and AI 100-1 lets organizations choose which subcategories to apply. Select the MAP and MEASURE actions that match the assistant's risk, record why any action was skipped, and confirm with your own compliance or legal function which documentation they need to retain. The Playbook's last content update was August 2023 and NIST has said it will be revised after AI RMF 1.0 is revised, so confirm current guidance in the NIST AI RMF Playbook before you cite it in an assessment.
What auditors and assessors ask to see for a RAG assistant
This table maps the evidence artifacts an assessor typically asks for against the component that produces each one in a self-hosted RAG assistant. Read it as a checklist of what to design for before go-live, not as a list of certifications, since NIST AI RMF has no certification and only independent auditors or certification bodies issue attestations. Confirm with your own assessor which artifacts they require and which retention rules apply to your system.
| Evidence artifact | What it shows | Produced by | Confirm before go-live |
|---|---|---|---|
| System categorization and scope record | Intended use, users, data sources and connections to external networks, tied to MAP 2 outcomes | Design documentation maintained by the team, informed by the MAP 2.2 Playbook prompts | Whether any upstream data or external systems must be listed as dependencies |
| Documented knowledge limits and intended use | What the assistant should and should not answer, and where its corpus ends | Assistant configuration and system prompt versions in Dify or Open WebUI, plus the approved use statement | Which topics are excluded and how users are told about them |
| Human oversight procedure | How staff review, escalate or override answers, supporting MAP 2.2 on human use and oversight | Written procedure, with review queues or feedback records kept in the chosen platform | Who reviews sampled answers and how often the review is recorded |
| Evaluation datasets and results | Retrieval and answer quality against test questions, supporting MEASURE 2.5 on validity and reliability | Test sets stored under version control, scored with Langfuse evaluations or a similar tool | Coverage of out-of-scope and adversarial questions, and the acceptance threshold agreed with the business owner |
| Corpus and version change logs | Which documents were indexed, when, and which model, prompt and embedding versions answered | Vector store records, ingestion job logs and platform release history such as Dify or Open WebUI version tags | Whether deleted or superseded source documents are removed from the index and embeddings |
| Retained interaction logs | Prompts, retrieved passages, citations and answers for later review | Platform logs in Dify or Open WebUI, Langfuse traces and LiteLLM request logs | Retention period, access controls on logs and whether personal data in prompts is masked |
| Retention basis for logs | How long records are kept and why, tied to legal duties where they apply | Log storage policy applied to the components above | EU AI Act Articles 12 and 19 apply only to high-risk systems; confirm applicability with counsel before setting a period |
What tooling alone does not satisfy
A RAG assistant platform can index documents, answer questions and keep logs, but the NIST AI RMF expects evidence and decisions that software cannot produce on its own. The cards below set out what the organization must own, document and keep current after the platform is installed.
Validity evidence stays with the team
No platform produces proof that answers are valid and reliable for your documents. MEASURE 2.5 expects you to show validity, document the limits of generalizability and define operating conditions. Build a question set from real staff queries, score answers and citations, and record the results before go-live. Ask any partner which evaluation datasets they will hand over with the system.
Governance decisions need named owners
Choosing which document collections the assistant may answer from, which users may query them and when an answer must be escalated are decisions for the organization. The GOVERN function sets the accountability structure that Map and Measure depend on. Record who approved each decision, the date, and the review trigger, because a platform will not keep that record for you.
Risk acceptance is an organizational act
Each of the 12 generative AI risks in AI 600-1, including confabulation and information integrity, needs a documented decision to mitigate, accept, transfer or avoid it. A vendor can supply controls, but it cannot accept residual risk on your behalf. Assign a risk owner for each accepted risk and set a review date that matches your change process.
Staff training on knowledge limits
MAP 2.2 expects human users to understand the assistant's knowledge limits and how to oversee its output. Training must cover when answers come from outdated or incomplete sources, how to check citations, and when to stop relying on the assistant. Confirm with any partner what training material they will draft and whether it will be updated when collections change.
Playbook content may change
The Playbook's audit log shows its last content update in August 2023, and NIST says it will be updated after AI RMF 1.0 is revised. Treat suggested actions as a starting reference, not a fixed set, and check the Playbook version before each assessment. Record which version your mapping used so auditors can trace your reasoning.
Profile actions are chosen, not inherited
AI 600-1 lists suggested actions tied to AI RMF subcategories, but the framework says users choose which categories and subcategories apply. Select the GAI actions that fit your use case, mark the ones you exclude with a reason, and document the result. NIST expects a formal AI RMF review no later than 2028, so plan a periodic review of this selection.
Questions buyers ask before scoping NIST alignment for a RAG assistant
Is there a NIST certification for a RAG assistant?
No. There is no NIST AI RMF certification, accreditation or conformity assessment program. NIST states that it does not validate or endorse any individual organization or its approach to using the AI RMF. Alignment is a documented claim about how your assistant's risks were mapped, measured and managed, and any external assurance comes from an independent auditor or certification body that is separate from NIST and from DeployOpen. The AI RMF is also voluntary, so no regulator requires its use by default. Ask any partner who offers a "NIST certified" badge to show the program that issues it, and write down that the deliverable is documentation and evidence, not a certificate.
How long does a NIST scoping effort for a RAG assistant take?
The duration depends on the factors that set the size of the work: the number of document sources, the corpus size, how many user groups have different access rights, the sector and the number of risks you choose to assess from the Generative AI Profile. No duration is set in the sources for this work, so request a timeline in the written scope rather than relying on a general estimate. A good scope lists the phases, the sources in each phase, the decision points where your owners sign off, and what happens if a source turns out to be larger or messier than expected. Regulated sectors usually add review cycles for legal, risk and model governance teams.
What drives the cost of scoping NIST alignment for a RAG assistant?
Cost follows the same drivers as the work. The main ones are the number of document repositories and their access models, how many user groups and workflows the assistant serves, how many generative AI risks from the 12 defined in NIST AI 600-1 you assess in depth, the sector's documentation and review expectations, and how much evaluation the team must build (test sets, scoring, and repeat runs after each change). Ask for a quote that separates one-time work (mapping, risk register, initial test set, evidence templates) from recurring work (re-running evaluations after model or corpus changes, updating records as the framework is revised, and periodic reviews). DeployOpen does not publish a price list, and the quote follows from the written scope.
What do we need to provide for a NIST alignment engagement?
The work needs named owners on your side more than it needs tooling. Expect to provide:
- An accountable executive owner for the assistant and a risk owner who can accept residual risks
- Subject matter experts who write and review test questions with correct reference answers, since MEASURE work depends on them
- Read access to the document sources, identity systems and permission models the assistant will use, plus a current list of user groups
- Access to the team that operates the platform, so logging and change records can be checked
Ask the partner to list these dependencies in the scope with a named contact for each, and to state what happens if an owner or expert is unavailable on a planned date.
Can our own team do the NIST AI RMF scoping without a partner?
Yes, the framework is voluntary and NIST publishes its materials openly, so an internal team can work through the Map and Measure outcomes on its own. The AI RMF Playbook is a voluntary companion with suggested actions and documentation prompts, and NIST says it is neither a checklist nor a set of steps, so the team still has to decide which actions apply. Most internal teams find the hard parts are building a representative test set, measuring validity against real knowledge limits, and keeping evidence current as the corpus changes. A partner adds value where those tasks need repeated engineering work, but an internal team can own the decisions either way.
What does a fixed-scope NIST alignment engagement from DeployOpen cover?
Most DeployOpen engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. For a RAG assistant, a typical written scope would cover the system categorization and documented knowledge limits (MAP 2.2), the validity and reliability tests and documented generalizability limits (MEASURE 2.5), a risk register mapped to the relevant NIST AI 600-1 risks such as confabulation and information integrity, the evaluation harness and test set, and the evidence pack for your assessors. DeployOpen prepares these systems, controls and evidence but does not issue audit reports, attestations or certifications. Ongoing evaluation and testing can be added through a retainer.
Which NIST AI RMF version should a scoping plan reference?
Reference AI 100-1 (AI RMF 1.0, January 2023) and the Generative AI Profile in NIST AI 600-1 (July 26, 2024) explicitly in the plan, with the version date written down. NIST states that AI RMF 1.0 is being revised as part of the White House AI Action Plan and has not published a revised version as of 17 October 2026, and NIST expects a formal review with community input no later than 2028. The Playbook's last content update was in August 2023 and AIRC says it will be updated after the framework is revised. Ask the partner how they will track revisions and record which version each assessment used, so evidence does not age silently.
Which parts of the RAG design does the NIST work test first?
Start with the places where a RAG assistant can state something unsupported or outside its sources. Engineers should confirm four things: retrieval returns the right passages for representative questions, answers cite passages that actually support them, the assistant declines or flags questions that fall outside the corpus, and permission filtering prevents answers from exposing documents a user cannot open. Check also how the assistant handles stale documents after re-indexing, and whether the chosen embedding and chat models behave consistently across updates. Confirm each of these with the platform you choose, since the behavior depends on the implementation.
Which license limits affect the platform choice
Dify and Open WebUI carry license terms that decide who you can serve and how the assistant looks, so check them before you pick a platform. Dify is released under the Dify Open Source License, a modified Apache License 2.0 that forbids using the source code to operate a multi-tenant environment without written authorization. In Dify's terms, one tenant is one workspace. An internal assistant for one organization is usually a single workspace, but a RAG assistant offered to customers or partners through the same installation may fall outside that allowance. The same license also forbids removing or modifying the logo or copyright information in the Dify console, which affects how the console is presented to staff.
Open WebUI works differently. Since v0.6.6 (April 19, 2025), its code is distributed under the Open WebUI License, which adds a branding clause. The clause forbids altering or removing Open WebUI branding in deployments that exceed 50 users in any rolling 30-day period, unless the operator holds an enterprise license or written contributor permission. Once a staff rollout grows past that threshold, the interface has to keep the Open WebUI branding unless a license is purchased. Enterprise licenses allow white-labeling, and pricing is quoted only after contacting enterprise sales with an expected seat count. Count active users per rolling 30 days, not licensed seats, and record that count in your governance file.
Confirm the exact terms with counsel before you commit to either platform, and put the answer in the requirements. Ask counsel whether your planned audience (internal staff, contractors, customers or partners) counts as multi-tenant use under the Dify license, whether a managed or hosted deployment by a service partner changes that analysis, and how the Open WebUI threshold applies to your user counting method. Ask the implementation partner to show how the chosen version's license file reads, and record the version you tested. Licenses change between releases, so re-check terms at each upgrade.
How DeployOpen scopes NIST alignment for a RAG assistant
The sequence turns the NIST AI RMF and its Generative AI Profile into a scoped set of decisions, documented knowledge limits, validity tests and evidence that the organization can hand to its own assessors.
Confirm scope and sector obligations
DeployOpen works with the organization's legal, risk and business owners to define which assistant is in scope, who uses it, which document collections it can reach and which sector rules apply. The AI RMF is voluntary, so the scope is set by the organization's own obligations and commitments, not by a NIST requirement. Done means a written scope statement that names the users, the sources and the decision owner for each risk area.
Map GOVERN decisions to the assistant
Because GOVERN is cross-cutting, DeployOpen records the policies, roles and accountability the organization already has, then assigns owners for source content, answer quality and incident response. The organization decides which GOVERN categories and subcategories it adopts, since NIST does not present them as a checklist. Done means a governance register that links each decision to a named accountable role and to the assistant's scope statement.
Document MAP 2.2 knowledge limits
DeployOpen documents what the assistant knows and does not know, including source coverage gaps, document dates, settings outside the intended use and dependencies on upstream content and connected systems. The Generative AI Profile ties confabulation and information integrity risks to this work, and it names end users as the AI actor for MAP 2.2. Done means a knowledge limits record that tells users how to read citations, when to escalate and how humans should oversee answers.
Build the MEASURE 2.5 test set
DeployOpen builds a representative question set from real workflows, with expected sources and answers agreed with the business owners. Tests cover retrieval quality, citation accuracy, refusal behavior on out-of-scope questions and performance on documents outside the development corpus, so the limits of generalizability are recorded. Done means a versioned test set, documented pass criteria and a dated results report that the organization can rerun after each change.
Configure logging and citations on the stack
DeployOpen configures the self-hosted assistant, its vector database and its model gateway to record queries, retrieved chunks, citations and model versions in the organization's own storage. Citations link each answer to the source passage and version, so reviewers can trace claims. Retention periods for these logs are a decision for the organization and its counsel, with Article 19(1) of the EU AI Act as one reference if the system is high-risk. Done means a logging specification and a sample trace that a reviewer can follow from answer to source.
Hand over the evidence package
DeployOpen assembles the scope statement, governance register, knowledge limits record, test results, logging specification and change log into one evidence package, then walks the owners through it. DeployOpen prepares this evidence but does not issue audit reports, attestations or certifications, and NIST does not certify AI RMF use. Independent auditors or the organization's own assessors decide what the evidence supports. Done means a signed handover with a named owner for each artifact and a schedule for re-running the MEASURE tests after material changes.
How DeployOpen works on this
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure. For a RAG knowledge assistant mapped to the NIST AI RMF, that means writing down the knowledge limits and validity evidence first, then building the assistant and the records that show how it was tested and governed.
How we scope
- Written scope fixes which document sources, user groups and generative AI risks from NIST AI 600-1 are in and out of the engagement.
- Scope records the MAP 2.2 knowledge limits, intended uses and human oversight roles that the assistant's documentation will cover.
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts; ongoing support is available as a retainer.
How we build
- Deploy the chosen platform, such as Dify, Open WebUI, RAGFlow or AnythingLLM, with a self-hosted vector database in your own infrastructure.
- Build evaluation sets that test retrieval, citation accuracy and answer validity against the MEASURE 2.5 operating conditions and limits.
- Test human-AI configuration with real staff workflows, and run standard software testing on connectors, permissions and failure handling.
How we hand over
- Deliver a documented map of the assistant's dependencies on upstream documents, models and external networks, with ownership assigned.
- Provide test results, known limits and the validity evidence that supports the MAP and MEASURE outcomes, in a form your reviewers can audit.
- Hand over runbooks for re-indexing, model changes and incident handling, with admin training for the teams who will run the system.
How we support
- Keep the platform patched and upgraded, tracking release changes such as Dify's monthly minor releases, and retest after each change.
- Re-run evaluation sets when source documents, models or prompts change, and update the risk records accordingly.
- Offer continuous security testing of the assistant, including prompt injection through documents, under an optional retainer.
