NIST guidance on AI agent security: what NIST says

What NIST guidance says about AI agent security today

NIST has no certification for securing AI agents, and its agent-specific guidance today sits in Section 3.5 of AI 100-2e2025 and in the agent overlays planned under the COSAiS project. The AI RMF itself is voluntary: NIST's use cases page states that it does not validate or endorse any organization's use of the framework, so a customer asking how your agents are secured is asking for evidence you produce, not a NIST certificate. Your organization owns the controls. An implementation partner can prepare the agent inventory, logging, tool permissions and test results, but only an independent assessor issues an audit report or attestation.

What you must build depends first on whether the agent can act. An agent that only drafts text for a person to review faces the generative AI risks described in the Generative AI Profile. An agent that calls tools, such as browsing, running code or writing to a ticketing system, inherits the same direct and indirect prompt injection risks as any generative system, with added risk that an attacker can hijack it to execute code or exfiltrate data. Treat any tool-using agent as exposed to prompt injection, because NIST says current mitigations do not fully prevent it. Confirm with your architects which tools each agent can reach and whether any of them read untrusted content.

NIST's design advice follows from that assumption. AI 100-2e2025 describes separating models with different permissions and letting a model reach untrusted data only through well-defined interfaces. For evidence, ask your team to document each agent's tool list, permission scopes and data sources, then test it against tool-output injection. The publication names AgentDojo for this purpose, and Garak and PyRIT as open-source aids for adversarial testing. Check the COSAiS project page for the status of the single-agent and multi-agent overlays before you write them into a customer commitment, since the outline of those overlays may still change.

Which agent risks apply to your deployment

An agent with no tools is a model in a loop that prompts itself, but it cannot call a function, browse, query a database or run code. AI 100-2e2025 defines an agent as software that iteratively prompts a model, processes its outputs and feeds results back into the next prompt, so even this configuration carries the generative AI risks that apply to any model exposed to untrusted input. The main risk is that injected text changes the output the user or a downstream system relies on, without any action being taken. Because nothing executes, the autonomous agent threat and the exfiltration path named in GAI Profile action MS-2.7-001 are mostly reduced to what the model can say.

Output handling: Confirm that downstream systems treat model output as untrusted text, not as commands or markup.
Loop limits: Record the maximum number of iterations and what stops the loop.
Data in context: Inventory what enters the prompt, including system prompts, memory and retrieved text.
Threat assessment: Document the MS-2.7-001 threats you judged not applicable and why.
Testing: Keep prompt injection test results against the model and its system prompt.

Before relying on this tab, check that the deployment truly has no tools, since a tool added later moves it into a higher-risk tab.

How indirect prompt injection through tool output works

AI 100-2e2025 ties indirect prompt injection to resource control. Many generative AI uses have a model interacting with resources it does not own, such as an internet-connected agent that browses pages or a retrieval-augmented generation (RAG) system that reads documents from a shared repository. An attacker does not need access to the prompt. They plant instructions in content the agent will later read, and those instructions can change what the system does, which tools it calls and what data it returns. The attacker's leverage comes from the agent's access to content, so the design question is which resources the agent can reach and what it can do with what it reads.

The same publication states that current mitigations do not fully protect against prompt injection. For that reason, NIST's guidance supports an assumption that designers should plan around: prompt injection is possible whenever a model is exposed to untrusted input, including tool output, web pages, email bodies and retrieved passages. In practice, the reader should map every tool and data source an agent can read, mark each as trusted or untrusted, and ask what the worst outcome would be if an instruction hidden in that source were followed. Where the answer includes code execution, data export or changes to external systems, the design should limit what the model can reach directly rather than relying on the model to ignore the instruction.

Teams that want to test this assumption can start with the tools AI 100-2e2025 names. AgentDojo measures how agents respond to prompt injection carried in data returned by tools, which matches the path described above. Confirm the current version, supported agent frameworks and licensing with the project before adopting it. For a fuller treatment of the attack classes, mitigation families and testing approaches, see the sibling guide on NIST guidance on prompt injection: direct and indirect attacks.

Design measures NIST describes for agent systems

NIST's guidance on agents starts from an assumption: prompt injection cannot be fully prevented by current mitigations, so design has to limit what a compromised model can reach and do. The five measures below are architectural decisions that an organization owns and an implementation partner can help build and document. Each one needs confirmation against the actual agent in your environment.

Separate models by permission

AI 100-2e2025 describes using multiple LLMs with different permissions, so that the model reading untrusted content is not the same model that holds privileged tool access. In practice this means a lower-privilege model summarizes or classifies external input, and a separate component with narrow rights decides on any action. Check which model sees raw external data, which one can call tools, and whether a single prompt can move between them.

Route untrusted data through interfaces

NIST describes letting models reach untrusted data only through well-defined interfaces. Instead of giving an agent open access to a web page, mailbox or document store, the agent calls a fixed function that returns a constrained, typed result. Confirm which data sources the agent can query, what schema the interface enforces, and whether any path lets the model issue arbitrary requests to external systems.

Assume injection for untrusted input

NIST says application designers may assume prompt injection is possible whenever a model is exposed to untrusted input. This is a design baseline, not a test result: the system should be built so that a successful injection has limited effect. Document every point where external text enters a prompt, including tool output and retrieved documents, and confirm that the design does not depend on the model refusing malicious instructions.

Restrict tool permissions

Agents that call tools can be hijacked to execute code or exfiltrate data, so each tool should carry only the rights its task needs. Scope credentials to a single function, use read-only access where writes are not required, and block outbound network paths that the task does not use. Ask for a list of every tool the agent can call, its credentials, and what a misused call could reach.

Treat agent output as untrusted

Before an agent's output triggers an action, such as sending a message, changing a record or running code, the output should be validated as untrusted input. Validation can include schema checks, allow-lists of permitted actions, and human approval for high-impact steps. Confirm where validation sits in the workflow, which actions bypass it, and whether logs record both the proposed action and the decision that allowed it.

What the planned COSAiS agent overlays cover

What is COSAiS and why does it matter for AI agents?

COSAiS is NIST's Control Overlays for Securing AI Systems project, which is building tailored sets of security controls on top of the SP 800-53 control catalog for specific AI use cases. For organizations that already run SP 800-53 or FedRAMP-style control programs, it offers a way to map AI-specific security measures onto controls they already document and assess. Agent overlays matter because agents take actions with tools, so they add risks that a generic model overlay may not address. Check the NIST CSRC COSAiS project page for the current list of drafts and their status before planning around any of them.

Which five AI use cases does COSAiS plan to cover?

COSAiS plans overlays for five use cases:

Adapting and using generative AI, such as assistants and large language models
Using and fine-tuning predictive AI
Single-agent AI systems
Multi-agent AI systems
Security controls for AI developers

The two agent use cases are the ones most relevant to tool-using LLM systems. Confirm the exact scope of each planned overlay on the CSRC project page, since use case definitions can change between drafts.

How do COSAiS overlays relate to SP 800-53?

An overlay is a set of controls that tailors an existing baseline to a particular system type, so COSAiS builds on the SP 800-53 Rev. 5 catalog rather than replacing it. An organization using an agent overlay would still run its SP 800-53 control program, with the overlay adding or adjusting the requirements that apply to agent systems. Your assessor will expect the same evidence discipline either way, so treat the overlay as a tailoring layer on controls you already document. Confirm with your assessor how they will treat a draft overlay before you adopt it in an authorization or audit scope.

Are the agent overlays final or still in draft?

Treat the agent overlays as drafts unless the CSRC project page says otherwise. NIST publishes COSAiS work in stages, and status changes as drafts are revised, so any date or release you read in secondary material may already be out of date. Before you plan a control implementation, check the project page for the current status of each use case, the version number of each document and the public comment history. Do not assume a planned overlay will match a final release in wording or scope.

Do the planned overlays give specific guidance on prompt injection?

The published NIST source on agent risk, AI 100-2e2025, says current mitigations do not fully protect against prompt injection, and it advises designers to assume injection is possible whenever a model reads untrusted input. Prompt injection through tool output is one of the central agent risks, so an agent overlay would be expected to address it, but you should not assume the overlay's specific control language before it is published. In the meantime, design controls around the assumption in AI 100-2e2025, such as limiting tool permissions and separating models that read untrusted data from those that act on it.

Can an internal team prepare for COSAiS agent controls on its own?

Yes, an internal team can start preparing without waiting for final overlays. The work usually includes inventorying agents and the tools they can call, documenting the permissions and data each agent can reach, logging tool calls, and mapping existing SP 800-53 controls to agent-specific risks. Costs come mainly from staff time for inventory and documentation, plus any testing tools you run. External help is often used for engineering the controls, testing them against prompt injection, and documenting evidence. DeployOpen implements, hardens and tests self-hosted platforms in the client's own environment and prepares evidence, but it does not issue assessments or authorizations, which remain with the organization's assessor.

What should I ask an assessor before relying on a draft overlay?

Ask the assessor whether they will accept a draft overlay as a basis for the assessment, and whether they expect the organization to show how it would meet the overlay's intent if the final text differs. Ask how they will test agent-specific controls, what evidence they expect for tool permissions and logs, and whether they will reassess when the overlay is finalized. Confirm the answer in writing, because assessor expectations determine the scope of work more than the draft itself. Also confirm the overlay version you are referencing against the CSRC project page.

Mapping agent security evidence to AI RMF functions

The NIST AI RMF does not name agent controls, and no NIST document maps AI 100-2e2025 agent measures to AI RMF subcategories as an official crosswalk. Organizations that want to show agent security under the framework have to make the connection themselves. The most natural anchor is the characteristic the framework calls secure and resilient, which covers resistance to attacks and the ability to recover. For a tool-using agent, the evidence that supports that characteristic is the set of controls that limit what the agent can reach when its input is untrusted, such as tool permissions, separation between models with different privileges, and logging of every tool call. Record each control as your own design decision, then note which Map, Measure or Manage subcategory it supports in your documentation.

The Generative AI Profile helps here because AI 600-1 ties its suggested actions to AI RMF subcategories. Action MS-2.7-001, for example, asks organizations to assess the likelihood and magnitude of threats including autonomous agents and compromised dependencies, which places agent threat modeling under Measure. Use the profile as a starting vocabulary, not as a verified mapping, and confirm the subcategory references against the current text yourself. The NIST AI RMF Playbook offers suggested documentation prompts for each subcategory, but its audit log shows the last content update in August 2023 and NIST has said it will be updated after the framework is revised. Expect some Playbook wording to change, and build your evidence index around the subcategory IDs rather than the Playbook's exact phrasing.

Because the framework is voluntary, has no checklist and has no certification, the output of this mapping is internal evidence, not a conformity claim. Ask your team to answer three questions for each agent deployment: which AI RMF subcategories the agent's controls are cited against, who owns the evidence and how often it is refreshed, and what would trigger a go or no-go review under MANAGE 1. An independent auditor can test whatever you have documented, but no auditor issues a NIST AI RMF certificate, so the record must stand on its own when a customer or regulator asks for it.

How to prepare agent security evidence for an assessment

This sequence turns an agent deployment into a body of evidence that an independent assessor can test against the design measures and prompt injection findings NIST describes. The organization owns each output, DeployOpen prepares the systems and records, and only the independent assessor issues any report or opinion.

Inventory agents and their tools
List every AI agent in scope, including its underlying model, the functions it can call, the tools it can use such as web browsing or code interpreters, and any memory or planning components. AI 100-2e2025 defines agents by this cycle of prompting, output processing and feedback, so the inventory should record each loop point where a model output triggers an action. The organization owns the inventory, and an implementation partner can help extract tool bindings from configuration. The step is complete when each agent has a named owner and a tool list that engineers have confirmed against the running system.
Define trust boundaries and permissions
For each agent, map which data sources and tools carry untrusted content, such as web pages, retrieved documents or tool outputs, and which actions are high impact, such as writing to systems or executing code. AI 100-2e2025 notes that designers may assume prompt injection is possible whenever a model sees untrusted input, so the design should separate agents with different permissions and limit untrusted data to defined interfaces. The output is a boundary diagram and a permission table showing who can invoke each tool, with the reasons for each grant. The step is done when an engineer outside the design team can trace every untrusted input to the actions it could influence.
Run prompt injection tests and record results
Test each agent for direct prompt injection and for indirect injection through data returned by tools, which is the case AI 100-2e2025 highlights through resource control. The publication names AgentDojo for measuring vulnerability to injection through tool data, and Garak and PyRIT as aids for adversarial ML vulnerabilities. Record the test set, model and tool versions, configuration, inputs, observed actions and pass or fail results for every run, and keep the raw logs. The output is a versioned test report that a reviewer can rerun. Confirm with the tool maintainers how each test harness is licensed and deployed before adopting it in a self-hosted environment.
Document residual risk and go or no-go decisions
Record each confirmed failure from testing, the mitigation applied, and the risk that remains after mitigation. NIST says current mitigations do not fully protect against prompt injection, so residual risk should be stated plainly rather than assumed closed. A named accountable executive should record a go or no-go decision for each agent, the conditions attached to it, and the monitoring that will detect misuse after deployment. The output is a signed risk acceptance record per agent. This step is complete when every open finding has an owner, a decision and a review trigger.
Arrange an independent assessor
Engage an independent auditor, assessor or certification body to test the evidence, since only those parties issue reports or certificates. Ask the assessor to confirm the assessment scope, the criteria it will apply, and whether it will test the agent design measures directly or sample the records. The implementation partner prepares the evidence package and answers technical questions but does not attest to the result. The output is a written scope and an assessor engagement letter that references the inventory, boundary, test and decision records. Ask any assessor how it treats NIST AI RMF material, since there is no AI RMF certification to issue.

Common mistakes when securing LLM agents

Organizations securing LLM agents tend to repeat the same four errors: reading the AI RMF as a certification, trusting mitigations NIST calls incomplete, giving agents more tool access than their task needs, and assuming a tested agent stays safe after its model or tools change. Each card names the error and the check that prevents it.

Treating the AI RMF as a certification

The AI RMF is a voluntary framework, and NIST does not validate or endorse any organization's use of it. An agent program that says it is "NIST certified" will not satisfy a customer or regulator who asks for independent evidence. Ask your team to state which AI RMF subcategories the agent program addresses, then have an independent assessor or auditor test the controls you claim. Only that assessor can issue a report or attestation.

Relying on incomplete prompt injection mitigations

AI 100-2e2025 states that current mitigations do not fully protect against prompt injection. Teams that deploy a filter or system prompt and treat the agent as protected are building on a control NIST describes as incomplete. Design on the assumption that injection is possible whenever the model reads untrusted input, for example by separating models with different permissions or limiting untrusted data to defined interfaces. Confirm this design with your own red team.

Granting agents broad tool permissions

NIST notes that attackers can hijack agents that take actions with tools, leading to arbitrary code execution or data exfiltration. An agent given a shared service account, write access to every system, or unrestricted outbound network access gives an injected instruction a wide blast radius. Scope each tool to the specific functions and inputs the task needs, require separate approval for high-impact actions, and review the permission list whenever a tool is added.

Skipping tests after model or tool changes

Agent behavior depends on the model, the tools it can call and the data it retrieves, so a change to any of them can alter the risk. Tools such as AgentDojo, which NIST names for measuring vulnerability to prompt injection through tool-returned data, are useful only if they run again after each change. Set a rule that a new model version, tool, plugin or retrieval source triggers a repeat of the injection and permission tests, and record the results as evidence.

Questions buyers ask about NIST and AI agent security

Is there a NIST certification for AI agent security?

No. NIST does not certify AI agents, systems or organizations, and it does not run an accreditation or conformity assessment program for AI security guidance. The publications that cover agents, including NIST AI 100-2e2025 and the Control Overlays for Securing AI Systems (COSAiS) project, are technical guidance. Some agent-related overlays are still in development, so a vendor claiming a NIST agent certification is describing something that does not exist. Ask any provider to name the independent body that issued the report or attestation they are offering, and to show the scope it covers.

Does NIST require our organization to control AI agents?

Not directly. The NIST AI Risk Management Framework is intended for voluntary use, and NIST's FAQ states that neither private nor public sector organizations are required to use it. Your obligation to control agents usually comes from elsewhere: a customer contract, a sector regulator, a state law that names the framework, or your own risk appetite. Find out which of these applies before you scope the work, because the contract or regulator determines what evidence you must produce. NIST guidance is then a source of design measures and test methods you can cite.

What does an agent security program need from our team?

It needs named owners on both the engineering and risk sides, because the work crosses both. Engineering must document every tool an agent can call, the permissions each tool carries, the data sources it reads and the points where untrusted content enters the context. Risk and compliance must decide which agents are in scope, what the acceptable behavior is and who approves a deployment. Plan for these inputs:

An inventory of agents, models, tools and data connections
Architecture diagrams that show trust boundaries and permission scopes
Access to logs from agent runs and tool calls
Someone who can decide when an agent should be disabled

Can our own team do agent security work without an outside partner?

Yes, if your team has security engineers who understand LLM applications, and if the agent estate is small and stable. Many organizations start internally with the NIST AI 100-2e2025 taxonomy and the design assumption that prompt injection is possible whenever a model reads untrusted input. The work becomes harder when several agents share tools, when agents act in production systems, or when an assessor will later review the evidence. An outside implementation partner can prepare systems, testing and documentation, but your organization keeps ownership of its controls and decisions.

How long does agent security work take, and what drives the timeline?

The supplied research does not give a duration for agent security work, so treat any number you hear as unverified until the provider shows its basis. The timeline depends mainly on how many agents exist, how many tools each one can call, whether those tools write to production systems, how much logging already exists and how quickly your teams can review the design. Ask any provider for a written scope listing agents, tools, environments and deliverables, and for the dependencies that could push the schedule, such as access approvals or pending design decisions.

What does an implementation partner do, and what does the independent assessor do?

The implementation partner prepares the systems and the evidence. DeployOpen, for example, plans, hardens, integrates and documents self-hosted open-source platforms in your own infrastructure, and runs security testing, including testing of AI systems and agents. Your organization owns its controls and the decisions about risk. An independent auditor, assessor or certification body is the only party that issues an assessment report or certificate. DeployOpen does not issue audit reports, attestations or certifications. Confirm this split in the written scope so that each party's deliverables are clear before work starts.

How DeployOpen prepares agent security evidence

DeployOpen is a Chainscore Labs company that plans, implements, hardens, documents and supports self-hosted open-source platforms in the client's own infrastructure, and runs security testing that includes AI systems and agents. For NIST guidance on AI agent security, we help teams prepare the systems, controls and evidence an organization needs to show how its agents are scoped, tested and governed, while the organization owns its controls and any report comes from an independent assessor.

How we scope

Agree in writing which agents, tools, data sources and permissions fall inside the engagement.
Record which agent architecture applies, such as single-agent or multi-agent, and which tools it can call.
Identify where untrusted content such as web pages, documents or tool output can reach the model.
Confirm which NIST publications and internal control references the written scope will map to.

How we build

Configure agent permissions so tool calls are limited to the functions each agent needs.
Set up separate model roles with different permissions where the design calls for them, and document the boundary.
Run prompt injection tests, including data returned through tools, using open-source test tools and benchmarks such as AgentDojo, Garak and PyRIT.
Log agent actions and tool calls so reviewers can trace what an agent did and why.

How we hand over

Deliver a written record of the agent design, trust boundaries and known limits of current mitigations.
Provide test plans, test results and remediation records mapped to the agreed scope.
Document the operating procedures the client needs to run, update and retire agents safely.
Hand over evidence packages for the client's internal reviewers and for the independent assessor the client selects.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after handover.
Re-test agents when tools, models, permissions or data sources change, as agreed in the retainer scope.
Review NIST publications as they change, including the COSAiS overlays, and agree in writing how any update affects the system.
Confirm with the client's auditor or assessor which evidence they will accept before the final package is issued.

NIST guidance on AI agent security