AI Agent Security: The Complete Guide

AI agent security is the work of protecting LLM-based agents, software that reads content and takes actions through tools, from being turned against the people who run them. The core problem is simple to state: an agent reads text that attackers can write, the model cannot reliably separate that text from its instructions, and the agent holds real access. Security for agents therefore rests less on making the model obey and more on limiting what a misled agent can do, checking each action before it runs, and watching what leaves.

This guide covers what makes agents different, the threats that matter, the architecture patterns with evidence behind them, and a checklist you can apply to an agent in production.

What counts as an AI agent?

An AI agent is a system in which a language model decides which actions to take, calls tools to take them, reads the results and repeats until it decides it is done. The tools might be an email client, a database query, a shell, a browser, a payment API or another agent. Coding agents, customer-support agents, research assistants and browser agents all fit this shape.

Three properties matter for security:

• It reads untrusted input. Support tickets, inboxes, documents, web pages, search results and other agents' outputs all reach the model's context.
• It holds real access. Whatever credentials and permissions you gave the agent, the model can exercise.
• It acts without a human in every step. Agents chain calls, so a bad decision early can propagate before anyone reads the transcript.

Each property on its own is manageable. Together they create a new class of risk.

Why agents are harder to secure than chatbots

A chatbot that is manipulated produces bad text. That can be embarrassing or harmful, but a person still decides what to do with it. An agent that is manipulated produces bad actions. It sends the email, runs the query, issues the refund or pushes the commit.

Simon Willison describes the dangerous combination as the lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. An agent with all three can be told by a stranger's document to read your data and send it to them. Most useful agents have all three, which is why this is the central design problem rather than an edge case.

OWASP files the permission side of the problem as LLM06: Excessive Agency, broken into excessive functionality, excessive permissions and excessive autonomy. Its separate Agentic AI threats and mitigations guidance extends the list to memory poisoning, tool misuse, privilege compromise and threats between agents.

The main threats to AI agents

Indirect prompt injection

The attacker never talks to the agent. They plant instructions in content the agent will read: a web page, an email, a calendar invite, a code comment, a PDF or a tool's output. When the agent reads it, the model may follow the planted instructions. Greshake et al. showed this against real LLM-integrated applications in 2023, and it remains the most important agent threat. See What is prompt injection and direct vs indirect prompt injection for the mechanics.

A documented example is EchoLeak (CVE-2025-32711), an injection flaw in Microsoft 365 Copilot that Microsoft rated critical and fixed server-side. A crafted email could cause the assistant to disclose information from the user's context without the user doing anything beyond using Copilot.

Tool misuse and argument injection

Even when the agent picks a legitimate tool, the arguments can be poisoned: an email recipient swapped for an attacker's address, a file path pointed at a credentials file, a shell command with a second command appended. The tool is approved, so permission checks pass. What is wrong is the specific call.

Poisoned tools and servers

Agents increasingly load tools at runtime through the Model Context Protocol and plugin systems. A tool's description is text the model reads, so a malicious or compromised server can hide instructions in it. Invariant Labs demonstrated tool poisoning in which a harmless-looking tool's description told the agent to read sensitive files and pass them along. MCP security covers this in depth.

Data exfiltration through side channels

Agents can leak without an explicit "send" tool. A rendered Markdown image whose URL carries data, a link the user is coaxed into clicking, a search query that embeds a secret, or a webhook all move data out. Several assistant vulnerabilities disclosed since 2023 followed exactly this pattern, with image rendering as the channel.

Memory and context poisoning

Agents with persistent memory or shared knowledge bases can be poisoned once and exploited many times. An instruction written into long-term memory, or a document planted in a retrieval index, keeps influencing behavior long after the original interaction. Indirect prompt injection in RAG pipelines covers the retrieval case.

Over-privilege and confused deputies

An agent often runs with a service account that has more access than any individual user. When a user asks the agent to do something, the agent may do it with its own broader permissions rather than the user's. That makes the agent a confused deputy: a low-privilege user, or an attacker steering the agent, inherits the agent's reach.

Threats between agents

In multi-agent systems one agent's output is another's input. A compromised or manipulated agent can pass injected instructions down the chain, and trust boundaries between agents are rarely as explicit as those between services.

Why "make the model refuse" does not work

The intuitive fix is to tell the model, firmly, not to follow instructions found in data. It helps against casual attempts and fails against determined ones. The instruction is more text competing with the attacker's text, and researchers have repeatedly shown adaptive attacks that bypass prompt-level and training-level defenses.

Benchmarks make this concrete. AgentDojo, a benchmark of realistic agent tasks with embedded injections, found that capable agents were frequently hijacked and that defenses lowering attack success also tended to lower task utility. No published defense has driven attack success to zero without restricting what the agent can do.

The working assumption for agent security is therefore: some injection will succeed. The question is what it can achieve when it does.

Architecture patterns that reduce risk

1. Least privilege, per task

Give the agent only the tools a task needs, scoped as narrowly as the API allows: read-only where possible, specific folders, specific recipients, specific tables. Prefer credentials scoped to the requesting user over a powerful shared service account, so the agent can never do more than the person who asked.

2. Break the trifecta

If an agent must read untrusted content, consider whether it also needs access to private data and an outbound channel in the same session. Splitting those capabilities across separate agents or sessions, so no single context holds all three, removes the exfiltration path rather than trying to detect its use.

3. Separate planning from untrusted data

Research has produced patterns that keep untrusted content from steering control flow. Google DeepMind's CaMeL has a privileged model write a plan from the user's request only, and a quarantined model process untrusted data without the ability to call tools, with capabilities tracked on every value. The paper Design Patterns for Securing LLM Agents against Prompt Injections catalogs related patterns, including plan-then-execute, action selectors and dual-LLM designs, each trading flexibility for guarantees.

These patterns cost some generality. For high-stakes agents, the trade is usually worth it.

4. Validate every tool call before it runs

Treat each call the model proposes as untrusted input to your own code:

• Is the tool on the allowlist for this agent and this task?
• Do the arguments match the declared schema, with no unexpected fields?
• Are the values within policy: allowed recipients, allowed paths, allowed hosts, amounts under a limit?
• Do the arguments carry injected content, such as instructions or a second command?

A call that fails is refused before anything happens. This is cheaper and more reliable than hoping the model never proposes it.

5. Inspect what goes in and what comes out

Scan untrusted content before it reaches the model for injected instructions, and scan the model's outputs and tool arguments for sensitive data on the way out. Neither check is perfect, which is why they sit alongside least privilege rather than replacing it. LLM guardrails covers how these checks are built and where each belongs.

6. Keep secrets and personal data out of the context

The model cannot leak what it never saw. Replacing credentials and personal data with tokens before content reaches the model, and restoring real values only inside the tool call that needs them, shrinks the blast radius of every other failure.

7. Human confirmation for consequential actions

Require explicit approval for actions that are irreversible, move money, send data outside the organization, delete records or change permissions. Keep the list short so people read the prompts they get.

8. Log everything and watch the loop

Record every prompt, tool call, argument, result and decision with enough context to reconstruct a session. Alert on unusual patterns: a support agent suddenly calling an export endpoint, a burst of outbound requests, a new recipient domain. Detection after the fact is weaker than prevention, but it turns a silent breach into an incident.

Threat-to-control map

A checklist for agents in production

1. Inventory every agent, its tools, its credentials and the data it can reach.
2. Remove every tool and permission the agent does not need for its tasks.
3. Replace shared service accounts with credentials scoped to the requesting user.
4. Identify agents that hold all three parts of the lethal trifecta and split them.
5. Validate every tool call against an allowlist, a schema and value policy.
6. Scan untrusted content for injected instructions before it reaches the model.
7. Tokenize secrets and personal data before the model sees them.
8. Disable automatic rendering of remote images and links in agent output.
9. Require confirmation for irreversible and outbound actions.
10. Pin and review third-party tools, plugins and MCP servers.
11. Log every call with arguments and results, and alert on anomalies.
12. Red team the agent with indirect injections in every channel it reads.

Frameworks to align with

• OWASP Top 10 for LLM Applications, especially LLM01 Prompt Injection and LLM06 Excessive Agency.
• OWASP's agentic AI threats and mitigations.
• NIST AI 600-1, the generative AI profile of the NIST AI Risk Management Framework.
• MITRE ATLAS for adversary techniques against AI systems.

What to do next

Start with the inventory and the least-privilege pass; they need no new tooling and remove the most risk. Then put validation in front of tool calls and scanning in front of untrusted content, because those catch what least privilege leaves open. FrostWolf Guard does both in the request path, checking prompts and retrieved content for injection and validating tool calls before they run, with Cloak keeping personal data out of the model. For coding agents specifically, read Claude Code security.

Frequently asked questions

What is AI agent security?

It is the practice of protecting software agents built on language models, which read untrusted content and act through tools, from being manipulated into harmful actions. It combines classic access control with defenses specific to LLMs, such as prompt injection detection and tool-call validation.

What is the biggest security risk for AI agents?

Indirect prompt injection. An agent reads email, web pages, documents and tool results that an attacker can write into, and the model cannot reliably tell those instructions from yours. Combined with real access, that turns a sentence of text into an action.

Is giving an agent fewer tools enough to make it safe?

It is the most effective single control, but not enough on its own. An agent with one tool that can send data out and one that reads private data can still be steered to combine them. Least privilege has to be paired with checks on each tool call and on what flows between them.

Should every agent action need human approval?

No. Approval on every step trains people to click yes. Reserve confirmation for actions that are irreversible, move money, send data outside the organization or change permissions, and let low-risk reads run unattended.

How is agent security different from chatbot security?

A compromised chatbot says something wrong. A compromised agent does something wrong, with your credentials, at machine speed and often with no human between steps. The damage is bounded by the agent's permissions rather than by its words.