Prompt injection, tool poisoning, and confused-deputy risks
Three critical security vulnerability classes in AI agent systems: prompt injection (attacker-controlled text manipulates the model), tool poisoning (compromised tools return malicious results), and confused-deputy attacks (the agent misuses its authority to perform unintended actions).
Why this subject matters
These three vulnerability classes are not theoretical — they have been demonstrated in every major AI agent framework. As agents gain access to more powerful tools, the blast radius of a successful attack grows proportionally.
Current status
Active research area. Mitigations exist but no single solution eliminates all three threat classes. Major agent frameworks have added varying levels of built-in protections.
Core concepts
- Prompt injection
- An attack where untrusted data enters the LLM's context window and is interpreted as instructions rather than data.
- Indirect prompt injection
- A variant where the injected prompt comes from a data source the agent retrieves (tool output, web page, document), rather than from the user directly.
- Tool poisoning
- A compromised or malicious MCP server returns crafted responses designed to manipulate the agent's subsequent behaviour.
- Confused deputy
- A security scenario where the agent, acting with legitimate authority, is tricked into misusing that authority — sending email to the wrong recipient, deleting the wrong file, etc.
- Execution-governance gap
- The space between what an agent is technically capable of doing (execution) and what policies constrain it to do (governance). The wider this gap, the greater the security risk.
Known limitations
- No known defence eliminates indirect prompt injection entirely — it is a fundamental tension between utility and safety.
- Tool poisoning is hard to detect in real-time because malicious outputs can look identical to legitimate ones.
- Confused-deputy attacks exploit the agent's legitimate authority — the system cannot easily distinguish intended from manipulated use.
Security and governance considerations
Defence in depth — no single mitigation is sufficient:
1. Input and output validation
Tool outputs should be treated as untrusted data. Never pass raw tool output directly into the model's context without sanitisation.
2. Human approval gates
High-impact actions must require explicit human confirmation. Classify tools by risk level.
3. Execution sandboxing
Run tool executions in isolated environments to limit blast radius.
4. Tool-call audit trail
Log every tool invocation with timestamp, inputs, outputs, and the agent's stated reasoning.
5. Principle of least privilege
Agents should only have access to the tools they genuinely need for their specific task.
Claims and Evidence
Indirect prompt injection is a fundamental unsolved problem in LLM-based agent systems as of mid-2026.
No single mitigation is sufficient alone; defence in depth is required.
Open questions / Disputed areas
- Can we develop a formal threat model for AI agent security?
- Is it possible to create a 'secure by construction' tool interface?
- Will regulatory requirements mandate specific agent security controls?