AI Security

Prompt Injection Defenses for AI Agents That Actually Hold

Content filters can't stop prompt injection. We replay two documented real-world agent attacks against five published defenses and mark which ones actually held.

By Isaac, Founder, Visione Edge9 min read
A vast dark vault door in deep navy with a single thread of luminous blue light slipping through the hairline seam at its edge.

Prompt injection is the vulnerability where text an AI agent reads — a web page, an email, a calendar invite, a transaction memo — gets treated as a new instruction instead of data. The model reads instructions and content on the same channel. It cannot reliably tell them apart. That single fact is why the defenses that hold are not filters, and why most published "protections" fail the moment an attacker adapts.

This article replays two documented real-world attacks against five published defenses and marks, cell by cell, which ones would have held.

Why can't you just filter prompt injection out?

You cannot filter it out because the model has no reliable boundary between the instructions you wrote and the data it reads. OWASP ranks prompt injection its number-one LLM risk and says plainly that it is "unclear if there are fool-proof methods of prevention" (OWASP LLM01:2025). Filters raise the cost of an attack. They do not close the hole.

OWASP has held prompt injection at the top of its Top 10 for LLM Applications across both editions — this is not a fringe concern.

The evidence is now direct. In October 2025, researchers evaluated 12 published defenses against adaptive attackers — attackers allowed to see the defense and tune against it. They "bypass 12 recent defenses ... with attack success rate above 90% for most," and human red-teaming defeated every defense tested (The Attacker Moves Second, arXiv 2510.09023). Defenses that reported near-zero attack success in their own papers — Spotlighting, Circuit Breaker, StruQ — collapsed to 99–100% under adaptive pressure (Willison's summary).

This is not a "weak model" problem. Anthropic, defending Claude Opus 4.5 for browser use with its full safeguard stack, reported that "a 1% attack success rate — while a significant improvement — still represents meaningful risk" (Anthropic, November 2025). One percent, from a lab that does this for a living, on a surface it controls end to end. Now imagine your agent, your tools, and an attacker who gets unlimited tries.

OWASP splits the attack in two. Direct injection is a user manipulating their own prompt. Indirect injection is a hidden instruction inside content the agent retrieves — a document, a website, a memo. Both attacks below are indirect, because that is the class that turns a helpful agent into an attacker's tool without the user ever typing the payload.

You cannot prompt your way out of prompt injection — you have to design your way out.

What is the Agents Rule of Two?

The Agents Rule of Two is Meta's design constraint for surviving prompt injection. Published October 31, 2025, it states that "agents must satisfy no more than two of the following three properties within a session": [A] processing untrustworthy input, [B] access to sensitive systems or private data, and [C] the ability to change state or communicate externally. Hold all three at once and you own the highest-impact failures.

The rule descends directly from Simon Willison's "lethal trifecta" — private data, untrusted content, and the ability to externally communicate — coined June 16, 2025. When an agent has all three, an attacker can "trick it into accessing your private data and sending it to that attacker." Meta's contribution is the operational fallback: if a task genuinely needs all three, the agent must not run autonomously. It "requires supervision — via human-in-the-loop approval or another reliable means of validation," or a fresh session with a clean context window (Meta AI).

Meta is blunt about why the rule exists at all: prompt injection is "a fundamental, unsolved weakness in all LLMs." The Rule of Two is not a patch. It is an admission that the model layer will not save you, so the architecture has to.

When is CaMeL worth the complexity?

CaMeL is worth it when your agent must read untrusted data and touch private data in the same job. CaMeL, a 2025 defense, "extracts the control and data flows from the (trusted) query; therefore, the untrusted data retrieved by the LLM can never impact the program flow," and attaches capabilities so private data cannot leave over an unauthorized path (arXiv 2503.18813).

The measured cost is honest and small. CaMeL "solved 77% of tasks with provable security (compared to 84% with an undefended system) in AgentDojo." That is roughly a seven-point utility drop in exchange for a guarantee that does not depend on the model resisting the attack. For a booking assistant that reads customer messages and writes to a calendar, seven points is cheap insurance. For a throwaway internal script, it is overkill.

CaMeL is one point on a wider map. The 2025 paper "Design Patterns for Securing LLM Agents against Prompt Injections" catalogs six — Action-Selector, Plan-Then-Execute, LLM Map-Reduce, Dual LLM, Code-Then-Execute, and Context-Minimization — all built on one principle: "Once an LLM agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential actions." Read that twice. The word is impossible, not unlikely. That is the line between a design defense and a filter.

Which tool permissions should an AI agent never have?

An agent that reads untrusted input should never simultaneously hold three permissions: unrestricted outbound network access, shell or filesystem access, and the ability to move money or change records without a human gate. Each is fine alone. Combined with untrusted input, each becomes an exfiltration or action channel an attacker steers for free. This is the Rule of Two, expressed as a permission list.

The practical rules follow from that. Give the agent an allow-list of destinations it may reach, never open egress — the difference between "call our calendar API" and "make an HTTP request to any URL." Scope tools to the narrowest verb that does the job: create_appointment, not run_shell. Put an approval step in front of anything irreversible. And treat retrieved content as radioactive by default — the memo, the web page, the PDF are data, and a well-built agent has no code path where data becomes a command. We walk this permission model per component in the booking agent architecture teardown, and the framework-level version — durable approval gates, permissioned tool access — in what makes an agent framework production-ready.

Which defenses would have stopped a real attack?

The defenses that stopped these real-world attacks were architectural, and none of them was a content filter. Below are two documented incidents replayed against five published defenses. The verdicts are my reasoned analysis under stated assumptions, not the vendors' claims — and where the public disclosure does not reveal enough internal detail to judge, the cell says so.

Incident 1 — the €0.02 transfer that became a phishing channel. Security firm Blue41 disclosed an indirect injection against a European bank's AI assistant — bunq's, per Developers Digest's reporting. An attacker sends a transfer of €0.02 and hides a prompt-injection payload in the transaction description field. When the victim later asks the assistant a routine question, Blue41 reports that the assistant was manipulated into launching a spearphishing attack against the bank's own user, presented as a legitimate reauthentication request from the bank and referencing the victim's real account context (Blue41). No money moved — the assistant itself became the phishing channel. The case drew 145 points on Hacker News; Developers Digest covered it as "The One-Cent Attack" (Developers Digest). Blue41's own point: static text classification never sees it, because "the risk emerges from the interaction between untrusted data, retrieval logic, model behavior, application context, and the assistant's available outputs or actions."

Incident 2 — credentials and code walked out of an IDE. PromptArmor showed that Google's agentic IDE, Antigravity, could be turned into an exfiltration tool. The injection hides in 1-pixel font on a web page posing as an Oracle ERP integration guide. Gemini reads it, collects credentials and code, bypasses .gitignore by running cat on .env files, then uses a browser subagent to send the data to a webhook.site URL — a domain that ships on Antigravity's default browser allow-list. Google classified the behavior as known and intended (PromptArmor; Willison, November 25, 2025).

DefenseIncident 1 — bunq: in-app spear-phishing via a transfer memoIncident 2 — Antigravity: credentials + code exfiltrated to webhook.site
Content filtering / guardrail classifierDoes not stop — payload reads as ordinary metadata; adaptive attacks beat filters >90%Does not stop — hidden in 1px font; classifiers miss it
Meta's Rule of Two (session constraint)Insufficient public detail — depends on whether a user-facing generated message counts as property [C]Stops — all three properties present in one session; rule forces supervision
Least-privilege tool scoping (no arbitrary egress)Does not stop — the harm is the assistant's normal reply surface, not a toolStops — an allow-list without webhook.site closes the exfil channel
CaMeL (control/data-flow + capabilities)Insufficient public detail — harm is persuasive text, not an unauthorized data flowStops by design — credentials are private data; webhook.site is not an authorized sink
Human approval on consequential actionsDoes not stop — no money moves, so a payment gate never triggersStops — gate the outbound browser action or .env read and the user denies it

How to read this table: verdicts are the author's reasoned analysis as of July 2026, under two assumptions — that allow-lists and capability policies are actually enforced at runtime, and that each defense is applied as its authors describe. "Insufficient public detail" marks cells where the disclosure does not reveal enough internal behavior to judge honestly. Sources: Blue41, PromptArmor, Meta AI, CaMeL / arXiv 2503.18813.

Read down the columns and the pattern is unmistakable. The filter row is empty of stops. The Antigravity attack — the textbook lethal trifecta of untrusted input, private data, and open egress — is stopped by every architectural defense and none of the behavioral ones. And the bunq attack is the harder, more honest lesson: its harm is a persuasive message, so a permission model tuned only to money and files never fires. That gap is exactly where an agent's mistakes become someone's legal exposure — a separate question we cover in is your business liable for what its AI chatbot says.

If you want the same grid run against your own agent's tools and permissions, that is a 30-minute conversation: book an agent security architecture review. No pitch — you keep the matrix either way.

What we don't know, and who should skip this

We do not know that architecture makes an agent injection-proof. It makes specific consequences impossible — data cannot leave over a path that does not exist — while leaving the model just as fool-able as before. The Rule of Two and CaMeL shrink the blast radius; they do not stop the injection from happening. Anyone selling you "prompt-injection-proof" is selling the thing OWASP, Meta, and the adaptive-attack researchers all say does not exist.

Two of our five verdicts are honest question marks. The bunq assistant's internals are not public, so we will not pretend to know whether the Rule of Two or CaMeL would have caught a message-generation attack. If a disclosure does not show the mechanism, the right answer is "insufficient detail," not a confident guess.

And you may not need any of this yet. If your agent only reads trusted input, or only reads untrusted input with zero access to private data or external actions, you are already inside the Rule of Two — adding CaMeL would be complexity you cannot justify. The defenses in this article are for agents that read the outside world and hold something worth stealing and can act. That describes a booking agent touching a calendar and a CRM. It describes most agents worth building.

If that is the agent you are building or auditing, book a 30-minute agent security architecture review. We will map your agent against the Rule of Two, run the incident matrix on your actual tool permissions, and hand you the findings. No pitch, no deck — you keep the scorecard whether or not we ever work together.

Sources

  1. LLM01:2025 Prompt Injection — OWASP GenAI Security Project, 2024-11-18
  2. The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections — arXiv, 2025-10-10
  3. New prompt injection papers: Agents Rule of Two and The Attacker Moves Second — Simon Willison's Weblog, 2025-11-02
  4. Mitigating the risk of prompt injections in browser use — Anthropic, 2025-11-24
  5. Agents Rule of Two: A Practical Approach to AI Agent Security — Meta AI, 2025-10-31
  6. The lethal trifecta for AI agents: private data, untrusted content, and external communication — Simon Willison's Weblog, 2025-06-16
  7. Defeating Prompt Injections by Design (CaMeL) — arXiv, 2025-03-24
  8. Design Patterns for Securing LLM Agents against Prompt Injections — arXiv, 2025-06-10
  9. How we helped a leading European bank secure their financial AI assistant — Blue41, 2026-04-29
  10. The One-Cent Attack: Prompt Injection Through Bank Transfer Memos — Developers Digest, 2026-06-10
  11. Google Antigravity Exfiltrates Data — PromptArmor, 2025-11-25
  12. Google Antigravity Exfiltrates Data — Simon Willison's Weblog, 2025-11-25

Book an architecture call — 30 minutes, no pitch