Start a conversation Contact
← Research

Prompt injection is not a bug you patch. It is what the interface is.

Every mitigation for prompt injection is a filter placed in front of a component that cannot distinguish instructions from data. The defensible architecture assumes the model will be turned against you and limits what that is worth.

The security review reaches the model and stops. Somebody asks about prompt injection, somebody answers that there is input validation and a system prompt that instructs the model to ignore attempts to change its instructions, and the review moves on. It is not a satisfying answer and nobody has a better one, so the meeting agrees to revisit it.

The reason nobody has a better one is that the question was asked at the wrong layer. A language model receives a single stream of text. Some of that text is what you wrote, some of it is what the user typed, and some of it is whatever came back from a document, a web page, a calendar invitation or a support ticket. The model has no channel that carries authority and no channel that carries only content. It has one channel. Instructions in it are instructions, whoever put them there.

The claim: prompt injection cannot be eliminated at the model, so the design question is not how to stop it but what an attacker gets when it works. That question has architectural answers, and they are answers a review can check.

Why the filtering answer keeps failing

OWASP has kept prompt injection at the top of its list for language model applications across both editions of the Top 10 for LLM Applications, and its own guidance is careful to describe defence in depth rather than a control that closes the class. That framing is correct and it is usually read too optimistically. Layered mitigation reduces the rate. It does not change the property, because the property is that natural language has no syntax for “treat the following as data only”.

The direct form of the attack — a user typing instructions at an assistant — is the one everybody tests for and the least interesting. The indirect form is the one that has produced real incidents: instructions hidden in content the system was asked to process. An email nobody opened. A page a research agent visited. A row in a spreadsheet. A ticket submitted by a customer.

Microsoft’s 2025 EchoLeak issue in Microsoft 365 Copilot is the clean example, because it required nothing of the victim at all: a crafted email arriving in the mailbox was enough for the assistant, doing its ordinary job of reading context, to be induced to exfiltrate data it was legitimately entitled to see. The vulnerability record sits in Microsoft’s update guide as CVE-2025-32711. The assistant was not compromised. It was used, at its full existing privilege, by someone who was not its user.

That pattern has a name in security that predates all of this. It is a confused deputy: a component with legitimate authority, persuaded to exercise it on behalf of someone who has none.

The four things worth doing

Where the defensible controls sit, none of them at the model Fig. 01
  1. Control 01 Privilege The system acts with the requesting user’s entitlements, never with a service account that can see everything.
  2. Control 02 Separation Untrusted content is fetched, summarised and quarantined by a component that holds no tools, before it reaches one that does.
  3. Control 03 Irreversibility Every action the system can take is classified as reversible or not, and the irreversible ones require a human who can see what they are approving.
  4. Control 04 Egress Where output can go is constrained by policy — allowed destinations, no arbitrary URLs, no rendering of attacker-supplied links.

The first control is the one that would have contained most of the published incidents. An assistant running as the person asking can only ever leak what that person could already read. An assistant running as an application identity with access to the whole corpus is a single instruction away from being a search engine for an attacker.

The fourth is the most neglected and the cheapest. Exfiltration in these attacks almost always relies on a channel out — an image the client will fetch, a link the user will click, a webhook the agent may call. Constraining where output can travel does not stop the injection. It removes the payoff, which is the only thing an attacker was ever after.

The question to ask about every tool

How to decide whether a tool can be exposed to untrusted content Fig. 02

If a stranger could choose when this tool runs and with what arguments, what is the worst outcome?

  • Nothing leaves and nothing changes Safe to expose. Read-only, scoped, no side effects. Most retrieval and lookup tools sit here, provided the scope is the user’s.
  • Something changes, but it can be reversed and it is logged Expose with a rate limit, an audit record and an owner. Draft, tag, schedule. The kind of action a person can undo the next morning.
  • Something leaves the boundary, or cannot be undone Do not expose it to a path that reads untrusted content. Put a person on it. Payments, sends, deletions, permission changes, external posts.

Running that question over an existing agent takes an afternoon and reliably finds one tool in the third category that is reachable from the first — usually because a capability was added for an internal use case and the same agent later gained the ability to read email or browse.

What to require before an agent reads anything a stranger wrote

What this does not tell you

None of this makes a system safe from prompt injection, and no supplier can offer that either. The honest description of where the field stands is that the attack class is open, the research is active, and the controls above reduce the consequence rather than the likelihood. Anyone selling a product that closes it is describing a filter and calling it a fix.

It also does not mean assistants over untrusted content should not be built. They should — that is most of the useful work. It means the design review has to ask what the system can do rather than what it can be told, and those are different questions with different answers.

The person who decides differently is the architect signing off the tool list. Every entry on it is a thing a stranger may eventually get to call. If that sentence changes which tools go in, the review has done its job.

Filed under · Architecture · Security · Agents · Architecture Inference Institute · 30 Jun 2026

Bring us the question

Reading this because it is on your desk right now?

That is the conversation we are best at. Thirty minutes, a written summary, no obligation.