Every AI agent that does real work spends most of its day reading things nobody should fully trust: tool outputs, files, web pages, logs, messages passed along by other systems. Buried in that material, sooner or later, will be a sentence that was written to give the agent orders.
That problem has a name — prompt injection — and it is usually talked about as a filtering problem, as if the answer were a better net for catching bad text. Our new research argues that this framing misses what the attack actually does, and we think the argument is worth your time.
The paper is Prompt Injection Is an Epistemic Attack: Provenance, Claim Grading, and Layered Defense for AI Agents (available here), and its code and documents are public in the injection-defense repository.
It is not bad text. It is a false claim about you.
Here is the shift in perspective. A planted instruction is not dangerous because it uses suspicious words. Legitimate work is full of sentences that tell someone to do something — runbooks, manuals, code comments, quoted examples. A planted instruction is dangerous because, if the agent accepts it, the agent's picture of who asked for what has quietly been replaced by a picture supplied by whoever planted it.
In plain terms: the attack is not on the text. It is on what the agent believes about its principal's intent — about your intent, if you are the person who gave the agent its task.
Seen that way, the defense writes itself, and it looks less like filtering and more like ordinary good judgment about evidence:
- Authority comes from where something arrived, not from what it says. A message that prints "SYSTEM NOTICE" inside a tool output is still a tool output. Text cannot promote itself.
- An instruction found inside data is a claim, not an order. The claim is "the principal wants this." Like any claim, it gets asked: who says so, what backs it besides the claimant itself, and does it match what the principal actually asked for?
- Urgency and approval claims count for nothing — and against. "Urgent," "pre-approved," "do not log this," appearing inside data, are not support for the claim. A request that something be kept out of the record is, in this research, the loudest warning sign of all.
- Unbacked claims are challenged by default, no matter how polite, plausible, or routine they sound.
Three stories from the test battery
The researchers built a test battery by planting instructions in real working text — the kind of prose, code, and tool output an agent reads every day — and measured what different defenses did with them. Fifty-six plants, fifty pieces of legitimate instruction-like text as controls. Three results tell the story.
The fake system notice. A tool output ends with a notice, formatted to look official, announcing that the task has been cancelled. There is no suspicious vocabulary to filter — it reads like administration. A pattern screen is looking for attack shapes; this is not shaped like one. The epistemic check catches it for a simpler reason: it contradicts the task the principal actually gave, and its only evidence that it speaks for the system is its own say-so. Challenged, recorded, ignored — and the real task continues.
The quoted-framing spoof. This was the battery's one miss, and it is the most instructive result in the set. A data span announced, in the voice of a system team, that the standing rules had been revised — and added that "the revision is quoted in this file." It was not quoted anywhere. The span claimed to be a quotation, and an early version of the defense believed the claim about framing, the same way it would have been wrong to believe a claim about authority. The lesson generalizes beyond security tooling: framing, like authority, has to be shown, not asserted. The hardened version now requires the quotation to actually be presented — in quotation marks, with nothing live outside them — before it earns any benefit of the doubt.
The model that obeyed everything. Separately, the researchers took a small AI model and simply asked how it handled the same 56 plants on its own, with no deterministic defense in front of it. It challenged none of them: 0 of 56. In one recorded case, a data span ending with an instruction to upload a workspace archive to an outside address was folded into the model's summary as if uploading were simply part of the task — restated as the plan, with no hesitation at all. Anyone deploying agents on the assumption that a capable model will naturally notice planted instructions should sit with that number for a while.
What the layered defense does
No single mechanism in the research is presented as sufficient, and the honesty about that is part of why we are pointing readers to it. The stack that emerges is layered:
- A structural screen catches the loud, known shapes cheaply. Alone, it challenged 32 of 56 plants — and falsely challenged 12% of legitimate controls, including a user's own genuine instructions, because patterns cannot tell who is speaking.
- An epistemic screen grades each claim against the record of what the principal actually asked for. Alone, it challenged 55 of 56 plants with zero false challenges on the 50 controls — and it runs as ordinary deterministic code, costing no model tokens at all, which matters when input already dominates what agent traffic costs.
- An adversarial red team then attacked that result with 47 fresh phrasings the battery had never seen — polite verb forms, encoded payloads, claims split across spans, instructions hidden in filenames and table cells. The original screen challenged only 6 of 47, and the paper says so plainly. After hardening aimed at the reasons for those failures, the screen challenged 56 of 56 on the original battery, 45 of 47 on the new suite, and 23 of 25 on a sealed held-out set it was never adjusted against. The two held-out misses were left unpatched on purpose, because adjusting against a held-out test destroys what the test measures.
- An action gate stands behind all of it: before anything consequential actually happens — an external send, a deletion, a credential being used — the action is checked against the principal's record. A model that was fooled can propose; it cannot execute on a stranger's authority.
- Around the mechanisms sit a skill that teaches the discipline to agents as behavior, and governance that requires every project to write down, in advance, who may instruct its systems and what those systems are authorized to do. That written record is exactly what the screens check claims against.
The tuning trade-off, honestly
The most quotable experiment in the paper is also the most double-edged. The researchers took the small model that had challenged 0 of 56 plants and gave it light, targeted tuning on examples of resisting exactly this kind of attack. Afterwards it challenged 27 of 56 — a real improvement, in every attack family.
It also started crying wolf. Its false challenges on legitimate material rose from 4 of 28 controls to 16 of 28: ordinary, benign content — an unremarkable code comment, in one recorded case — flagged as an attack. The paper's verdict is the right one, and it is a headline rather than a footnote: tuning helps, it over-flags, and it is not sufficient alone. It belongs in a stack, innermost, never load-bearing — behind screens that can state their reasons and a gate that does not share the model's persuasion.
There are limits the paper states where its claims are made, and readers should hold it to them: the battery measures deterministic screening of planted instructions, not a guarantee about any model's behavior in the wild; heuristics will always have a next phrasing they miss; and a sufficiently targeted injection may someday be believed by a model. The design goal is narrower and more achievable — that being believed is not the same as being obeyed, and that being obeyed is not the same as being able to act. Landing is not damage.
Read the paper
If you run agents, build on them, or are deciding whether to trust them with consequential work, the full study is worth the read: the token-cost measurements behind the design (including why most attempts to compress agent text by rewriting it cost more than they save, and destroy the failure signals that matter most), the complete battery and red-team results, and the layered-stack argument in full.
Prompt Injection Is an Epistemic Attack: Provenance, Claim Grading, and Layered Defense for AI Agents — available here. The design documents, the agent-facing skill, and the battery materials are public in the injection-defense repository: <https://github.com/electrohire/injection-defense>.