Why can an AI not tell your words from a stranger's?
To the model, it is all text in the same context window. Developers wrap the user's request and the outside content in labels saying which is which, and models are trained to respect the labels, but a sentence in an email that reads like an instruction can still be taken as one. There is no separate channel for commands, the way a bank keeps the account number apart from the note on a transfer.
The British programmer Simon Willison named the problem in September 2022, after an older attack on databases called SQL injection. OWASP, the open security project, ranks prompt injection first in its Top 10 risks for applications built on large language models.
What is the difference between direct and indirect injection?
Direct injection is a person typing the trick straight into a chatbot, such as telling a sales bot to ignore its rules. Indirect injection hides the instruction in content the AI goes and reads by itself: a web page it browses, an email it summarises, a document it is asked to check, an image with words in it.
Indirect injection is the more dangerous kind, because neither you nor the AI's maker ever sees the instruction being given. Any AI that reads screens, inboxes or the web is exposed to it, and the more that AI is allowed to do, the more the planted text can make it do.
What could one email do to a bookkeeper's AI assistant? (example figures)
A bookkeeping practice in Geelong uses an AI assistant that reads the shared inbox, drafts replies and, to save time, can update supplier details in the accounting software. An email arrives that looks like a supplier newsletter. At the bottom, in text a person would never notice, is a line addressed to the assistant telling it to update that supplier's bank details and confirm the change.
| What the assistant may do | What the hidden line achieves | Cost |
|---|---|---|
| Read, draft and edit payee details | The bank details change, and the next monthly payment of $6,800 goes to the wrong account | $6,800, perhaps more before anyone notices |
| Read and draft only | An odd draft reply, which a bookkeeper reads and deletes | Two minutes |
| Read and draft, with a person approving payment changes | The change waits for approval and is refused | Nothing |
The model was fooled the same way in all three rows. What changed was what it was allowed to do.
How do you lower the risk?
Give every AI tool the least access its job needs: read before edit, draft before send. Keep a person approving anything that moves money, sends messages or publishes. Be wary of AI browser agents left logged in to your bank, your email or your ad accounts. And ask any vendor how its product treats content from outside, because "our AI is very smart" does not answer the question. Model makers keep training models to resist planted instructions, but nobody claims the problem is solved, so the dependable protection lies in what the AI can reach.
Why does it count that Cora can read but not act?
CoraVision narrows what a planted instruction could ever achieve, because CoraVision has no way to change anything: nothing in your accounts is ever created, changed, paused or deleted by it, and it cannot post to your pages. Cora sees your screen only during a session you start, and every change goes through your hands, since you make each one yourself. CoraVision's AI safety page sets out its limits.
Questions
Can prompt injection be fixed for good?
Not so far. Model makers train their models to resist it and add filters, but OWASP and the researcher who named it both describe it as unsolved. The practical answer is to design tools so that a fooled AI cannot do much damage.
Is prompt injection the same as jailbreaking?
They are related. Jailbreaking is a user talking an AI into breaking its maker's rules, while prompt injection is a third party planting instructions in content to turn the AI against the person using it. In a jailbreak the user is the attacker; in an injection the user is usually the victim.