Prompt injection
Prompt injection is an attack in which instructions embedded in content the model reads — a retrieved document, a web page, a user message — cause it to ignore its original instructions and act on the injected ones instead.
It is the defining security problem of retrieval and agent systems, because retrieved content is untrusted input that arrives in the same channel as instructions. There is no complete fix; mitigation means treating retrieved text as data, constraining what tools can do, and testing injection resistance as an explicit eval category rather than assuming it.
Related terms
- Context window
A context window is the maximum amount of text, measured in tokens, that a language model can consider in a single request, covering the prompt, any retrieved documents, the conversation history and the generated response together.
- Tool use
Tool use is a language model’s ability to call defined external functions — querying a database, sending a request, performing a calculation — by emitting a structured call that an application executes and returns results for.
- Eval harness
An eval harness is the test infrastructure for a non-deterministic AI system: a dataset of representative inputs, a method for scoring each output, and a threshold that determines whether a run passes or fails.
Tell us what you are building
Send us the problem in a paragraph. You will get a straight answer on whether we can help, and what we would do first.
Book a call