Context window
A context window is the maximum amount of text, measured in tokens, that a language model can consider in a single request, covering the prompt, any retrieved documents, the conversation history and the generated response together.
A large window removes a hard limit but does not remove the need for good retrieval: models attend unevenly across long contexts, and cost scales with what you send. Retrieving fewer, better passages usually outperforms filling the window, and it is cheaper.
Related terms
- RAG (retrieval-augmented generation)
Retrieval-augmented generation is a technique where relevant documents are fetched from an external store and inserted into a language model’s prompt, so the model answers from that specific source material rather than from its training data alone.
- Chunking
Chunking is the process of splitting source documents into smaller passages for embedding and retrieval, and the chunk size and boundary strategy largely determine how well a RAG system can find relevant information.
- Prompt injection
Prompt injection is an attack in which instructions embedded in content the model reads — a retrieved document, a web page, a user message — cause it to ignore its original instructions and act on the injected ones instead.
Tell us what you are building
Send us the problem in a paragraph. You will get a straight answer on whether we can help, and what we would do first.
Book a call