Chunking
Chunking is the process of splitting source documents into smaller passages for embedding and retrieval, and the chunk size and boundary strategy largely determine how well a RAG system can find relevant information.
Chunks that are too small lose the context needed to answer; chunks that are too large dilute the embedding and retrieve imprecisely. Splitting on semantic boundaries — sections, headings, paragraphs — generally beats fixed character counts. It is the single most common root cause of disappointing RAG performance.
Related terms
- RAG (retrieval-augmented generation)
Retrieval-augmented generation is a technique where relevant documents are fetched from an external store and inserted into a language model’s prompt, so the model answers from that specific source material rather than from its training data alone.
- Vector database
A vector database stores text as numerical embeddings and retrieves entries by semantic similarity rather than keyword match, forming the retrieval layer of most RAG systems.
Tell us what you are building
Send us the problem in a paragraph. You will get a straight answer on whether we can help, and what we would do first.
Book a call