noun · AI technique
Chunking
also called document splitting
Cutting a long document into bite-sized pieces so the AI can find the right piece later instead of the whole book.
Splitting long documents into smaller pieces that are embedded, stored and retrieved individually in a search or RAG system.
Before indexing, the application splits each document into chunks, by size, paragraph or heading, often with some overlap. Each chunk becomes its own embedding, so a chunk has to make sense on its own when it is retrieved.
Real-life example
You load a 300-page manual into a help bot. Asked about resetting a device, the bot answers with a half-sentence that starts mid-procedure. The manual was cut every 500 words without regard to headings, so the steps landed in two different pieces. Splitting at section boundaries fixes it.
Playground
Cut a manual into pieces and see whether the answer to a question survives in one piece.
Cut the manual into chunks
Question: “How do I descale the kettle?”
- Chunks
- 4
- Answer kept whole
- 100%
- Unrelated text in best chunk
- 28%
One chunk holds the whole answer with little else around it. A search will find it cleanly.
The manual, cut up
Welcome to your Zephyr kettle, a fast and quiet way to boil water. Fill it with water between the MIN and MAX lines before each use. Press the power switch to start heating. The kettle switches itself off when the water boils. Never switch it on while it is empty. To descale the kettle, fill it halfway with equal parts water and white vinegar. Boil it, then leave the mixture to sit for twenty minutes. Pour it away and rinse the kettle twice with clean water. Wipe the outside with a soft damp cloth and never put the base in water. If the kettle will not heat, check that the base is plugged in and the switch is on. For repairs, contact support with your order number. The kettle comes with a two-year limited warranty. Keep the cord away from hot surfaces and out of reach of children.
underlined is the answer. A thin bar marks where a new chunk starts, and dotted words belong to two chunks.
Each chunk becomes its own embedding. If the answer is split across two chunks, neither embedding fully represents it.
Illustrative: one short manual and one question, counting chunks in words. Real systems often count tokens and split on headings.
Where the analogy breaks
-
There is no perfect chunk size. Pieces that are too small lose their context, and pieces that are too big blur the match and waste room in the context window.
-
Cutting through the middle of an idea hurts retrieval. Splitting on headings or paragraphs, and letting neighboring chunks overlap a little, keeps ideas whole.
-
A mistake here cannot be fixed later by the model. If the right passage was cut apart or buried, the search will not bring it back.