noun · AI technique
RAG
also called retrieval-augmented generation
An open-book exam. Before answering, the AI looks up the right pages and keeps them next to it while it writes.
Retrieval-augmented generation: fetching relevant documents at request time and including them in the model's input so its answer can draw on them.
The application searches your documents for passages related to the question, then adds the best matches to the prompt. The model answers from that supplied text, so what it knows depends on what the search found.
Real-life example
You ask a support bot about your company's refund rules. The bot does not remember them from training. It searches the policy documents, pulls the two most relevant paragraphs, and sends them along with your question. The answer quotes the real policy because the real policy was in front of it.
Playground
Ask a pretend store's support bot a question with and without a lookup, and see what it does when the answer is not in the sources.
Ask the support bot
“Most stores allow returns within about 14 days.”
Answered from memory. It sounds sure, but nothing was checked.
Help centre search
Each passage is scored by how well it matches the question. Only the best ones are sent along.
What the model actually receives
Question: How long do I have to return an item?
~10 tokens, counted against the context window
Illustrative: a six-passage help centre for a pretend shop, with simple keyword matching standing in for a real embedding search.
Where the analogy breaks
-
RAG does not teach the model your documents. It hands over a few passages for one request, and the next request starts over.
-
If the search misses the right passage, the answer will be wrong with confidence. The weakest part is often retrieval, not the model.
-
Retrieved text still has to fit. It uses up the context window, so more passages means less room for everything else.