Retrieval-Augmented Generation

What is retrieval-augmented generation?

Retrieval-augmented generation (RAG) is a method that connects a language model to an external knowledge source. Instead of producing an answer from the knowledge stored inside the model alone, the system first retrieves relevant documents or passages from a database and supplies them to the model as additional context. The approach was introduced in 2020 by Patrick Lewis and colleagues.

How does RAG work?

RAG combines two components. A retriever searches a knowledge base — often via semantic vector search — and finds the passages most relevant to a user's question. These are passed, together with the original question, to a generator (the language model), which formulates the final answer. In the original paper, the authors used a neural retriever over a Wikipedia index and a sequence-to-sequence model as the generator.

As preparation, the source texts are split into small chunks, converted into vectors and stored in an index; only this index makes fast semantic search possible. In research terms, RAG combines parametric knowledge (stored in the model) with non-parametric knowledge (the external, searchable source).

Why is RAG useful?

Plain language models have a fixed knowledge cut-off and can fabricate facts. RAG mitigates both problems: because the answer is grounded in retrieved, checkable sources, the risk of hallucination drops, and new knowledge can be added without retraining the model — you simply update the knowledge base. Sources can also be cited, making answers easier to verify.

RAG is not error-free either: if the retriever surfaces irrelevant or contradictory passages, the answer can still be wrong. RAG matters for learning with AI too: tools that answer from your own materials — such as lecture notes or scripts — work on this principle at their core. Their quality depends heavily on how good the supplied sources are. A NotebookLM alternative for learning is one example of this pattern in practice.