How RAG (retrieval-augmented generation) works

An animated explainer of retrieval-augmented generation: chunking, embeddings, the vector database and how retrieved text grounds the LLM answer.

Chapters

  1. Indexing: First, documents are split into chunks. An embedding model turns each chunk into a vector, numbers that capture its meaning, and stores it in a vector database.
  2. User asks a question: A user asks a question. The same embedding model turns the question into a vector, so it lands on the same meaning map as the chunks.
  3. Top-k most similar chunks: The vector database finds the chunk vectors sitting closest to the question and returns the top-k, here the four most similar chunks, in milliseconds.
  4. Prompt → grounded answer: The prompt combines instructions, the retrieved chunks and the question. The LLM writes an answer grounded in those chunks, citing its sources instead of guessing.

Make an animated explainer of your own topic →

More explainers

← All explainers

Try AI Flowchart free →