RAG Is Retrieval, Not Memory

The mistake everyone makes first

The first thing people say when they want an AI to answer questions about
their own documents is: "Let's train the model on our data." It sounds
right. It's also the single most common misconception in applied AI.

You almost never train (or fine-tune) a model to teach it facts. Training
adjusts billions of internal weights — it's slow, expensive, and it bakes
knowledge in as a fuzzy statistical average, not as retrievable text. Update
one PDF and you'd have to retrain. Worse, the model still can't tell you
where an answer came from, and it will happily invent details that sound
plausible.

RAG — Retrieval-Augmented Generation — solves the knowledge problem without
touching the model at all. The model doesn't remember your data. It
retrieves it, fresh, every time you ask.

What RAG actually does

Retrieval happens in two phases: an offline indexing phase you run once
(and re-run when content changes), and an online query phase that runs on
every question.