How does the look-up step find the right passage?
Before anyone asks anything, the documents are cut into chunks of a few paragraphs, and each chunk is turned into an embedding: a list of numbers that stands for its meaning. When a question arrives it gets the same treatment, and the system pulls out the chunks whose numbers sit closest to the question's. Many systems run an ordinary keyword search alongside, because model numbers and part codes need exact matches. The best few chunks go into the prompt with an instruction to answer only from them.
The method takes its name from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University. AI tools that answer from your own files usually work this way.
Why does looking things up cut down on made-up answers?
Because the model has the answer in front of it. Asked from memory about a towing limit, a model produces a likely-sounding number; given page 34 of the manual, it quotes page 34. Updating the system means replacing a document, with no retraining (compare fine-tuning), and the answer can carry its source so a person can check it in seconds.
The weak point moves to the search. If the search pulls the wrong passage, the model answers the wrong question with confidence. If an out-of-date file is still in the pile, the answer is out of date. RAG lowers the rate of hallucination. It does not end it.
What would a caravan dealer load into one? (example figures)
A caravan dealer in Albury sells 14 models. Staff field about 60 phone and email questions a week, such as "What is the ball weight on the 19-footer?" or "Does the warranty cover the inverter?", and each takes about 6 minutes of digging through manuals and PDFs. The dealer loads the owner's manuals, the warranty terms, the service menu (12-month service, $420) and the current price list into a RAG tool.
| Before | With RAG | |
|---|---|---|
| Time per question | 6 minutes | 1 minute to check the cited page |
| Hours a week | 6 | 1 |
| Staff time at $38 an hour | $228 | $38 |
| Tool at $150 a month | None | About $35 a week |
| Saving | About $155 a week |
One rule outweighs the rest: when the price list changes, the old one comes out the same day. A tool that can still find last year's list will quote last year's prices, politely and with a page number.
Is an AI that searches the web doing RAG?
Yes. When a chat app searches the web before answering, it is doing retrieval with the internet as its library, and the answer is only as good as the pages it found. Google's AI Overviews are built the same way from pages in Google's index, which is why answer engine optimisation is about being the page that gets picked.
Does CoraVision answer from your own figures?
Yes, and the idea behind RAG, answer from the source in front of you, is the idea behind how CoraVision handles your marketing figures. Once connected, CoraVision reads Google Ads, Meta and Google Analytics 4, and at launch Google Merchant Center (your products and how they perform) and Google Tag Manager (your containers). It reads once on connecting and every night after, and it only reads: nothing in those accounts is created, changed, paused or deleted.
Questions
Do I need RAG for a chatbot on my website?
If the chatbot answers questions about your business, yes in practice. Without it the model answers from general knowledge and fills the gaps with guesses. Most chatbot builders do this when you upload files or point them at your site, so ask which files it answers from and how you update them.
Is RAG better than fine-tuning?
For facts, nearly always: you change a document instead of retraining a model, and the answer can cite its source. Fine-tuning is the tool for style and format. Some systems use both.