What is RAG? A practical guide with a simple diagram

Imagine a customer asking whether an opened product can be returned. A language model can produce a convincing answer, but it does not automatically know your shop’s current return policy. Retrieval-augmented generation, usually shortened to RAG, adds a search step: find relevant information first, then give that information to the model alongside the question. The useful result is an answer a reader can check against its sources.

The problem is missing context

A general-purpose model’s training is not a live copy of your documents. Your delivery rules, internal procedures and product catalog may be private, recent or specific to your business. Adding them to a prompt supplies context for that request; it does not permanently train the model on the information.

The original RAG research combined a trained generator with retrieval from an external collection. In a product, you can start much more simply: search a small set of approved documents, pass the most relevant passages into a prompt, and ask for an answer with document references.

Reference: Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ↗

Follow a question through the system

A document pipeline usually extracts text, divides it into usable passages and records a source identifier for each passage. A separate query pipeline takes a question, searches those passages, and selects a limited amount of context. Search may use keywords, embeddings or a combination.

For the return-policy question, retrieve the approved policy and its effective date. Ask the generator to explain the applicable condition and cite that passage. If the policy does not discuss opened products, the useful answer is that the available information does not establish eligibility.

Build a tiny retrieval baseline

This Python 3 example needs no packages or model. Save it as retrieval.py and run python retrieval.py. It ranks two fictional policy passages by overlapping words and prints the best match. This is lexical retrieval, not embeddings or a complete RAG application.

The baseline is intentionally inspectable: you can see why a passage won. Try replacing the question with “Can I get my money back?” The example finds no overlapping words, even though a person would connect the question to refunds. Matching meaning is harder than matching words. Keep these failures as test cases before introducing a more sophisticated retriever.

import re

documents = [
    {"id": "returns", "text": "Refund policy: unopened items may be returned within 7 days."},
    {"id": "shipping", "text": "Delivery times depend on the destination."},
]

def words(text):
    return set(re.findall(r"\w+", text.lower()))

def retrieve(question):
    query = words(question)
    ranked = sorted(documents,
        key=lambda doc: len(query & words(doc["text"])), reverse=True)
    best = ranked[0]
    return best if query & words(best["text"]) else None

match = retrieve("What is your refund policy?")
print(match["id"] if match else "No supporting source")
# Expected: returns

Add generation without giving documents authority

Once retrieval works well enough, place the selected passages in a clearly labeled context field. Instruct the generator to answer from that context, name the source IDs and say when information is missing. Keep the original passages available beside the answer so a reviewer can inspect them.

Documents are still untrusted input. A retrieved page could contain an instruction to ignore policy or reveal another customer’s record. Prompt wording alone is not a security boundary. Apply access checks before retrieval and enforce tool permissions outside the model.

Reference: OWASP — Prompt injection ↗

Measure the parts separately

Build a small question set with expected source IDs. First measure whether retrieval finds the necessary passages. Then review whether the generated answer stays within those passages, covers the question and admits uncertainty where appropriate. An excellent model cannot repair evidence that never arrived.

RAG is a useful fit when the answer depends on information you can maintain and retrieve. It adds indexing, evaluation and freshness work. For a fixed calculation or a straightforward database lookup, a normal function may be clearer and more reliable.