Retrieval-Augmented Generation: Pipeline and Failures

Updated September 20, 2026· 7 min read

Retrieval-augmented generation (RAG) pairs a language model with retrieval from an index so the model can write from supplied passages instead of from training weights alone. Lewis and colleagues formalized the approach in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020; arXiv 2005.11401v4, April 12, 2021). Their models combined a pre-trained seq2seq generator with a dense vector index of Wikipedia. They reported more specific, diverse, and factual language than a parametric-only baseline on the tasks they tested. That is a research result, not a guarantee for every production assistant.

Teams use AI agents and automation when retrieval needs to sit inside a governed workflow. The generator is still a large language model.

A training cutoff describes the base model. It is not an absolute limit on a RAG system. Retrieval can add newer or private facts when those facts are in the index and the retriever finds them. If the index is stale, the cutoff problem is back.

Index, retrieve, rerank, assemble, generate

Google Cloud’s RAG overview describes retrieval, optional reranking, and then generation that is supposed to stay on the supplied facts. OpenAI’s retrieval and file search docs describe chunking, embeddings, keyword plus semantic search, and a ranker before the model writes.

Step	What happens	What can break
Index	Documents are parsed, split, and stored	A file never lands, or the parse drops a table
Retrieve	The question is matched to chunks	A near-miss chunk ranks above the right one
Rerank	Top hits are rescored	A weak ranker keeps the wrong passage
Assemble	Passages are packed into context	Two passages disagree, or the useful sentence was cut
Generate	The LLM writes the answer	The sentence is fluent and not in the context

Fine-tuning still solves a different problem. It can teach tone or a task pattern. It does not refresh a price list. A system can combine both approaches: tune behavior and retrieve current facts.

Failure modes

Failure	What you see	First check
Missing source	The answer omits a policy that exists in the CMS	Was the document ingested?
Stale index	The answer quotes last quarter’s price	When did the index last refresh?
Poor chunk	The model missed a sentence that sits one scroll down	Did the split cut the definition from its terms?
Wrong retrieval	A related but different product is quoted	Did the query match the intended title?
Context conflict	Two retrieved passages disagree	Which one is current, and who decides?
Unsupported generation	A confident sentence has no supporting passage	Can a reviewer highlight the source sentence?

A citation next to a sentence is not proof the sentence appears in that source. Ask for the supporting span.

Google Cloud is direct: if retrieved information is irrelevant, generation can be grounded on the wrong material. Evaluation of groundedness, coverage, and instruction-following is a separate job from reading a fluent paragraph.

Where marketers meet retrieval

Attribute retrieval only where a primary source says it is used.

• Google Search AI features. Google’s AI optimization guide (July 10, 2026) describes retrieval-augmented generation, also called grounding, for AI Overviews and AI Mode: systems retrieve pages from the Search index, then generate a response with links. See also Google AI Overviews.
• Internal assistants. A team can index approved docs, past briefs, and brand rules. The assistant answers from retrieved company text when the index is complete and current.
• Support or sales bots. Current pricing and policy belong in the index. They do not belong only in the model’s training.

OpenAI file search is a hosted retrieval tool over files you upload to a vector store. That is RAG-style retrieval inside that product. It is not a claim that every ChatGPT reply used your site.

Teams can build indexes from approved site content and brand guidance, then test source coverage and citation fidelity separately. A fluent answer can still hide a missing or irrelevant document.

Frequently Asked Questions

What does RAG stand for?

Retrieval-Augmented Generation: retrieve relevant passages from an index, assemble them as context, then generate an answer. The 2020 Lewis et al. paper paired a seq2seq model with a Wikipedia index.

Does retrieval guarantee a grounded answer?

No. A missing file, a stale index, a bad chunk, or a wrong hit can still send the model poor context. The model can also ignore the passages and add unsupported sentences.

Is a model’s knowledge cutoff the same as a RAG system?

No. A cutoff describes the base model’s training. Retrieval can add newer or private facts if those facts are in a fresh index. If the index is stale, the cutoff problem returns.

Do AI Overviews use RAG?

Google’s Search documentation describes retrieval-augmented generation, also called grounding, for AI Overviews and AI Mode: pages are retrieved from the Search index before the response is written. That is Google Search product behavior, not a claim about every chat product.

Sources

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401, v4 April 12, 2021, accessed September 20, 2026.
• Google Cloud, What is Retrieval-Augmented Generation (RAG)?, accessed September 20, 2026.
• OpenAI, Retrieval, accessed September 20, 2026.
• OpenAI, File search, accessed September 20, 2026.
• Google Search Central, Optimizing for generative AI features, updated July 10, 2026, accessed September 20, 2026.