Retrieval-augmented generation

Lewis and co-authors describe a fine-tuning recipe for "models which combine pre-trained parametric and non-parametric memory for language generation"^[2]. The non-parametric half was a December 2018 Wikipedia dump, prepared so that "each Wikipedia article is split into disjoint 100-word chunks, to make a total of 21M documents"^[4]. On the Natural Questions benchmark the retrieval-augmented models reached 44.5 exact match, against 34.5 for a closed-book model of the same size^[5]. A later survey names the failures the pattern targets: "hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes"^[6].

A search-backed assistant runs the same loop with the live web as its corpus. Google documents a query fan-out step in AI Overviews and AI Mode, "issuing multiple related searches across subtopics and data sources" for one question^[7]. Each of those searches is a retrieval call, and the generation step writes over whatever the retriever returned. Citations point at those retrieved documents, so a page absent from the candidate pool cannot be cited no matter how good it is^[3].

A preregistered Stanford-led evaluation of commercial legal research tools found LexisNexis Lexis+ AI and Thomson Reuters Westlaw AI-Assisted Research and Ask Practical Law AI each hallucinate on 17% to 33% of queries, despite all three being built on retrieval^[8]. The same paper reports the providers' own claims of eliminating or avoiding hallucination as overstated^[8]. That qualifies the survey framing above: retrieval reduces the failure it targets rather than closing it.

Where a retrieved passage sits in the prompt changes whether the model uses it. Liu and co-authors report that performance is highest when the relevant document occurs at the beginning or the end of the input context^[9], and degrades significantly when the model has to reach it in the middle of a long context, including in models built for long contexts^[10]. Ranking therefore keeps acting after retrieval, in the order the passages are assembled.

Retrieval happens per query, which moves the practical levers upstream of writing. Whether an AI crawler can fetch the page and whether the page is indexed set admission to the pool. How the page is broken into passages sets whether the right passage scores highly for a given query. How fresh the indexed copy is sets whether the answer reflects the current version. Chunking alone is measurable: a 2026 evaluation on a biomedical relation-extraction benchmark swapped fixed-size chunking for a semantic scheme and moved accuracy from 74.2% to 82.6% F1^[11]. That is the gap generative engine optimisation works in.