Retrieval-Augmented Generation: Architecting Trustworthy Knowledge Systems on Top of Large Language Models

Abstract

Fluency and reliability are not the same property, and large language models have far more of the first than the second. A model's factual content is fixed at the point training concludes; its errors are distributed through billions of parameters rather than isolated in any inspectable location; and no signal in its output distinguishes a memorized fact from a well-phrased guess. These three properties — staleness, hallucination, and opacity — are not independent defects but different symptoms of the same design choice: knowledge stored implicitly in weights instead of explicitly in a retrievable structure. Retrieval-augmented generation (RAG) responds by splitting question answering into two stages, one that retrieves candidate evidence from an external, updatable corpus, and one that conditions generation on that evidence rather than on parametric memory alone. This review treats RAG as a systems-engineering discipline. Beginning from the specific failure modes that motivate retrieval augmentation, the discussion works through the component design of a production pipeline — ingestion, chunking, embedding, indexing, hybrid retrieval, and reranking — before turning to the evaluation methodology needed to determine whether a deployed system is actually behaving reliably. The literature surveyed spans dense passage retrieval, approximate nearest-neighbor search, vector database engineering, and automated RAG evaluation, and it converges on a claim central to this review: retrieval quality, not generator choice, is what governs whether a RAG system can be trusted. Consequences for high-stakes domains — finance, healthcare, law — are discussed, along with adaptive retrieval, agentic RAG, and evaluation standardization as directions likely to shape where the field goes next.