What Is RAG in AI?

In AI, RAG stands for retrieval-augmented generation. It is a technique that lets an AI model look up relevant information from a specific source, such as company documents or the web, before it answers. Instead of relying only on what it learned during training, the system retrieves relevant information, adds it to the prompt as context, and then generates a response based on it.

What RAG means

The acronym RAG has other meanings outside AI. In project management, for example, it often refers to red, amber and green status ratings. In AI, it almost always means retrieval-augmented generation, and the name describes the process:

• Retrieval: finding relevant information in a collection of documents, a database or the web.
• Augmented: adding that information to the request sent to the AI model.
• Generation: the model writing its answer using that information.

The term comes from a 2020 research paper led by Patrick Lewis, with researchers from Facebook AI Research, University College London and New York University, presented at the NeurIPS conference. Amazon Web Services describes RAG as the process of optimizing a large language model's output "so it references an authoritative knowledge base outside of its training data sources before generating a response."

For the short version, see RAG in the LATHIC Glossary.

Why AI models need to look things up

A large language model learns from a huge amount of text during training, then stores what it learned as patterns inside the model. That creates three practical problems:

• Its knowledge stops at a point in time. A model does not automatically know about events or changes after its training data was collected.
• It does not know your private information. A general model has never seen your company's policies, product manuals or contracts.
• It can fill gaps with plausible guesses. When a model lacks information, it may produce an answer that sounds right but is not.

Retraining a model every time information changes would be slow and expensive. RAG offers a different route: keep the model as it is, and give it the right information at the moment it answers.

How RAG works

A RAG system has two phases: preparing the information in advance, and using it when someone asks a question.

Preparing the knowledge source:

1. Collect the documents. Gather the material the AI should draw on, such as help articles, policies, reports or product specifications. This collection is often called a knowledge base.
2. Split them into chunks. Long documents are broken into smaller passages so the system can find and use just the relevant parts.
3. Index them for search. Each chunk is typically converted into an embedding, a list of numbers that represents its meaning, and stored in a searchable database such as a vector database. Many systems also keep traditional keyword search.

Answering a question:

4. Retrieve. When a user asks a question, the system searches the knowledge base for the passages most relevant to it, often by meaning rather than exact wording.
5. Augment. The best-matching passages are added to the prompt, usually with instructions such as "Answer using only the information below and cite your sources."
6. Generate. The language model writes an answer based on the retrieved passages. Many systems include citations so users can check the original source.

A simple example

An employee asks their company's HR assistant: "How many unused vacation days can I carry into next year?"

• Without RAG, a general AI model might give a generic answer about common vacation policies, or guess.
• With RAG, the system retrieves the passage from the company's current leave policy, adds it to the prompt, and the model answers: "You can carry over up to five days, according to section 4.2 of the Leave Policy," with a link to the document.

The model's writing ability did not change. What changed is the information it had in front of it.

Where you encounter RAG

RAG is behind many AI tools that answer from specific or current information:

• AI search features. Tools such as ChatGPT search can look up web pages and cite sources in their answers, a retrieval-based approach. OpenAI's own help documentation cautions that search results and citations "can be incomplete, outdated, or incorrect."
• Customer support assistants that answer from a company's help center and product documentation.
• Workplace assistants that search internal files, wikis and policies.
• Professional research tools in fields such as law, which retrieve relevant cases and documents before summarizing them.

When an AI tool shows its sources or knows about information newer than its training, some form of retrieval is usually involved. Documents you upload may be searched this way or placed directly into the model's context, depending on the tool.

RAG vs fine-tuning

RAG gives a model relevant information at the moment it answers. Fine-tuning changes the model itself through additional training. They solve different problems, and they can be used together.

If someone in a meeting says, "We're going to use RAG instead of fine-tuning," they usually mean the team wants the AI to answer from a set of documents that can be updated, rather than retraining a model. A company might use both: fine-tuning for how the assistant responds, and RAG for what it knows.

RAG vs LLM

RAG and an LLM are not alternatives. An LLM is the model that understands and generates language. RAG is a system design that feeds that model relevant information before it generates a response.

A RAG system needs an LLM to write its answers. An LLM can run with or without RAG. For more on the model side, see What Is a Large Language Model?.

Does RAG stop AI hallucinations?

No. RAG can reduce made-up answers, but it does not eliminate them.

A preregistered study by researchers at Stanford and Yale, published in the Journal of Empirical Legal Studies in 2025, tested 2024 versions of AI legal research tools that providers had marketed as avoiding hallucinations using retrieval-augmented generation. The tools hallucinated less than a general-purpose chatbot, but each still produced hallucinated answers between 17% and 33% of the time in the study's tests. The providers have updated their tools since.

RAG systems can still go wrong in several ways:

• Retrieval misses. The search pulls the wrong passage, an outdated version or incomplete information.
• The source is wrong. If the documents contain errors, the answer can repeat them.
• The model misreads the source. It can misinterpret, overstate or blend retrieved passages.
• The answer is not there. If no relevant information exists, the model may still produce an answer instead of saying it does not know.
• Documents can contain hidden instructions. Retrieved webpages, emails or files can include text designed to manipulate the AI, a risk known as indirect prompt injection.

That is why good RAG systems show their sources, and why checking those sources still matters. What Is an AI Hallucination? explains why AI models make things up in the first place.

Why RAG matters

RAG is a widely used way for organizations to make AI work with their own information. You are likely to hear it in conversations about internal AI assistants, customer support tools and AI search.

Knowing how it works helps you ask practical questions about any AI tool that claims to "use your data":

• What sources does it search? And who decides what goes in?
• How current are those sources? Outdated documents produce outdated answers.
• Does it show citations? Linking claims to their sources, known as citation grounding, makes answers checkable.
• What happens when the answer is not in the documents? A well-designed system should say so.
• Does it respect access permissions? People should only get answers drawn from documents they are allowed to see.

RAG also connects to AI agents, which can decide on their own when to search for information as part of a larger task. What Is Agentic AI? covers that broader idea.