What is RAG?

After the previous series, we get the LLMs.

As "world-class next-word fillers," they're amazing but limited in their usage.

The chatbot-only versions fall short, as the range of answers is predefined. What we mean is that the answers are stored within the parameters.

After you ask a question, you'll get the most statistically accurate set of words.

These might be facts, but they also very plausibly sound gibberish.

That's where RAG comes into play.

Retrieval-augmented generation is what gives LLMs much-needed grounding in actual knowledge that can be easily verified.

Key takeaways:

- RAG helps AI systems provide more accurate and up-to-date answers by pulling information from external sources like databases when needed. This makes the responses relevant and correct. It's particularly useful in areas where accurate information is critical, such as healthcare or customer service.

- RAG reduces mistakes in AI-generated content by verifying the information against real-world data before giving an answer. This is important because it prevents the AI from making up answers that sound right but are actually wrong.

- Using RAG, AI systems can stay current without constant updates. It makes them cheaper and easier to maintain. The AI can adapt to new information by simply accessing updated data sources, rather than being retrained frequently.

Large Language Models, and all other generative AI systems, to be fair, suffer from a phenomenon we call hallucination. This means that the model is very confident that it provides the right answer (output), while it does not.

There are three main aspects of why it happens:

• The models generate answers based on patterns seen in training data. These might be factual, but they don't have to. Information can easily be mixed, as in certain contexts, the word that comes next differs from what we're asking about.
• The model is trained on historical data, and its "current reality" is set back from the actual present time. Without regular updates, it might provide inaccurate information.
• AI may overgeneralize. It creates a "one-size-fits-all" output that seems logical but cannot be applied to a specific question or scenario.

That's a problem, as in most cases, we want to use LLMs in their strengths: summarizing, chatting, providing instructions, and brainstorming.

That's where we see the highest value in LLM-based applications and systems.

The thing is, we cannot let them influence people, knowing that they are misguided.

Reducing hallucinations is crucial to put these models to use as specialist agents, not general models.

RAG basics

Meta introduced RAG in 2020 as an assisting technique to joint language generation with information retrieval.

It is simply (although the math behind it is far from simple) feeding the LLMs with information tailored to a specific query.

What we mean is the information is:

- precise

- contextual

- domain-specific

- based on real-time data

The flow of LLM has slightly changed. Before we get a response, we first retrieve the data from a knowledge base. The context is updated, and a new, more accurate answer is presented.

Here's a more detailed description of how RAG works:

1. The user types in a question (called a query). The query is then normalized and further processed by a model.

2. The retrieval component uses a vector-based search (that's embedding) to find the best match in the knowledge base.

3. The information retrieved is used to update the context for our LLM. Usually, it's an additional input with a higher similarity score. This information will augment the general answer to the model learned during initial training.

4. LLM generates an updated, more detailed, and relevant response.

RAG use cases

RAGs are exactly what the world needs to build AI agents at scale.

Previously, each model had to be fine-tuned.

This means that it had to be trained on additional datasets to perform better in certain areas of interest. While doing so, its general capabilities shrank.

RAG, on the other hand, powers up the LLM without diminishing its general knowledge. This comes at the cost of increased time and resources. There is a tradeoff, but considering overall spending on genAI, it's safe to say it's marginal.

Before we move to use cases, it's important to say that both RAG and fine-tuning can be used together. It's used mostly when we want the model to shine in a particular task.

Now, about the use cases. Let's treat this as an introduction to AI Agents, as RAGs are the baseline for most "expert" systems. These systems are applied to support a certain area, like a set of processes. We want them to execute tasks and communicate with other systems. RAGs make it much easier.

Now, here's a list of areas where RAG-powered LLMs are really useful:

1. Answering questions

RAG's strength here lies in retrieving relevant documents or pieces of information in response to a query, and then generating an answer based on this retrieval. This is a must in domains like customer support, medical inquiries, or any scenario where precision and reliability are critical.

2. Integrating knowledge dynamically

We mean that traditional language models are limited by the data they're trained on. Remember the "September 2021 data cutoff" for GPT-3.5? RAG models mitigate this by integrating current, external data sources. It's a much cheaper and safer way to keep them up-to-date. It makes them effective for industries like news, finance, and science, where new important data appears every second.

3. Creating content

RAG models help AI agents produce more informative and factually accurate content. By using external data, these agents can generate content that is well-informed, diverse in perspective, and rich in detail.

4. Conversating

Conversational AI, like chatbots and virtual assistants, RAG models provide a way to talk more naturally and informatively. They can retrieve conversational cues and relevant facts from their knowledge base in real time, making dialogues more engaging and responsive.

5. Training AI agents

Integrating RAG into AI agent development helps reduce some of the computational costs associated with training very large models. Instead of requiring a model to memorize lots of information, RAG allows the model to retrieve information when needed, focusing computational resources on understanding context and generating appropriate responses.

Few things to remember.

The effectiveness of RAG models depends heavily on the quality and relevance of the data it retrieves.

Retrieval operations increase latency. Optimization is important to keep the user experience high and the system flow smooth.

Training RAGs is quite complex.

Nonetheless, imagine the incredible systems that can be built if we combine the capabilities presented above.

Next week, we'll talk a bit about AI agents. Stay tuned!

For more ML and AI insights, subscribe or follow Sparkbit on LinkedIn.

If you're looking to start an AI project, you can book a free consultation with our CTO here: https://calendly.com/jedrek_sparkbit/ai-consultation