Retrieval Augmented Generation (RAG)

RAG is a powerful technique that combines two key components:

• Retrieval: Load files from external sources (e.g. PDF / Word / HTML / Notion)
• Augmented Generation: Improve the quality of LLM responses with retrieved documents

Why RAG?

• LLMs are trained on historic data without any updates to the modern information, if we are able to load in external data sources, we can mitigate this issue and stop it from hallucinating real-world activities.
• RAG can also be used to create custom chatbot with private data for organisations.

Step 1: Retrieval

LangChain provides extensive support for document loading through its Document Loaders.

• In this guide, we will take be using pypdf and its community integration to load in a PDF file:

from langchain_community.document_loaders import PyPDFLoader
# Initialise the loader with files to load
loader = PyPDFLoader(
    'sample.pdf'
)
# Load the files to the Document Abstraction
docs = loader.load()