Find out how to Use Hybrid Seek for Higher LLM RAG Retrieval

Constructing a complicated local LLM RAG pipeline by combining dense embeddings with BM25

The essential Retrieval-Augmented Generation (RAG) pipeline uses an encoder model to go looking for similar documents when given a question.

This can also be called semantic search since the encoder transforms text right into a high-dimensional vector representation (called an embedding) during which semantically similar texts are close together.

Before we had Large Language Models (LLMs) to create these vector embeddings, the BM25 algorithm was a very talked-about search algorithm. BM25 focuses on necessary keywords and appears for exact matches within the available documents. This approach is known as keyword search.

If you would like to take your RAG pipeline to the subsequent level, it is advisable to try hybrid search. Hybrid search combines the advantages of keyword search and semantic search to enhance search quality.

In this text, we’ll cover the speculation and implement all three search approaches in Python.

Table of Contents

· RAG Retrieval
∘ Keyword Search With BM25
∘ Semantic Search With Dense Embeddings
∘ Semantic Search or Hybrid Search?
∘ Hybrid Search
∘ Putting It All Together
·…