Hybrid Search for RAG: Combining Keyword and Vector Retrieval

Hybrid search combines BM25 keyword retrieval with dense vector search to increase the share of relevant chunks your pipeline surfaces before generation. BM25 excels at exact term matching — model names, error codes, abbreviations, proper nouns. Dense vector search excels at semantic similarity, capturing meaning even when the user's phrasing differs from the document's. Reciprocal Rank Fusion (RRF) merges their ranked lists into a single ordering that outperforms either method alone without the need to retrain your embeddings.

Most RAG systems start with vector only retrieval. That works well until users search for something specific. An engineer who types “CUDA_ERROR_OUT_OF_MEMORY” needs the chunk that contains that exact string. A vector model trained on general text will spread its attention across semantically adjacent chunks that never mention the error code at all. Hybrid search closes that gap.

Failure mode 1: vector only misses exact tokens. Dense embeddings represent meaning statistically. When two model names share most of their training context — say, two versions of the same base model — their embeddings land close in the vector space. A query for the specific version may surface the wrong document. The same applies to numeric identifiers, error codes, product SKUs, and rare acronyms that the embedding model saw infrequently during pretraining.

Failure mode 2: keyword only misses conceptual queries. BM25 requires token overlap. A user who asks “how do I prevent my agent from repeating tool calls” will not match a chunk that discusses “idempotency in agentic workflows” if they share no words. Pure keyword systems penalise paraphrase, synonyms, and the natural variation in how different writers describe the same concept.

The core intuition: a term is valuable when it appears in this chunk (term frequency) and appears rarely across the whole corpus (inverse document frequency). A chunk that contains the query term “CUDA_ERROR_OUT_OF_MEMORY” and is one of only three chunks in your corpus that ever mention it scores very high. A chunk that contains “the” is not rewarded at all because “the” is in every document.

For RAG purposes, you build a BM25 index over your chunks at ingest time. At query time you run the same query string against that index and receive a list of (chunk, score) pairs ranked by relevance. This is what Elasticsearch, OpenSearch, and lightweight libraries like BM25S and Tantivy do behind the scenes.

BM25 is not a neural model. It has no understanding of synonyms or paraphrase. But that is also why it is so reliable for exact lookups: there is no ambiguity in how it scores a match. Either the token is there or it is not.

The formula for each document:

RRF(d) = ∑ ( 1 / (k + rank_i(d)) )

Where k is a constant (typically 60) and rank_i(d) is the position of document d in retriever i, counting from 1. If a document does not appear in a retriever's results, its contribution for that retriever is zero.

The k=60 constant matters. It dampens the impact of extreme rank differences. A document ranked 1st by one retriever and 60th by the other scores similarly to a document ranked 5th by both. This makes the fusion robust to a single retriever being very confident and the other being uncertain.

The key property is that documents appearing in both lists get a score from both terms in the sum. Documents appearing in only one list get a score from a single term. This naturally surfaces chunks that both retrievers agree on, which tends to mean they are genuinely relevant.

At ingest time: chunk your documents as you normally would, embed each chunk and upsert into your vector store, then index the same chunks as text documents in a BM25 capable store. Both indexes receive the same chunks — the content is identical, only the index structure differs.

At query time: run the user query through your embedding model, then issue the vector search and keyword search in parallel. Collect the top k results from each (20 to 50 per retriever is typical before fusion), apply RRF across both result lists, then take your final top k from the fused list and pass to the LLM.

Many vector databases now include native hybrid search so you do not need a separate keyword index. Weaviate, Qdrant, Meilisearch, and MongoDB Atlas all support BM25 plus vector search in a single query. If you are already on one of those stacks, enabling hybrid mode is a configuration change rather than an infrastructure addition.

For teams on pure vector stores like Pinecone or FAISS, adding Elasticsearch as the keyword index is the common path. The extra operational cost is real — you now maintain two indexes and a fusion layer — but for corpora with heterogeneous content the retrieval quality gain is substantial enough to justify it. The vector database comparison guide covers which stores have native hybrid support and which require a separate keyword index.

For teams that are unsure, the retrieval evaluation test described earlier (50 representative queries, tag each as exact lookup vs conceptual, measure precision at top 5) will give you a data-driven answer within an afternoon. The full retrieval stack design — including how hybrid search fits with chunking and reranking strategies — is covered in the production RAG guide.

Two indexes to maintain. Every ingest pipeline must write to two systems. Deletes and updates must propagate to both. Schema changes in one index may not automatically apply to the other. You need monitoring on both paths.

Tuning the k constant. k=60 works well as a default but some domains benefit from different values. Teams with very large corpora sometimes lower k to 20 to surface niche results more aggressively. Track the effect empirically before deploying a non-default value.

Fusion is not reranking. RRF combines positional signals. It does not understand whether a chunk is actually relevant to the query — it only knows that both retrievers ranked it highly. If you want deeper relevance scoring, add a cross encoder reranker after the fusion step.