GEMINI-RAG
A progressive, branch-by-branch retrieval engineering curriculum dissecting the limitations of naive vector search. Evolves from raw ChromaDB similarity queries into Maximal Marginal Relevance (MMR) diversity, dynamic SelfQueryRetriever AST query decomposition, and mathematical embedding compression filters.
WHY NAIVE RAG BREAKS IN PRODUCTION
Semantic Clustering & Context Pollution
Most introductory RAG tutorials rely exclusively on naive cosine similarity: embed query, fetch top-K nearest neighbors, and dump them into an LLM prompt. In real production datasets, this creates severe failure modes:
1. Semantic Redundancy: All top 5 chunks often contain virtually identical sentences repeated across document sections.
2. Lost in the Middle: Long uncompressed context windows degrade the LLM's attention heads.
3. Unfiltered Noise: Asking "What happened in 2023?" still pulls 2022 documents if the lexical similarity is slightly higher.
Isolating Techniques on Dedicated Branches
Rather than building a monolith where architectural trade-offs are obscured, I structured Gemini-RAG as an educational, branch-based laboratory.
Each Git branch isolates a single retrieval primitive with working test scripts, State of the Union corpora, and comparative engineering notes—allowing developers to step through the evolution from raw vector search to autonomous query decomposition.
STEP-BY-STEP RETRIEVAL PROGRESSION
origin/similarity-search
Cosine Similarity Search
The foundational baseline. Generates 768-dim embeddings via Google Generative AI and queries ChromaDB via collection.query(). Demonstrates raw semantic retrieval and documents clustering traps.
origin/mmr
Maximal Marginal Relevance
Solves semantic redundancy. Fetches a wide candidate pool (fetch_k=20) and iteratively penalizes chunks similar to already selected results, balancing query relevance with information diversity.
origin/metadata-filtering
Hard Metadata Scoping
Applies categorical partition gates (e.g. where={"filename": "2023.txt"}) prior to vector calculations. Ensures queries targeting specific source documents never waste compute on irrelevant corpora.
origin/self-query
LLM Self-Query Retriever
Employs an LLM query constructor with typed AttributeInfo schemas. Translates natural language questions into structured query ASTs that automatically extract both the search phrase and dynamic metadata filters.
origin/compression
Contextual Compression
Deploys LLMChainExtractor over candidate passages. Discards sentences unrelated to the prompt before injecting context into the answering model, drastically reducing prompt tokens and context pollution.
origin/compression-filter
Mathematical Embedding Filters
High-speed gatekeeper. Replaces the costly secondary LLM compression pass with an EmbeddingsFilter using mathematical similarity thresholds, pruning low-relevance chunks with zero extra LLM API latency.
SEPARATION OF CONCERNS: CHROMADB VS. LANGCHAIN
ChromaDB: Pure Index Execution
ChromaDB is purposefully engineered as a fast, specialized vector storage engine. Its core task is indexing high-dimensional embeddings and computing vector distances (Cosine, L2, IP) at bare-metal speed.
Chroma intentionally omits complex application heuristics like MMR re-ranking or natural-language query translation from its core engine to preserve high-throughput indexing and minimal memory overhead.
LangChain: Memory-Level Reranking
LangChain provides the orchestration logic that wraps around the vector engine. In MMR, LangChain requests an over-sampled candidate pool from Chroma (fetch_k=20) into application memory.
The MMR algorithm runs in Python space, iteratively computing pairwise cross-similarity penalties across the 20 candidates to select the final diverse top 5 chunks before passing them to Google Gemini.
AUTONOMOUS QUERY DECOMPOSITION & COMPRESSION
from langchain_chroma import Chroma
from langchain_google_genai import GoogleGenerativeAIEmbeddings, ChatGoogleGenerativeAI
from langchain.chains.query_constructor.base import AttributeInfo
from langchain.retrievers.self_query.base import SelfQueryRetriever
# 1. Define explicit metadata schema bounds for the LLM Query Constructor
metadata_field_info = [
AttributeInfo(
name="source",
description="The filename or document origin, e.g. 'state_of_the_union_2022.txt'",
type="string",
),
AttributeInfo(
name="year",
description="The calendar year the speech or report was delivered (2022 or 2023)",
type="integer",
),
]
document_content_description = "Official US Presidential State of the Union Addresses"
llm = ChatGoogleGenerativeAI(model="gemini-2.5-pro", temperature=0)
# 2. Build SelfQueryRetriever: Natural Language -> Structured Query AST
retriever = SelfQueryRetriever.from_llm(
llm=llm,
vectorstore=vectorstore,
document_contents=document_content_description,
metadata_field_info=metadata_field_info,
verbose=True
)
# Asking: "What were the remarks on infrastructure in the 2022 address?"
# Emits: query='infrastructure', filter=Comparison(comparator=eq, target='year', value=2022)
results = retriever.invoke("What were the remarks on infrastructure in the 2022 address?")from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import EmbeddingsFilter
from langchain_google_genai import GoogleGenerativeAIEmbeddings
embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001")
# Mathematical Gatekeeper: Discard retrieved chunks with cosine similarity < 0.76
embeddings_filter = EmbeddingsFilter(
embeddings=embeddings,
similarity_threshold=0.76
)
compression_retriever = ContextualCompressionRetriever(
base_compressor=embeddings_filter,
base_retriever=vectorstore.as_retriever(search_kwargs={"k": 10})
)
# Returns only high-signal passages without invoking expensive secondary LLM extraction
compressed_docs = compression_retriever.invoke("Inflation reduction strategies")CURRENT STAGE & THE HARD PART
Reproducible Benchmarking Suite
All 6 branches are fully functional and reproducible. Users can switch branches via git checkout [branch-name] to inspect the exact architectural delta between baseline similarity search, MMR diversity reranking, metadata filtering, self-query AST translation, and embeddings compression.
Multi-Hop Query Latency Bottlenecks
While LLMChainExtractor dramatically cuts prompt context tokens, running a secondary LLM inference pass across 10 retrieved chunks adds 1.2s to 2.8s of round-trip latency. Current active research focuses on hybrid vector-BM25 routing and small quantized cross-encoders to eliminate secondary LLM calls.