Let's Data ScienceLEARN • BUILD • STAY AHEAD

LearnBuilding RAG Systems & Vector Search

Building RAG Systems
& Vector Search.

Build production retrieval from first principles — BM25, dense embeddings, hybrid search, reranking, ANN indexes, and evaluation. Eight modules where every technique is measured before and after on one realistic corpus. Runnable Python, no API keys.

Course Overview

Modules

8

Duration

~9 hours

Helpwell Running CorpusEvery Concept MeasuredRunnable Python (Pyodide)BM25 → HNSW → Rerank

Module 1 is free. The full 8-module course is part of Pro.

Learning Modules

Each module combines animated explanations, hands-on Python practice, and a knowledge check. Every concept is anchored to the same fictional incident-response SaaS — Helpwell — so you watch one retriever, AskHelpwell, improve from BM25 to a measured, grounded system.

Module 1: The Retrieval Problem

Why a bigger context window is not a free retriever. The lost-in-the-middle U-curve, chunking, and BM25 built from scratch in numpy — plus your first Recall@5 scoreboard on the Helpwell corpus.

Sections

1RAG vs Fine-Tuning vs Long-Context — Answers to Different Questions

2Lost in the Middle — Why a Bigger Window Isn't a Free Retriever

3Chunking — Deciding the Unit of Recall

4Keyword Search From Scratch — TF-IDF, Then the Two Fixes That Make BM25

5The Gold Set — You Can't Improve What You Don't Measure

Module 2: Embeddings as Retrieval Geometry

Intermediate Pro

Meaning becomes geometry. What an embedding encodes, cosine vs dot product, normalization, the curse of dimensionality, and a brute-force dense kNN that finally matches "rollback" to "revert".

Sections

1From Words to Vectors — What an Embedding Actually Encodes

2Measuring Closeness — Cosine, Dot Product, and Normalization

3Brute-Force kNN — Your First Dense Retriever

4The Curse of Dimensionality — Why High-D Similarity Is Weird

5Dense vs Keyword — Shipping AskHelpwell v1

Module 3: Chunking & Metadata

Intermediate Pro

You retrieve chunks, not documents. Fixed, recursive, and semantic splitting; the precision-vs-context size tradeoff measured directly; and metadata filtering by version, area, and freshness.

Sections

1You Retrieve Chunks, Not Documents

2Fixed, Recursive, and Semantic Splitting

3Chunk Size — The Precision vs Context Tradeoff

4Metadata Filtering — Version, Area, and Freshness

5Re-Chunking AskHelpwell

Module 4: Hybrid Search & Fusion

Intermediate Pro

Two witnesses beat one. Where dense and keyword each fail, Reciprocal Rank Fusion, weighted blends, and MMR to kill near-duplicate results — shipping AskHelpwell v2.

Sections

1Two Witnesses — Where Each Retriever Wins and Loses

2Reciprocal Rank Fusion — Combining Ranked Lists

3Weighting the Blend

4MMR — Killing Redundancy in the Top-k

5AskHelpwell v2 — Hybrid + Diversity

Module 5: Scaling Search

Brute force has a ceiling. IVF and k-means cells, the HNSW graph walk, vector quantization, and the recall-vs-speed dial you actually tune in production.

Sections

1Brute Force Has a Ceiling

2IVF — Searching the Right Neighborhood

3HNSW — Navigating a Graph of Vectors

4Quantization — Trading Precision for Memory

5The Recall-vs-Speed Dial

Module 6: Re-Ranking & Measuring Retrieval

Bi-encoder recall vs cross-encoder precision. Two-stage retrieval, and the evaluation that proves it worked: Recall@k, MRR, and nDCG over a labeled gold set.

Sections

1Bi-Encoder Recall vs Cross-Encoder Precision

2Two-Stage Retrieval — Retrieve Wide, Re-Rank Narrow

3Recall@k and MRR

4nDCG — Grading the Whole Ranking

5AskHelpwell v3 — Measured End to End

Module 7: Query Transformation

Sometimes the question is the problem. HyDE, multi-query, step-back prompting, and decomposing multi-hop questions that single-shot retrieval can never answer.

Sections

1When the Question Is the Problem

2HyDE — Retrieve With a Hypothetical Answer

3Multi-Query and Step-Back

4Decomposition — Multi-Hop Questions

5AskHelpwell v4 — Rewriting the Question

Module 8: Grounding, Citations & Answer Evaluation

Closing the loop. Context assembly, grounding and citations, abstaining when evidence is thin, and answer-level faithfulness evaluation — AskHelpwell, final form.

Sections

1Assembling the Context Window

2Grounding and Citations

3Knowing When to Say "I Don't Know"

4Did Retrieval Actually Work? Answer-Level Evaluation

5AskHelpwell Final — Grounded, Cited, Evaluated

Building RAG Systems & Vector Search | Let's Data Science | Let's Data Science