TL;DR
-
This path teaches RAG in dependency order - each step unlocks the next, from vectors to end-to-end evaluation.
-
Six core steps: embeddings → chunking → vector search → hybrid search → reranking → evaluation.
-
Budget 2–3 weeks at 5–8 hours/week, including a hands-on project indexing 100+ documents.
-
Retrieval is 80% of the work - most beginners over-invest in prompts and models while recall stays broken.
-
Each step links to a deep guide - this page is the roadmap, not the textbook.
Who This Path Is For
Follow this sequence if you:
- Need to build doc Q&A, support bots, or internal search with LLMs
- Already understand Large Language Models basics
- Want a ordered checklist instead of reading RAG once and guessing what to learn next
Not for you yet? Complete Become an AI Engineer Phase 1 first.
Path at a Glance
RAG couples a dense vector index of external knowledge with a sequence-to-sequence generator. At query time, the retriever selects relevant passages and the generator conditions its answer on that evidence.
Source: Meta AI
| Step | Topic | Time | Prerequisite |
|---|---|---|---|
| 1 | Embeddings | 2–3 hr | LLM basics |
| 2 | Chunking | 2 hr | Step 1 |
| 3 | Vector search & storage | 3 hr | Steps 1–2 |
| 4 | Hybrid search | 2 hr | Step 3 |
| 5 | Reranking | 2 hr | Step 4 |
| 6 | Evaluation | 3 hr | Working pipeline |
| Capstone | End-to-end RAG | 4–6 hr | All steps |
Total: ~18–21 hours of focused study + project time
Step 1: Embeddings (2–3 hours)
Read: Embeddings · Embedding Models
What you'll learn
- How text becomes dense vectors; API vs open-source models; model versioning
Hands-on: Embed 20 sentences; verify paraphrases cluster. Checkpoint: Explain why model changes require full re-embedding.
Step 2: Chunking (2 hours)
Read: Chunking Strategies
What you'll learn
- Chunk sizes, overlap, structure-aware splitting, and metadata (tenant, doc type, URL)
Hands-on: Chunk one PDF three ways; test five questions. Checkpoint: Documented chunk strategy with rationale.
Step 3: Vector Search & Storage (3 hours)
Read: Vector Databases · RAG (indexing section)
What you'll learn
- ANN indexes, top-k retrieval, pgvector vs dedicated DBs, incremental indexing
Hands-on: Index chunks; run ten queries; inspect top-5. Checkpoint: Retrieve-only pipeline with latency logged.
Step 4: Hybrid Search (2 hours)
Read: Hybrid Search · Metadata Filtering
What you'll learn
- BM25 + vector fusion, metadata pre-filters, when hybrid beats pure vector
Hands-on: Add BM25; re-run failed queries from Step 3. Checkpoint: Hybrid search with tenant/doc-type filters.
Step 5: Reranking (2 hours)
Read: Re-ranking
What you'll learn
- Retrieve many, rerank few; cross-encoders vs bi-encoders; latency tradeoffs
Hands-on: Rerank top-20 to top-5; compare MRR@5. Checkpoint: Hybrid → rerank → top-5 to LLM.
Step 6: Evaluation (3 hours)
Read: Retrieval Evaluation · RAG Evaluation · Hallucination Detection
What you'll learn
- recall@k, faithfulness, golden sets, RAGAS automation, CI regression gates
Hands-on: 30-question eval set; score retrieval and generation separately. Checkpoint: Baseline scores; know which layer fails.
Capstone: Production-Ready RAG App (4–6 hours)
Integrate all six steps into one application:
- Ingest 100+ documents with structure-aware chunking
- Index with versioned embeddings in a vector DB
- Hybrid search + metadata ACL filters
- Rerank to top-5 before generation
- Citations in every answer
- Eval set with weekly regression run
Common Sequencing Mistakes
| Mistake | Why it hurts | Fix |
|---|---|---|
| Prompt tuning before retrieval works | Model invents answers | Step 6 diagnostics on recall@k first |
| Skipping hybrid search | Misses exact codes and IDs | Step 4 before declaring retrieval "done" |
| k=20 into the LLM without rerank | Noise degrades answers | Step 5: retrieve many, rerank few |
| No metadata on chunks | Cross-tenant leaks, wrong doc types | Step 4 metadata filters |
| One giant eval at the end | No signal during learning | Eval 10 questions after each step |
Production Checklist
Before shipping RAG to users:
- Embedding model ID stored in chunk metadata
- Incremental re-index on document change
- Hybrid search + reranker in production path
- ACL filters at database query level (not prompt-only)
- Citations linked to source documents
- Golden eval set (30+ questions) with CI regression
- P95 latency per stage: embed, search, rerank, generate
- Fallback when retrieval returns zero results
FAQs
Can I skip hybrid search?
Only if queries are pure paraphrases. Production systems almost always add keyword search for codes, SKUs, and rare terms.
RAG vs fine-tuning?
RAG for changing knowledge. Fine-tuning for behavior and format. See Fine-tuning after this path if you need both.
References
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)
- LangChain Retrieval Documentation
- LlamaIndex RAG Documentation
- Pinecone RAG Guide
Further Reading
- OpenAI Embeddings Guide
- Cohere Rerank Documentation
- RAGAS Evaluation Framework
- Meta AI RAG Paper Repository
Summary
- Follow embeddings → chunking → vector search → hybrid → rerank → eval in order.
- Fix retrieval before generation - measure recall@k at every iteration.
- Build a small eval set early; grow it as you discover failure modes.
- The capstone project matters more than completing readings - ship something with citations and scores.