RAG Systems with ChromaDB and Cross-Encoder Reranking

Scrape any website, chunk content with RecursiveCharacterTextSplitter, embed in ChromaDB, combine keyword and semantic search with hybrid retrieval, and improve accuracy with Cross-Encoder reranking.

Start learning at learnwithparam.com. Regional pricing available with discounts of up to 60%.

What You'll Learn

• Build RAG systems with ChromaDB as the vector store
• Intelligent text chunking with RecursiveCharacterTextSplitter
• Hybrid retrieval combining keyword and semantic search
• Cross-Encoder reranking for improved accuracy
• Effective RAG prompt engineering

Tech Stack

• FastAPI - High-performance async Python web framework
• ChromaDB - Embedded vector database for similarity search
• Sentence Transformers - Local embedding generation
• Cross-Encoder - Neural reranking for better relevance
• BeautifulSoup - Web scraping and content extraction
• LLM Provider Pattern - Supports Fireworks, OpenRouter, Gemini, OpenAI
• Docker - Containerized development

Getting Started

Prerequisites

• Python 3.11+
• uv (installed automatically by make setup)
• An API key from any supported LLM provider

Quick Start

# One command to set up and run
make dev

# Or step by step:
make setup          # Create .env and install dependencies
# Edit .env with your API key
make run            # Start the FastAPI server

With Docker

make build          # Build the Docker image
make up             # Start the container
make logs           # View logs
make down           # Stop the container

API Documentation

Once running, open http://localhost:8000/docs for the interactive Swagger UI.

Challenges

Work through these incrementally to build the full application:

1. The Scraper - Extract and clean content from any website
2. The "Naive" Chatbot - Ask questions without RAG context
3. Building the Index - Chunk, embed, and store content in ChromaDB
4. The Full RAG Pipeline - Retrieve relevant chunks and generate answers
5. The Trustworthy Chatbot - Ground answers strictly in retrieved context
6. The Quality Chatbot - Add Cross-Encoder reranking for better accuracy
7. The Multi-Source Chatbot - Index multiple URLs and query across sources

Makefile Targets

make help           Show all available commands
make setup          Initial setup (create .env, install deps)
make dev            Setup and run (one command!)
make run            Start FastAPI server
make build          Build Docker image
make up             Start container
make down           Stop container
make clean          Remove venv and cache

Learn more

• Start the course: learnwithparam.com/courses/rag-reranking-chromadb
• AI Bootcamp for Software Engineers: learnwithparam.com/ai-bootcamp
• All courses: learnwithparam.com/courses