Mini RAG - Semantic Search Fundamentals

This project is a hands-on exploration of how modern AI retrieval systems work internally.

Instead of directly jumping into frameworks or full RAG pipelines, this project focuses on understanding the core foundation step-by-step:

Text → Embeddings → Similarity Search → Retrieval

The goal is not just to "use AI tools", but to understand what actually happens behind systems like:

• ChatGPT Retrieval
• RAG Pipelines
• AI Search Engines
• Vector Databases
• AI Document Search Systems

---

What We Are Building

We are building a small semantic retrieval engine.

Traditional search systems work using exact keyword matching.

Example:

Query: "CEO of OpenAI"

Matches only if exact words exist.

Semantic Search works differently.

It tries to understand the meaning of text.

Example:

"Who runs OpenAI?"

can still retrieve:

"The CEO of the company is Sam Altman."

even though the words are different.

This is the core idea behind modern AI retrieval systems.

---

Concepts Covered So Far

1. Embeddings

The model converts text into dense numerical vectors called embeddings.

Example:

"The CEO of OpenAI"
      ↓
[0.12, -0.44, 0.91, ...]

These vectors capture semantic meaning instead of exact words.

Texts with similar meaning produce vectors that are closer together in vector space.

---

2. Sentence Transformers

We use:

SentenceTransformer('all-MiniLM-L6-v2')

This is a pretrained embedding model optimized for semantic similarity tasks.

Its job is to transform text into embeddings.

---

3. Semantic Search

Instead of:

• keyword search
• exact matching

we perform:

• meaning-based retrieval

This allows related sentences to be retrieved even when the wording changes.

---

4. Cosine Similarity

After converting text into vectors, we compare them mathematically using cosine similarity.

Higher cosine similarity score means:

• vectors are closer
• meanings are more similar

Example:

0.92 → highly similar
0.15 → weak similarity

---

5. Top-K Retrieval

Instead of retrieving only one result, we retrieve the Top-K most relevant sentences.

Example:

top_k = 2

This is how real retrieval systems work before passing context to LLMs.

---

6. Multi Query Retrieval

The system now supports multiple queries.

For each query:

1. Generate query embedding
2. Compare against stored sentence embeddings
3. Rank by similarity
4. Retrieve Top-K matches

---

Real World Insight

One important observation:

Semantic similarity ≠ factual understanding

Example:

A query about the CEO may sometimes retrieve:

• company-related information
• organization-related information

instead of the exact factual sentence.

Why?

Because embeddings capture semantic closeness, not strict factual reasoning.

This is one of the major challenges in real-world AI retrieval systems.

---

Why Modern RAG Systems Are More Advanced

Production systems improve retrieval using:

• Better embedding models
• Re-ranking models
• Hybrid search
• Metadata filtering
• Vector databases

This project focuses on understanding the foundation first.

---

Current Retrieval Flow

User Query
    ↓
Embedding Model
    ↓
Query Vector
    ↓
Cosine Similarity Search
    ↓
Top-K Retrieval
    ↓
Relevant Sentences

This is already the core retrieval backbone behind:

• RAG systems
• AI search
• semantic document retrieval
• vector database search

---

Technologies Used

• Python
• sentence-transformers
• scikit-learn

---

Installation

Install dependencies:

pip install sentence-transformers scikit-learn

---

Model Used

all-MiniLM-L6-v2

A lightweight and fast sentence-transformer model for semantic similarity tasks.

---

Example Queries

multiple_queries = [
    "Can you tell me about the CEO of OpenAI?",
    "Where is the company headquartered?",
    "Which company's headquarters are in San Francisco?",
    "The main goal of OpenAI?"
]

---

Example Output

Query: Where is the company headquartered?

Relevant Sentence:
The headquarters of the company is located in San Francisco.

Similarity Score: 0.7475

---

What Comes Next

This project will gradually evolve into a complete mini-RAG pipeline.

Next concepts:

• Chunking
• Vector Databases
• FAISS / ChromaDB
• Storing embeddings
• Retrieval from documents
• Context injection into LLMs
• Full RAG pipeline

---

Main Learning Goal

The focus of this project is not just building features.

The focus is understanding:

• how retrieval actually works
• why embeddings matter
• how semantic search differs from traditional search
• why vector databases exist
• how modern RAG systems are built internally

---

Phase 2 - Document Chunk Retrieval

The project has now evolved from simple sentence-level semantic search into chunk-based document retrieval, bringing it closer to real-world RAG retrieval systems.

What Changed

Earlier, retrieval was performed on small static sentences.

Current implementation:

• works on larger documents
• splits documents into chunks
• generates embeddings for chunks
• performs semantic retrieval over document chunks
• retrieves Top-K relevant chunks

Updated retrieval pipeline:

Document
   ↓
Chunking
   ↓
Chunk Embeddings
   ↓
Query Embeddings
   ↓
Cosine Similarity Search
   ↓
Top-K Relevant Chunks

---

# Phase 3 - FAISS Vector Indexing & Retrieval Architecture

The project has now evolved from simple cosine similarity search into indexed vector retrieval using FAISS.

This marks an important transition towards production-style retrieval systems used in modern RAG architectures.

---

# What Changed

Earlier retrieval flow:

```text
Query
   ↓
Cosine Similarity Against All Embeddings
   ↓
Top-K Retrieval

Current retrieval flow:

Query
   ↓
Query Embedding
   ↓
FAISS Vector Index
   ↓
Nearest Neighbor Search
   ↓
Top-K Relevant Chunks

---

Why FAISS Was Introduced

As document collections grow larger, brute-force similarity comparison becomes inefficient.

FAISS solves this by:

• indexing embeddings
• enabling fast nearest neighbor retrieval
• improving scalability for semantic search systems

---

High-Level Retrieval Architecture

---

Architecture Layers

1. Document Processing Layer

Responsible for preparing documents for retrieval.

Flow

Raw Document
      ↓
Chunking
      ↓
Embedding Generation
      ↓
Vector Storage

Responsibilities

• splitting documents into chunks
• generating semantic embeddings
• preparing vectors for indexing

---

2. Embedding Layer

The embedding model converts text into dense vector representations.

Model used:

SentenceTransformer('all-MiniLM-L6-v2')

Example:

"The CEO of OpenAI"
      ↓
[0.21, -0.44, 0.91, ...]

These embeddings capture semantic meaning instead of exact keyword relationships.

---

3. Vector Indexing Layer (FAISS)

Embeddings are stored inside a FAISS vector index for efficient retrieval.

Current index:

faiss.IndexFlatL2()

Uses:

• L2 Distance (Euclidean Distance)

Important difference:

Lower Distance = More Similar

Unlike cosine similarity:

• higher score was better

With L2 distance:

• lower distance is better

---

4. Query Retrieval Pipeline

When a user submits a query:

User Query
      ↓
Query Embedding
      ↓
FAISS Nearest Neighbor Search
      ↓
Top-K Relevant Chunks

The retrieved chunks become contextual retrieval output for downstream AI systems.

---

End-to-End Retrieval Flow

Raw Document
      ↓
Chunking
      ↓
Chunk Embeddings
      ↓
FAISS Vector Index
────────────────────────────
      ↓
User Query
      ↓
Query Embedding
      ↓
Nearest Neighbor Search
      ↓
Top-K Relevant Chunks

---

Key Learning From This Phase

The project has now evolved from:

simple semantic similarity search

into:

indexed semantic retrieval infrastructure

This introduces foundational concepts behind:

• vector databases
• semantic search engines
• scalable retrieval systems
• production RAG pipelines

---

Current Tech Stack

• Python
• sentence-transformers
• scikit-learn
• FAISS

---

Upcoming Improvements

Next planned concepts:

• persistent vector storage
• FAISS index serialization
• ChromaDB
• hybrid search
• reranking
• LLM context injection
• complete RAG pipeline