Standard Vector Search (RAG) often suffers from a "precision" problem: it retrieves documents that are semantically similar but factually irrelevant. Keyword search (BM25) is precise but misses context.
Neural-Rerank-Engine solves this by implementing a standard Retrieve & Rerank architecture:
- Stage 1 (Retrieval): Fast retrieval of the top 50 candidates using a Hybrid approach (Sparse BM25 + Dense Bi-Encoder Embeddings).
- Stage 2 (Reranking): A heavy Cross-Encoder (BERT) inspects the query-document pairs to re-order results with high semantic understanding.
- Maximize Recall: Use Hybrid Search to ensure the correct answer is somewhere in the top candidates.
- Maximize Precision: Use Reranking to push the best answer to the top 1.
- Latency Optimization: Balance the speed of FAISS with the accuracy of BERT.
We evaluated the system on a subset of the MS MARCO dataset. The results demonstrate a significant precision boost using the hybrid approach:
| Method | MRR@10 (Accuracy) | Avg Latency |
|---|---|---|
| BM25 (Sparse) | 0.5594 | ~7 ms |
| FAISS (Dense) | 0.7977 | ~64 ms |
| Hybrid + Rerank | 0.8282 | ~165 ms |
While this project implements the current industry standard (Bi-Encoder + Cross-Encoder), research in Neural Information Retrieval is moving fast.
Research Insight (2025): Listwise Reranking with LLMs :
A promising direction to reduce the computational cost of Cross-Encoders is explored in the paper "LLMs can reason over BM25 scores to Improve Listwise Reranking" (https://arxiv.org/pdf/2506.14086).
The paper suggests that instead of feeding full document text to a heavy model, we can feed retrieval scores and metadata to a lightweight LLM. The LLM can "reason" about the distribution of scores to re-rank documents efficiently.
Neural-Rerank-Engine/
├── data/
├── notebooks/
│ └── demo.ipynb
├── src/
│ ├── retrievers/ # Sparse (BM25), Dense (FAISS), & Hybrid Logic
│ ├── rerankers/ # BERT Cross-Encoder
│ ├── benchmark.py # Script to calculate MRR & Latency
│ └── build_indices.py # Pipeline to ingest data & build indices
└── requirements.txt