Adopt whole-file BM25 ranking - #256
Skyline-23 wants to merge 1 commit into
Conversation
- Rank file-first retrieval from raw path and source text with benchmark-compatible BM25. - Persist the file index during graph builds and fall back to live indexing for older caches. - Preserve legacy symbol and graph ranking paths behind focused internal test controls. - Cover tokenization, sidecar parity, prefix filtering, and ranking-layer isolation.
🌱 graft blast radius2 areas changed → 6 areas can be affected. 17 dependent symbols, depth 2. flowchart TB
A0(("Workspace Graph Federation<br/>6 symbols"))
A1(("CLI Engine<br/>3 symbols"))
A2(("MCP Tool Invocation<br/>3 symbols"))
A3(("Pull Request Review<br/>3 symbols"))
A4(("Viewer Build<br/>1 symbol"))
AX(("1 smaller area<br/>1 symbol"))
classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
class A0,A1,A2,A3,A4 reached;
classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
class AX tail;
Who knows this code — 3 people across 8 areas
Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no All 17 dependent symbols, grouped by areaWorkspace Graph Federation — 6 symbols in 4 files
CLI Engine — 3 symbols in 2 files
MCP Tool Invocation — 3 symbols in 1 file
Pull Request Review — 3 symbols in 2 files
Viewer Build — 1 symbol in 1 file
Synchronous Execution — 1 symbol in 1 file
Test signal per changed area — 2 ✓Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.
29 test suites also reference this code37 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.
Open the interactive graph → — click an area to see its dependent symbols at file:line. |
|
Verified locally on top of current The reason I'm not merging it straight away is a product-direction call for @shrishdwi, not a defect: this makes whole-file BM25 the default file-first path, which knowingly reclassifies the guarantees that just landed in #205/#137/#126 (exact top-lock, sibling-span delay, test-file de-ranking) as "legacy" behind @shrishdwi — this is a genuine "which retrieval philosophy is the default" decision. Options: (1) merge as-is (BM25 default, symbol heuristics behind the flag), (2) land it flipped — |
|
Thanks for the careful benchmark! Since this was opened, main's ask pipeline has changed a lot (IDF-weighted scoring, random-walk graph ranking, the fusion step in |
2 participants
Closes #257.
Summary
Motivation
Direct one-hop graph score propagation was not stable enough to ship. Low propagation weights had little effect, while strong propagation admitted graph clusters and caused repository-specific regressions.
The raw-file BM25 baseline improves pooled retrieval quality without language-specific rules or repository-specific tuning.
Benchmark
Four repositories, 155 valid historical PR cases:
Natural R@10 by repository:
NestJS has a small Natural R@10 regression, but its R@1, R@5, MRR, and stem-blind metrics improve. Pooled Natural and stem-blind results improve.
Validation
npm run buildgraft buildgraft checkgit diff --checkThe benchmark artifacts are retained separately in
graft-retrieval-bench/results/file-one-hop-product-bm25-*-k10-b400.json.