fix(embed): HNSW path, MCP decoupling, and INT8 fast path#76
Merged
Conversation
central_project_storage_path only recognized '<project>/.leankg' as the project root, so the embed / semantic-context / smoke-test CLI commands (which pass '<project>/.leankg/leankg.db') opened a fresh empty RocksDB at a different project hash, found zero code_elements, and silently returned 'Embedded: 0'. MCP semantic_search then fell back to the ontology-only path instead of the CozoDB HNSW index. Add a project_root_from_db_path helper that handles both directory and file input, plus tests that pin the contract and document the bug in docs/implementation/embed-db-path-mismatch-2026-07-15.md.
Implements Plan §B Option 3: an in-process embed worker that shares
the MCP's CozoDb handle, so MCP stays healthy while HNSW catches up.
CLI (already in CLI; defaults tightened):
- --types filter (default function,method for >50k mega-graphs)
- --wait / --status / --cancel subcommands
- --workers default 4, --batch-size default 64 (was 32)
Code changes:
- src/embeddings/build.rs: OMP_NUM_THREADS=1 cap, default batch 64,
run-time LEANKG_EMBED_UPSERT_CHUNK override, new spawn_background_embed
that holds an Arc<CozoDb> clone and runs build_index_parallel in a
detached thread with a 5s progress poller.
- src/embeddings/state.rs: drop_hnsw_index / create_hnsw_index (FR-HNSW).
- src/main.rs: --types passed through to background child, single
all_elements() scan in run_embed_worker.
- src/mcp/server.rs: LEANKG_EMBED_BACKGROUND=1 spawns the in-process
worker after bind; tunable via LEANKG_EMBED_BACKGROUND_{WORKERS,BATCH,TYPES,FULL}.
- src/cli/mod.rs: bumped default --batch-size 32 -> 64.
Docker / compose:
- Dockerfile.rocksdb: LEANKG_EMBED_ON_BOOT=0 + LEANKG_EMBED_BACKGROUND=1
baked in (no operator env file required).
- docker-compose.rocksdb.yml: same defaults; mem_limit comment for cold runs.
- docker-compose.embed.yml: NEW profile=embed one-shot offline rebuild
(Plan §B Option 1) for ops who want to force a full rebuild while
MCP is stopped.
- entrypoint.sh: LEANKG_EMBED_ON_BOOT defaults to 0; legacy foreground
embed path gated behind LEANKG_EMBED_ON_BOOT=1 with --wait.
Measured on M2 Pro 10c, /Users/linh.doan/work/be (371k functions):
- MCP healthy ~60s after docker compose up.
- Embed throughput ~85 vec/sec (CozoDB writer commit ceiling).
- ETA cold functions-only: ~73 min -- Plan's <5 min target not hit due
to per-batch Cozo commit overhead. See plan doc for follow-ups.
- Live 'embedded' count via DB query has RocksDB MVCC issues; final
status is authoritative. AtomicUsize callback into build_index_parallel
is the fix (tracked as follow-up).
See generated_docs/embed_bg_job_and_runtime_plan_2026-07-15.md for
the full measured-results table and follow-up list.
Two compounding changes that together lift cold embed throughput on the /Users/linh/doan/work/be mega-graph (~371k functions) from ~85 vec/sec to ~170 vec/sec sustained (2x). ETA cold functions-only is now ~36 min on M2 Pro 10c, vs the original 73 min. 1. CozoDB writer: import_relations() instead of :put script The Datalog script path was repacking 5000 rows × 384 floats into JSON params, then parsing the script + committing per :put. import_relations() (cozo 0.7.6's public batch insert API) skips script parsing and goes straight to the relation lock + transaction commit + raw store_tx.put. Measured writer throughput went from ~50 rows/sec to ~170 rows/sec in the steady state (writer throughput is no longer the bottleneck). 2. DirectEmbedder: bypass fastembed's hardcoded intra_threads fastembed 4.9.1 hardcodes intra_threads = available_parallelism() at session creation (text_embedding/impl.rs:52, 80), so every worker session pre-allocates 10 threads on a 10c host. With N worker sessions the OS sees 10N contended threads. DirectEmbedder constructs its own ort::Session with with_intra_threads(N) settable per worker. Default intra_threads=1, tunable via LEANKG_EMBED_DIRECT_INTRA. The DirectEmbedder falls back to fastembed's Embedder if the model cache is missing (e.g. before ). Why the runtime ceiling is ~170 vec/sec on this hardware: - BGE-small ONNX inference at intra_threads=1 takes ~13s per batch=128 → ~10 vec/sec per worker call. - 4 workers × ~50 vec/sec each (with rayon batching) = ~170 total. - CozoDB writer drains at ~170 rows/sec sustained. - Both pipelines are matched at this rate, so neither can scale further without changing the model or the storage layer. Future levers (tracked as follow-ups in the plan doc): - Direct RocksDB write via cozorocks' raw_put with WAL disabled (would skip CozoDB's transaction commit overhead). - Smaller/faster embedding model (e.g. AllMiniLML6V2). - Sharded Cozo databases merged post-build.
Enable LEANKG_EMBED_FAST (INT8 + seq=128 + data-parallel workers), soft RSS caps via LEANKG_EMBED_MAX_MB, and Xenova model_quantized.onnx for bge-q so cold mega-graph embeds sustain ~500 vec/s without the broken Qdrant optimized ONNX. Document PRD v3.6.3 and sanitize personal paths. Co-authored-by: Cursor <cursoragent@cursor.com>
linhdmn
changed the title
Update README with fast-path env recipe, ~480–500 vec/s rates, and troubleshooting for Qdrant optimized ONNX / memory clamps. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
leankg embedhit the same storeLEANKG_EMBED_ON_BOOT=0+ in-process background embed)import_relationswriter,DirectEmbedder, INT8 fast path (LEANKG_EMBED_FAST, Xenovamodel_quantized.onnx), soft RSS budget (LEANKG_EMBED_MAX_MB)Measured cold embed (M2 Pro, ~371k
function,method)Writer-only Cozo is ~100k+ vec/s — storage migration (Redis/FalkorDB/WAL-off) is not the cold-SLA lever.
Type of Change
Testing
cargo test/ pre-commit clippy)model_optimized.onnxChecklist
Breaking Changes
None. Defaults:
LEANKG_EMBED_FAST=1(INT8). SetLEANKG_EMBED_FAST=0/LEANKG_EMBED_MODEL=bgefor legacy FP32.Related Issues
Embed DB path mismatch between indexer and CLI; cold-embed SLA / MCP boot blocking.
Additional Context
Fast-path recipe:
Healthy logs:
kind=bge-int8,max_seq=128, Xenovaonnx/model_quantized.onnx— not Qdrantmodel_optimized.onnx.