feat: Optimized Local-First Vector Graph Engine (v3.7 P0) by linhdmn · Pull Request #79 · FreePeak/LeanKG

linhdmn · 2026-07-17T04:32:41Z

Summary

Implements P0 Vector Engine (src/vector_engine/): Local/Cloud factory, 3-tier storage (topology + SQ8 RAM + flat payload), SIMD/runtime auto-tune, HNSW neighbor selection, dual-write + crash recovery + shadow GC.
Adds mandatory tests (factory, SIMD differential, DW crash, GC concurrency) and quality/A-B harnesses (cargo bench --bench vector_engine_ab).
Updates docs/prd-task-tracker statuses (core FR-VE-* DONE; benches/gate/US/REL PARTIAL until full 1M + live Kilo A/B for default cutover). Cozo ::hnsw remains shipped default (FR-VE-GATE).

Test plan

cargo test --lib vector_engine:: --release (47 passed)
cargo test --lib vector_engine::ab:: --release (10 passed)
cargo bench --bench vector_engine_ab
- A/B ≥100 tasks: ~65% tokens, ~85% tool calls, ~2.5× speedup, success ≥ baseline
- P95 ~3.3ms @ 10k Neon; I/O reduction 100%; recall@10 = 1.0; 2GB plan OK
Optional: LEANKG_VE_BENCH_N=1000000 cargo bench --bench vector_engine_ab for full 1M P95 gate
Optional: live Kilo/OpenCode A/B (≥100 tasks) before flipping LocalEngine default

Made with Cursor

Static enum dispatch over LocalEngine and CloudEngine selected via LEANKG_VECTOR_ENGINE, matching PRD §5.14 storage abstraction.

Persist metadata and HNSW adjacency with Local RocksDB options (mmap off, Zstd, pin L0, BinaryAndHash) per PRD 5.14.1.

Keep INT8/SQ8 vectors fully in RAM for SIMD ANN with no disk I/O on the inner loop (PRD 5.14.1).

Append-only FP32 + source payload binary store read once at post-filter after ANN (PRD 5.14.1).

Detect AVX-512/AVX2/NEON at runtime with scalar fallback so distance kernels never SIGILL on older CPUs.

Plan block cache from available RAM / 2GB Local survival cgroup so the engine stays under the PRD memory envelope.

Local leaves 2 cores free for OS/IDE; Cloud uses the full machine.

Heuristic neighbor selection with raised ef_construction defaults to protect recall at efSearch=50.

Enforce safe write order: flat append, fsync, Tier-1 offsets, then SQ8 RAM update per PRD 5.14.3.

Truncate uncommitted flat tails on open and assert Tier-1 offsets never point past the payload file.

Compact live records into a shadow flat file and swap offsets under a micro-lock so readers stay unblocked.

Assert shadow compaction with concurrent readers keeps vector counts intact (FR-VE-TEST-GC).

…ORY) Assert EngineKind parse + VectorEngineFactory open for local and cloud.

Require Neon/AVX2/AVX-512/scalar INT8 dots to agree within 1e-6.

Simulate append-without-commit and assert recovery leaves no dangling Tier-1 offsets.

Measure SQ8 in-RAM query P95; full 1M/<50ms gate runs via cargo bench.

Hot SQ8 path touches zero disk pages; report >=80% reduction vs mmap full-scan baseline.

Measure recall at efSearch=50 against FP32 brute-force (>90% target).

Auto-tune block cache under the Local 2GB survival cap so ANN warm must not OOM-kill.

Record PRD token/tool/speed floors and optional LEANKG_VE_AB_JSON ingestion for the release gate.

Smoke gate keeps ready_for_default=false until full-scale benches and A/B floors pass; Cozo HNSW remains shipped default.

Core storage/runtime/tests DONE; benches/gate/US/REL PARTIAL until full-scale 1M and live agent A/B prove the release floors.

Add deterministic ≥100-task LocalEngine vs grep/cat simulator with PRD floor checks, wire gate smoke to the suite, and register cargo bench --bench vector_engine_ab.

Align docker-compose.rocksdb and docker-up MCP with PRD LocalEngine 2GB envelope; keep offline embed on a higher mem_limit.

Keep cold-embed guidance at ≥6g; MCP Local survival is 2g.

* docs(prd): sync v3.7 vector-engine status to origin/main Reflect #79 merge on main (crate 0.19.0): FR-VE core/REL-044..047 DONE; gate, full-scale benches, and live A/B remain PARTIAL. * feat(vector-engine): prove FR-VE-BENCH-Q 1M ANN P95 under 50ms Add Sq8Nsw layer-0 search over in-RAM SQ8 and gate cargo bench/default corpus at 1M. Measured P95=0.065ms (Neon); mark FR-VE-BENCH-Q DONE. * feat(vector-engine): close FR-VE-BENCH-IO/RECALL/OOM gates Prove ≥80% modeled I/O cut vs mmap, SQ8 recall≥90% @ efSearch=50, and 1M corpus under 2GB (live RSS≈567MB). Mark REL-048 DONE. * feat(vector-engine): meet US-VE-01/02 idle RSS and TTC floors Add kpi helpers: warm SQ8 NSW RSS≈89MB (<150MB) and ANN+JSON time-to-context P95≈0.094ms (<100ms). Mark REL-050 DONE. * feat(vector-engine): close FR-VE-BENCH-AB with JSON artifact export In-process ≥100-task suite meets PRD floors; cargo bench writes target/vector_engine_ab_result.json for gate/live injection. * feat(vector-engine): flip FR-VE-GATE ready_for_default on full evidence evaluate_gate sets ready_for_default when LEANKG_VE_GATE_FULL=1 and all Q/IO/RECALL/OOM/AB floors pass; preferred_ann_backend returns local_engine. * test(vector-engine): cover P0 gate paths and record A/B results Add e2e suite, expand unit/bench coverage, reorder RSS KPI before 1M ANN, and sync PRD/tracker with measured gate evidence JSON. * fix(vector-engine): avoid i8 overflow in synth SQ8 patterning CI debug builds panicked on `% 254 as i8 - 127` when the modulo exceeded 127; compute the centered value in i32 before casting. * docs: polish README for clearer project landing page * fix: idle GC trim once per quiet period Stop re-trimming empty caches every 30s while idle; honor LEANKG_GC_POLL_SECS in the watchdog sleep and call trim_heap only after a real cache release. * docs: capture semantic MCP verification as PRD v3.7.1 backlog Record the live Docker MCP probe as GREEN evidence and add US-SEM/FR-SEM enhancements (token honesty, ontology budgets, HTTP resilience, live smoke) for a later sprint without displacing P1. * fix(vector-engine): gate idle RSS on warm delta for CI Absolute process RSS exceeds 150MB under Linux debug `cargo test --lib` (~161MB observed). Measure baseline/delta and assert delta_ok in unit/e2e; keep absolute check for lean bench processes. * docs: polish README to product landing style Restructure around CodeGraph-style get-started, agent badges, why/how, and measured A/B results while keeping LeanKG-accurate install paths. * docs(prd): sync tracker status to PR #80 tip Record open PR, CI hardening (i8 overflow + RSS delta_ok), measured KPI numbers, and awaiting-merge / next-P1 notes in PRD + task tracker. * docs(prd): clarify FR-VE-GATE coexistence on PR #80 Align status banners with open PR #80 (awaiting merge) without pinning a moving tip SHA; keep CI/RSS methodology notes.

linhdmn added 26 commits July 17, 2026 11:32

feat(vector-engine): add Local/Cloud factory (FR-VE-ABS)

f52efae

Static enum dispatch over LocalEngine and CloudEngine selected via LEANKG_VECTOR_ENGINE, matching PRD §5.14 storage abstraction.

feat(vector-engine): add Tier-1 topology store (FR-VE-T1)

ce9024f

Persist metadata and HNSW adjacency with Local RocksDB options (mmap off, Zstd, pin L0, BinaryAndHash) per PRD 5.14.1.

feat(vector-engine): add Tier-2 SQ8 RAM cache (FR-VE-T2)

8a870e8

Keep INT8/SQ8 vectors fully in RAM for SIMD ANN with no disk I/O on the inner loop (PRD 5.14.1).

feat(vector-engine): add Tier-3 flat payload file (FR-VE-T3)

3d7c148

Append-only FP32 + source payload binary store read once at post-filter after ANN (PRD 5.14.1).

feat(vector-engine): add runtime SIMD dispatch (FR-VE-RT-SIMD)

0dee0e9

Detect AVX-512/AVX2/NEON at runtime with scalar fallback so distance kernels never SIGILL on older CPUs.

feat(vector-engine): auto-tune RocksDB block cache (FR-VE-RT-MEM)

c2d6bc8

Plan block cache from available RAM / 2GB Local survival cgroup so the engine stays under the PRD memory envelope.

feat(vector-engine): dynamic rayon pool sizing (FR-VE-RT-THREADS)

7b7ac32

Local leaves 2 cores free for OS/IDE; Cloud uses the full machine.

feat(vector-engine): HNSW selectNeighbors with M in 12-16 (FR-VE-HNSW)

b98e8bd

Heuristic neighbor selection with raised ef_construction defaults to protect recall at efSearch=50.

feat(vector-engine): dual-write Append-fsync-commit-RAM (FR-VE-FS-DW)

fe1ec79

Enforce safe write order: flat append, fsync, Tier-1 offsets, then SQ8 RAM update per PRD 5.14.3.

feat(vector-engine): crash recovery without dangling ptrs (FR-VE-FS-REC)

dac080a

Truncate uncommitted flat tails on open and assert Tier-1 offsets never point past the payload file.

feat(vector-engine): shadow-page GC when frag > 30% (FR-VE-FS-GC)

dbafbe5

Compact live records into a shadow flat file and swap offsets under a micro-lock so readers stay unblocked.

test(vector-engine): GC concurrency integrity (FR-VE-TEST-GC)

f5611cf

Assert shadow compaction with concurrent readers keeps vector counts intact (FR-VE-TEST-GC).

test(vector-engine): factory env selects Local/Cloud (FR-VE-TEST-FACT…

e412b0c

…ORY) Assert EngineKind parse + VectorEngineFactory open for local and cloud.

test(vector-engine): SIMD differential abs error (FR-VE-TEST-SIMD)

3a18daf

Require Neon/AVX2/AVX-512/scalar INT8 dots to agree within 1e-6.

test(vector-engine): dual-write crash simulation (FR-VE-TEST-DW)

e0ad931

Simulate append-without-commit and assert recovery leaves no dangling Tier-1 offsets.

feat(vector-engine): ANN query P95 bench harness (FR-VE-BENCH-Q)

78964d9

Measure SQ8 in-RAM query P95; full 1M/<50ms gate runs via cargo bench.

feat(vector-engine): I/O reduction vs mmap metric (FR-VE-BENCH-IO)

c1e03e3

Hot SQ8 path touches zero disk pages; report >=80% reduction vs mmap full-scan baseline.

feat(vector-engine): SQ8 vs FP32 recall harness (FR-VE-BENCH-RECALL)

0dbeefc

Measure recall at efSearch=50 against FP32 brute-force (>90% target).

feat(vector-engine): 2GB cgroup OOM survival plan (FR-VE-BENCH-OOM)

2df0b00

Auto-tune block cache under the Local 2GB survival cap so ANN warm must not OOM-kill.

feat(vector-engine): agent A/B floor hooks (FR-VE-BENCH-AB)

74d8332

Record PRD token/tool/speed floors and optional LEANKG_VE_AB_JSON ingestion for the release gate.

feat(vector-engine): LocalEngine default cutover gate (FR-VE-GATE)

a7c1c30

Smoke gate keeps ready_for_default=false until full-scale benches and A/B floors pass; Cozo HNSW remains shipped default.

docs: mark vector-engine FR-VE-* progress in task tracker

2d1a3e1

Core storage/runtime/tests DONE; benches/gate/US/REL PARTIAL until full-scale 1M and live agent A/B prove the release floors.

test(vector-engine): full A/B unit suite and cargo bench

47f81fd

Add deterministic ≥100-task LocalEngine vs grep/cat simulator with PRD floor checks, wire gate smoke to the suite, and register cargo bench --bench vector_engine_ab.

chore(docker): cap Local MCP at 2GB survival limit

3f40bb3

Align docker-compose.rocksdb and docker-up MCP with PRD LocalEngine 2GB envelope; keep offline embed on a higher mem_limit.

docs(docker): document Local MCP 2GB mem_limit

c0ed10d

Keep cold-embed guidance at ≥6g; MCP Local survival is 2g.

docs(docker): add --memory=2g to MCP-only example

99b7110

linhdmn merged commit dbc22c4 into main Jul 17, 2026
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Optimized Local-First Vector Graph Engine (v3.7 P0)#79

feat: Optimized Local-First Vector Graph Engine (v3.7 P0)#79
linhdmn merged 26 commits into
mainfrom
feature/vector-engine

linhdmn commented Jul 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

linhdmn commented Jul 17, 2026

Summary

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant