Skip to content

fix(mcp): serialize ADR access across processes#892

Open
SS-42 wants to merge 1 commit into
DeusData:mainfrom
SS-42:fix/adr-cross-process-lock
Open

fix(mcp): serialize ADR access across processes#892
SS-42 wants to merge 1 commit into
DeusData:mainfrom
SS-42:fix/adr-cross-process-lock

Conversation

@SS-42

@SS-42 SS-42 commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add project-scoped cache-local POSIX advisory locks for pipeline rebuilds shared by independent MCP/CLI processes using the same CBM_CACHE_DIR
  • keep rebuild/ADR/delete serialization for the same project without blocking unrelated project indexes
  • avoid watcher self-locking when a supervised worker performs the reindex
  • serialize manage_adr reads, writes, and legacy migration with same-project rebuilds, and reopen stale file-backed stores before ADR access

Environment observed

  • macOS arm64
  • multiple independent MCP/CLI processes using one shared CBM_CACHE_DIR

Repro covered by tests

  • one process holding a project lock makes another process try_lock report busy for the same project
  • distinct projects can acquire locks independently across processes
  • a blocking same-project lock in another process waits until the first process releases it
  • lock backend open failures release the process-local lock
  • manage_adr(update) cannot write while another process owns the same-project rebuild lock

Verification

  • git diff --check
  • ASAN_OPTIONS=detect_leaks=0 build/c/test-runner pipeline mcp (360 passed)
  • scripts/test.sh CC=clang CXX=clang++ (5920 passed, 1 skipped; watchdog regressions passed; security allow-list passed)
  • scripts/build.sh --version 0.8.1-adrfix.20260706a CC=clang CXX=clang++

@SS-42 SS-42 force-pushed the fix/adr-cross-process-lock branch 4 times, most recently from 42b1878 to c52e266 Compare July 5, 2026 23:20
@SS-42 SS-42 marked this pull request as ready for review July 6, 2026 00:47
@SS-42 SS-42 requested a review from DeusData as a code owner July 6, 2026 00:47
@SS-42 SS-42 force-pushed the fix/adr-cross-process-lock branch from c52e266 to ba39ffd Compare July 6, 2026 13:53
@DeusData DeusData added bug Something isn't working stability/performance Server crashes, OOM, hangs, high CPU/memory priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. labels Jul 7, 2026
@DeusData

DeusData commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Thanks for the process-serialization work. Triage: high-priority stability bug.

Review will focus on lock scope and failure behavior: same-project serialization without blocking unrelated projects, no watcher/self-lock deadlocks, safe stale-store reopen behavior, and ADR migration/write ordering across independent MCP/CLI processes.

@DeusData DeusData added this to the 0.9.1-rc milestone Jul 8, 2026
@DeusData

DeusData commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Thanks — this addresses a real gap (independent MCP/CLI processes sharing one CBM_CACHE_DIR had no cross-process serialization), and the lock hygiene is right: 0600 lock files under the cache dir, FD_CLOEXEC, unlock-before-close, per-project scoping so unrelated indexes don't serialize, and project names are sanitized upstream so the lock filename is traversal-safe. The fork-based cross-process tests are a nice touch. Two changes requested before merge:

  1. Vacuous pass in test setup (tests/test_mcp.c, the new lock tests): if (!cbm_mkdtemp(cache)) { PASS(); } turns a setup failure into a green test. Our suite treats that as a broken guard — a test that cannot establish its preconditions has FAILED, not passed (see the FAIL() macro doc in test_framework.h). Please use FAIL("mkdtemp failed") there (and in any sibling occurrences in the new tests).

  2. Windows is a silent no-opcbm_pipeline_try_lock_project returns true unconditionally under _WIN32, so Windows keeps today's cross-process races. That's a fine incremental scope (no regression), but please make it explicit: a comment at the #ifdef stating the limitation (+ ideally a one-line debug log), so the next Windows-locking pass knows this is intentionally unimplemented rather than accidentally missing. If you're up for it, LockFileEx on the same lock-file path is the natural follow-up — separate PR is fine.

Everything else looks merge-ready: 19/19 checks green, clean mergeable state against current main (impressive, given how much mcp.c/pipeline.c moved this week), scope matches the description exactly, no stray files.

Use project-scoped cache-local advisory locks so independent MCP and CLI processes sharing CBM_CACHE_DIR cannot rebuild the same project DB concurrently without blocking unrelated project indexes.

Serialize manage_adr reads and writes with same-project rebuilds, reopen stale file-backed stores before ADR access, and avoid watcher self-locking when a supervised worker performs the reindex.

Cover the cross-process lock behavior with fork-based regression tests.

Signed-off-by: SS-42 <noreply@incogni.to>
@SS-42 SS-42 force-pushed the fix/adr-cross-process-lock branch from ba39ffd to 0a8b63e Compare July 9, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. stability/performance Server crashes, OOM, hangs, high CPU/memory

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants