fix(llm): hoist the covered-files set out of the uncovered-files loop

[sub4biz]

Summary

extract_corpus_parallel's uncovered-files reconciliation (graphify/llm.py, ~line 2508-2517) rebuilds
{c.resolve() for c in covered} from scratch inside the generator passed to sorted(), once per item in
dispatched:

uncovered = sorted(
    p for p in dispatched
    if p.resolve() not in {c.resolve() for c in covered}   # rebuilt every iteration
)

That's O(len(dispatched) × len(covered)) Path.resolve() calls (each a real realpath syscall) instead
of the obviously-intended O(len(dispatched) + len(covered)). This PR hoists the set comprehension out of the
loop so it's computed once.

Impact

Found on a real ~23,000-file semantic-extraction run: this single reconciliation step sat in this exact call
chain continuously for over an hour (confirmed live via py-spy dump sampling — same stack frame,
climbing CPU time, no progress), before the run was interrupted rather than left to finish. The interrupt
traceback pointed at this exact line, independently confirming the py-spy diagnosis.

Measured

N=1000 (dispatched=covered=1000, scaled down from the real ~23,000 for a fast comparison):
  old (rebuild per item):  76.85s
  new (precomputed once):   0.25s   (~312x)

Quadratic growth means the real corpus size (~23x larger) projects to roughly 23² ≈ 529x worse than the
N=1000 old-code number — on the order of 11+ hours for this one step alone before this fix.

Fix

covered_resolved = {c.resolve() for c in covered}
uncovered = sorted(
    p for p in dispatched
    if p.resolve() not in covered_resolved
)

Pure hoist of a loop-invariant computation — verified the result set is identical to the old code via an
assert in the reproduction script (see commit message). No behavior change, no API change.

Testing

• Existing tests/test_chunking.py coverage for this exact code path (checkpoint scoping, out-of-scope node
  dropping, uncovered_files detection/warning) passes unchanged.
• Full existing test suite passes with no new failures.

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. No changes could be formally verified in this run.

---

Graphify review — findings

Hoists the covered set resolution out of the uncovered generator in extract_corpus_parallel, so resolve() runs once per entry instead of rebuilding the resolved set for every dispatched item. Fixes the O(dispatched × covered) blowup that stalled large-corpus runs for hours.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 675 functions depend on the 155 functions this change touches.

Health — this change adds coupling hotspots:

• new: deduplicate_entities() — 63 callers, 20 callees
• new: build_merge() — 43 callers, 14 callees
• new: extract_files_direct() — 17 callers, 20 callees
• new: _call_claude_cli() — 31 callers, 9 callees
• new: extract_corpus_parallel() — 26 callers, 10 callees
• new: dispatch_command() — 2 callers, 117 callees
• new: _call_llm() — 11 callers, 17 callees
• new: _call_openai_compat() — 23 callers, 8 callees
• …and 14 more — each is listed as a finding

Verification — 675 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 422 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract\_corpus\_parallel.

The verifier did not have enough to check extract\_corpus\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

· 22 more finding(s) on lines outside this diff (see the check run).

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

Hoists the covered resolution out of the uncovered generator in extract_corpus_parallel, precomputing _covered_resolved once instead of rebuilding the resolved set per dispatched file. Cuts resolve() syscalls from O(dispatched × covered) to O(dispatched + covered), fixing a multi-hour hang on large corpora.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 675 functions depend on the 155 functions this change touches.

Health — this change adds coupling hotspots:

• new: deduplicate_entities() — 63 callers, 20 callees
• new: build_merge() — 43 callers, 14 callees
• new: extract_files_direct() — 17 callers, 20 callees
• new: _call_claude_cli() — 31 callers, 9 callees
• new: extract_corpus_parallel() — 26 callers, 10 callees
• new: dispatch_command() — 2 callers, 117 callees
• new: _call_llm() — 11 callers, 17 callees
• new: _call_openai_compat() — 23 callers, 8 callees
• …and 14 more — each is listed as a finding

Verification — 675 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 422 function(s) in the blast radius were not formally verified this run

· 22 more finding(s) on lines outside this diff (see the check run).