fix(extract): don't classify deliberately-declined data JSON as failed (#2879) - #2885
fix(extract): don't classify deliberately-declined data JSON as failed (#2879)#2885rajarshidattapy wants to merge 1 commit into
Conversation
Graphify-Labs#2879) The JSON extractor declines data-shaped JSON by design (Graphify-Labs#1224) and already reports it as `{"nodes": [], "edges": [], "skipped": ...}`, but nothing consumed that marker. Because a `.json` extractor *is* registered, every declined file satisfied both halves of the failed-source test in extract() — zero nodes and an extractor exists — so it was returned in `failed_sources`, never stamped as processed in the incremental manifest, and re-queued on every subsequent run indefinitely. Honour the existing `skipped` marker at both call sites: it is exempt from `failed_sources` and from the Graphify-Labs#1666 "produced zero nodes" warning. A result that is empty without the marker is still treated as a failure.
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Treats extractor-declined files (those returning a skipped marker) as an intended outcome in extract() rather than a failure — they no longer get counted as empty sources, emit "zero nodes" warnings, or land in failed_sources (which previously re-queued them on every incremental run). Adds tests covering a declined data JSON and a guard that genuinely empty JSON is still reported as failed.
No blocking issues surfaced.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 1730 functions depend on the 563 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 473 callers, 42 callees - new:
_rebuild_code()— 98 callers, 51 callees - new:
extract_xaml()— 19 callers, 17 callees - new:
extract_js()— 80 callers, 3 callees - new:
dispatch_command()— 2 callers, 117 callees - new:
_get_extractor()— 26 callers, 6 callees - new:
run_pipeline()— 8 callers, 13 callees - new:
collect_files()— 17 callers, 6 callees - …and 22 more — each is listed as a finding
Verification — 1730 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1587 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify extract.
The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set
· 30 more finding(s) on lines outside this diff (see the check run).
|
Shipped in v0.9.48 via authorship-preserving cherry-pick so you keep contributor-graph credit. Thanks @rajarshidattapy! Release: https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.48 |
Fixes #2879.
The bug
extract()'s empty-result classifier could not tell "the extractor declined this file on purpose" from "extraction failed":Data-shaped JSON is deliberately not AST-walked (#1224 — it is left to the LLM semantic pass), but a
.jsonextractor is registered, so a declined file satisfies both halves of that test. It landed infailed_sources, whose stated purpose is to keep those paths out of the processed set, so the incremental manifest never stamped it — and it was re-queued on every subsequent run, forever. On the reporter's 3,329-file repo that was 110 files with a blankast_hash, all of them.json.The fix
extract_json()already reports the decline explicitly:That marker just had no consumer. This honours it at the two call sites that needed it:
failed_sources(Incremental manifest never retries files whose extractor failed on a missing extra #2543) skips results carryingskipped, so the CLI stamps them as processed;This is option (1) from the issue's Suggested direction, and needs no new marker —
_get_extractor()stays pure dispatch-by-extension.An empty result without the marker is still classified as failed, so the #1666/#2543 safety net is unchanged for genuine failures.
Tests
test_extract_declined_data_json_is_not_failed— a datameta.jsonyields emptyfailed_sourcesand no zero-nodes warning, while a siblingpackage.jsonstill extracts. Fails without the change.test_extract_genuinely_empty_json_still_failed— an extractor returning an empty result with no marker is still reported failed.