Skip to content

fix(extract): don't classify deliberately-declined data JSON as failed (#2879) - #2885

Closed
rajarshidattapy wants to merge 1 commit into
Graphify-Labs:v8from
rajarshidattapy:fix/declined-json-not-failed
Closed

fix(extract): don't classify deliberately-declined data JSON as failed (#2879)#2885
rajarshidattapy wants to merge 1 commit into
Graphify-Labs:v8from
rajarshidattapy:fix/declined-json-not-failed

Conversation

@rajarshidattapy

Copy link
Copy Markdown
Contributor

Fixes #2879.

The bug

extract()'s empty-result classifier could not tell "the extractor declined this file on purpose" from "extraction failed":

if (not _res.get("nodes")) and _get_extractor(_p) is not None:
    _failed_sources.append(_key)

Data-shaped JSON is deliberately not AST-walked (#1224 — it is left to the LLM semantic pass), but a .json extractor is registered, so a declined file satisfies both halves of that test. It landed in failed_sources, whose stated purpose is to keep those paths out of the processed set, so the incremental manifest never stamped it — and it was re-queued on every subsequent run, forever. On the reporter's 3,329-file repo that was 110 files with a blank ast_hash, all of them .json.

The fix

extract_json() already reports the decline explicitly:

return {"nodes": [], "edges": [], "skipped": "data json (not a config/manifest)"}

That marker just had no consumer. This honours it at the two call sites that needed it:

This is option (1) from the issue's Suggested direction, and needs no new marker — _get_extractor() stays pure dispatch-by-extension.

An empty result without the marker is still classified as failed, so the #1666/#2543 safety net is unchanged for genuine failures.

Tests

  • test_extract_declined_data_json_is_not_failed — a data meta.json yields empty failed_sources and no zero-nodes warning, while a sibling package.json still extracts. Fails without the change.
  • test_extract_genuinely_empty_json_still_failed — an extractor returning an empty result with no marker is still reported failed.

Graphify-Labs#2879)

The JSON extractor declines data-shaped JSON by design (Graphify-Labs#1224) and already
reports it as `{"nodes": [], "edges": [], "skipped": ...}`, but nothing
consumed that marker. Because a `.json` extractor *is* registered, every
declined file satisfied both halves of the failed-source test in extract()
— zero nodes and an extractor exists — so it was returned in
`failed_sources`, never stamped as processed in the incremental manifest,
and re-queued on every subsequent run indefinitely.

Honour the existing `skipped` marker at both call sites: it is exempt from
`failed_sources` and from the Graphify-Labs#1666 "produced zero nodes" warning. A result
that is empty without the marker is still treated as a failure.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

Treats extractor-declined files (those returning a skipped marker) as an intended outcome in extract() rather than a failure — they no longer get counted as empty sources, emit "zero nodes" warnings, or land in failed_sources (which previously re-queued them on every incremental run). Adds tests covering a declined data JSON and a guard that genuinely empty JSON is still reported as failed.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1730 functions depend on the 563 functions this change touches.

Health — this change adds coupling hotspots:

  • new: extract() — 473 callers, 42 callees
  • new: _rebuild_code() — 98 callers, 51 callees
  • new: extract_xaml() — 19 callers, 17 callees
  • new: extract_js() — 80 callers, 3 callees
  • new: dispatch_command() — 2 callers, 117 callees
  • new: _get_extractor() — 26 callers, 6 callees
  • new: run_pipeline() — 8 callers, 13 callees
  • new: collect_files() — 17 callers, 6 callees
  • …and 22 more — each is listed as a finding

Verification — 1730 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 1587 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract.

The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set

· 30 more finding(s) on lines outside this diff (see the check run).

@safishamsi

Copy link
Copy Markdown
Collaborator

Shipped in v0.9.48 via authorship-preserving cherry-pick so you keep contributor-graph credit. Thanks @rajarshidattapy! Release: https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.48

@safishamsi safishamsi closed this Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deliberately-skipped data JSON (#1224) is classified as failed extraction, so it is re-queued on every incremental run

2 participants