fix(export): restore graph.html for large graphs - #2894
Conversation
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Formal verification. 1 change(s) alter behavior, breaking input(s) attached.
Behavior changes: to\_html changes behavior, here is the input that shows it.
The verifier found a concrete input on which to\_html behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"G":"\(lambda \_g: \(\_g\.add\_nodes\_from\(\[\(1, \{\}\), \(2, \{\}\), \(3, \{\}\)\]\), \_g\.add\_edges\_from\(\[\(1, 2, \{\}\), \(1, 3, \{\}\), \(2, 3, \{\}\)\]\), \_g\)\[\-1\]\)\(\_\_import\_\_\('networkx'\)\.Graph\(\)\)","communities":"\{'k': 'v'\}","output\_path":"'racecar'","community\_labels":"\{'k': 'v'\}","member\_counts":"\{'a': 1\}","node\_limit":"2","learning\_overlay":"\{'a': 1, 'b…, the old code produced None but the new code produces False. Paste that input straight into a regression test.
Graphify review — findings
Adds a stale-HTML marker (.graph.html.stale) protocol so graphify/exporters/html.py's to_html writes atomically via write_text_atomic and returns a bool signaling whether output was produced, and threads that signal through cli.py's viz path and a new _reconcile_graph_html in watch.py that regenerates missing/stale/disabled HTML on the unchanged-topology update path. Drops the old _over_cap-driven node_limit=5000 fallback in cli.py in favor of _viz_node_limit(), including a <=0 opt-out that removes the HTML. Adds watch/labeling/CLI/html tests covering the marker lifecycle, node-limit policy, and reconcile behavior.
Worth a look
- Stale HTML marker left behind when to_html raises non-ValueError exception —
graphify/cli.py:2101· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review Execution auto-disposal is off for this run; enable it (with sandbox isolation) to have Graphify try to confirm or refute this automatically.
- Optional HTML recovery aborts if stale marker cannot be created —
graphify/watch.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review Execution auto-disposal is off for this run; enable it (with sandbox isolation) to have Graphify try to confirm or refute this automatically.
- Unchanged-topology rebuild can now delete existing graph.html when HTML is disabled —
graphify/watch.py:1018· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review Execution auto-disposal is off for this run; enable it (with sandbox isolation) to have Graphify try to confirm or refute this automatically.
- Unexpected HTML export failures no longer propagate from rebuild —
graphify/watch.py:1043· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review Execution auto-disposal is off for this run; enable it (with sandbox isolation) to have Graphify try to confirm or refute this automatically.
- Pre-advance HTML stale marker can be cleared by a concurrent reconcile against the old graph —
graphify/watch.py:1807· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review Execution auto-disposal is off for this run; enable it (with sandbox isolation) to have Graphify try to confirm or refute this automatically.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 686 functions depend on the 474 functions this change touches.
Health — this change adds coupling hotspots:
- new:
_rebuild_code()— 98 callers, 50 callees - new:
dispatch_command()— 2 callers, 118 callees - new:
to_html()— 17 callers, 11 callees - new:
run_pipeline()— 8 callers, 13 callees - new:
_stale_graph_sources()— 7 callers, 6 callees - new:
watch()— 5 callers, 7 callees - new:
_run_hook_guard()— 4 callers, 7 callees - new:
_reconcile_graph_html()— 5 callers, 4 callees - …and 2 more — each is listed as a finding
Verification — 686 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 643 function(s) in the blast radius were not formally verified this run
Formal verification
Behavior changes: to\_html changes behavior, here is the input that shows it.
The verifier found a concrete input on which to\_html behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"G":"\(lambda \_g: \(\_g\.add\_nodes\_from\(\[\(1, \{\}\), \(2, \{\}\), \(3, \{\}\)\]\), \_g\.add\_edges\_from\(\[\(1, 2, \{\}\), \(1, 3, \{\}\), \(2, 3, \{\}\)\]\), \_g\)\[\-1\]\)\(\_\_import\_\_\('networkx'\)\.Graph\(\)\)","communities":"\{'k': 'v'\}","output\_path":"'racecar'","community\_labels":"\{'k': 'v'\}","member\_counts":"\{'a': 1\}","node\_limit":"2","learning\_overlay":"\{'a': 1, 'b…, the old code produced None but the new code produces False. Paste that input straight into a regression test.
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_rebuild\_code.
The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `watch_path` is annotated `Path` — outside the synthesizable primitive/collection set
· 2 grounded finding(s) anchored inline below; 8 more finding(s) on lines outside this diff (see the check run).
| return False | ||
|
|
||
|
|
||
| def _reconcile_graph_html(out: Path, graph_data: dict) -> str | None: |
There was a problem hiding this comment.
_reconcile_graph_html()
high coupling complexity (Ca·Ce = 20).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return "rendered" | ||
|
|
||
|
|
||
| def _rebuild_code( |
There was a problem hiding this comment.
_rebuild_code()
fans out to 50 callees (efferent coupling); 98 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
abde2e1 to
ae86650
Compare
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Formal verification. 1 change(s) alter behavior, breaking input(s) attached.
Behavior changes: to\_html changes behavior, here is the input that shows it.
The verifier found a concrete input on which to\_html behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"G":"\(lambda \_g: \(\_g\.add\_nodes\_from\(\[\(1, \{\}\), \(2, \{\}\), \(3, \{\}\)\]\), \_g\.add\_edges\_from\(\[\(1, 2, \{\}\), \(1, 3, \{\}\), \(2, 3, \{\}\)\]\), \_g\)\[\-1\]\)\(\_\_import\_\_\('networkx'\)\.Graph\(\)\)","communities":"\{'k': 'v'\}","output\_path":"'racecar'","community\_labels":"\{'k': 'v'\}","member\_counts":"\{'a': 1\}","node\_limit":"2","learning\_overlay":"\{'a': 1, 'b…, the old code produced None but the new code produces False. Paste that input straight into a regression test.
Graphify review — findings
Adds a .graph.html.stale marker written before graph.json advances and cleared only on a successful render, so dispatch_command never leaves an older graph.html looking current after interruption or a failed viz step. Makes to_html write atomically and return a bool signaling whether output was produced, replacing the old _over_cap node-limit fallback with a single _viz_node_limit()-driven policy (including =0 to disable HTML). Adds _reconcile_graph_html to watch.py to rebuild or remove HTML on the unchanged-topology update path when it's missing, stale, or disabled.
No blocking issues surfaced. 19 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 695 functions depend on the 483 functions this change touches.
Health — this change adds coupling hotspots:
- new:
_rebuild_code()— 98 callers, 50 callees - new:
dispatch_command()— 2 callers, 119 callees - new:
to_html()— 17 callers, 11 callees - new:
run_pipeline()— 8 callers, 13 callees - new:
_stale_graph_sources()— 7 callers, 6 callees - new:
watch()— 5 callers, 7 callees - new:
_reconcile_graph_html()— 6 callers, 5 callees - new:
_run_hook_guard()— 4 callers, 7 callees - …and 2 more — each is listed as a finding
Verification — 695 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 652 function(s) in the blast radius were not formally verified this run
Formal verification
Behavior changes: to\_html changes behavior, here is the input that shows it.
The verifier found a concrete input on which to\_html behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"G":"\(lambda \_g: \(\_g\.add\_nodes\_from\(\[\(1, \{\}\), \(2, \{\}\), \(3, \{\}\)\]\), \_g\.add\_edges\_from\(\[\(1, 2, \{\}\), \(1, 3, \{\}\), \(2, 3, \{\}\)\]\), \_g\)\[\-1\]\)\(\_\_import\_\_\('networkx'\)\.Graph\(\)\)","communities":"\{'k': 'v'\}","output\_path":"'racecar'","community\_labels":"\{'k': 'v'\}","member\_counts":"\{'a': 1\}","node\_limit":"2","learning\_overlay":"\{'a': 1, 'b…, the old code produced None but the new code produces False. Paste that input straight into a regression test.
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_rebuild\_code.
The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `watch_path` is annotated `Path` — outside the synthesizable primitive/collection set
· 2 grounded finding(s) anchored inline below; 8 more finding(s) on lines outside this diff (see the check run).
| return False | ||
|
|
||
|
|
||
| def _reconcile_graph_html(out: Path, graph_data: dict) -> str | None: |
There was a problem hiding this comment.
_reconcile_graph_html()
6 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return "rendered" | ||
|
|
||
|
|
||
| def _rebuild_code( |
There was a problem hiding this comment.
_rebuild_code()
fans out to 50 callees (efferent coupling); 98 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
Shipped in v0.9.48 via authorship-preserving cherry-pick. Thanks @oleksii-tumanov! Release: https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.48 |
2 participants
Summary
label,cluster-only, andupdate, producing the existing aggregated community view for large graphs.graph.htmlduring an unchangedupdatewithout reclustering.0still disables HTML generation.Before and after
Before:
labelandcluster-onlyremovedgraph.htmlwhen the graph exceeded the visualization limit.updatereported that outputs were untouched and did not restore the missing file.After:
label,cluster-only, andupdategenerate the aggregated community view above the limit.updateregenerates missing or stalegraph.html.Result
On the final patch,
graphify update .produced:Testing
pytest tests/test_atomic_writes.py tests/test_export.py tests/test_labeling.py tests/test_watch.py -q(231 passed, 3 skipped)pytest tests/ -q --ignore=tests/test_non_regular_files.py --ignore=tests/test_pipeline.py --ignore=tests/test_security.py(4640 passed, 52 skipped); excluded environment-dependent suites passed separately (77 passed)ruff check .and the requiredpython -m tools.skillgenvalidation commands (passed)Fixes #2853