fix(pipeline): stable USAGE-edge ownership for directory-module languages (#787)#828
fix(pipeline): stable USAGE-edge ownership for directory-module languages (#787)#828spde wants to merge 2 commits into
Conversation
…rce misattribution across same-package files Three cases in the reproduce-first suite: exact distinct-source-file set for a class-level type reference (sequential path), stability of that set across 5 independent indexes, and the same assertions on a >50-file fixture that takes the parallel worker/merge path (where the collapse varies run to run). Set semantics mirror the spring-petclinic oracle: OwnerRepository must have exactly 7 distinct USAGE user files.
…ages (DeusData#787) Since the directory-based Java/Go module QNs (DeusData#667), every file in a package shares its module QN with the package's Folder node (or the Project node for root files). That collision corrupted USAGE/CALLS source attribution in three places: - cbm_gbuf_upsert_node kept the Folder/Project label on a colliding per-file Module def but still clobbered name/file_path/range in place, so the shared node's file_path became whichever file was processed last. - merge_update_existing (parallel worker-gbuf merge) applied unconditional "src wins", even relabelling the Folder node to Module — and worker merge order varies per run, which is the reported run-to-run nondeterminism. - The source-node finders (pass_usages/pass_parallel/pass_calls) looked up a class-level reference's enclosing QN (== the module/directory QN) directly, landing on that shared directory node: all same-package class-level references collapsed onto ONE source with an arbitrary file_path. Fix: the structural directory node is never touched by a colliding Module def (upsert AND merge paths, same guard), and the finders treat a Folder/Project hit as a miss, falling back to the per-file File node — so each file's class-level usages attribute to that file, deterministically. spring-petclinic: OwnerRepository USAGE users are again exactly the 7 grep-verified files on every fresh index (HEAD returned 4-6 with bogus, run-varying entries); 142-bean sweep on a Spring repo stays at 757/757 user-file recall; incremental re-index preserves inbound USAGE counts.
|
Thanks for the stable USAGE-edge ownership fix for #787. Triage: high-priority parsing correctness. Review will focus on the reproduce-first fixture across repeated indexes and the graph-buffer merge/upsert changes. The important invariant is deterministic source ownership for directory-module languages without regressing the #667 package/module behavior. |
Since DeusData#667 every file in a Java/Go package shares a directory-based module QN that collides with the pipeline's Folder/Project node. Class-level references resolved their enclosing scope to that shared node, whose file_path was clobbered by each file's always-emitted Module def — last writer wins (sequential upsert) or last-merged worker wins (parallel merge) — so USAGE-edge sources conflated across same-package files and varied run to run. Three coordinated changes: - cbm_gbuf_upsert_node: a Module def colliding with a Project/Folder node no longer updates ANY field (the DeusData#667 guard only protected the label). - merge_update_existing: same guard on the parallel worker-gbuf merge path; the ID remap stays outside the guard so worker edges still redirect onto the canonical node. - Source-node finders (pass_usages, pass_parallel, pass_calls) treat a lookup landing on a Folder/Project container as a miss and fall back to the per-file File node (new cbm_pipeline_node_is_dir_container helper). Adds tests/repro/repro_issue787.c: exact-source-set, 5-run stability, and >50-file parallel-path guards (red on the unfixed code in both sequential and parallel flavors). Distilled from DeusData#828: the author's commits lacked DCO sign-off trailers and could not be cherry-picked; content applied verbatim plus one clang-format fix in graph_buffer.c. Closes DeusData#787. Co-authored-by: Tommy Corbett <spde89@gmail.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Thank you — this was a precise, well-diagnosed fix. All three mechanisms you identified (the label-only guard leaving name/file_path/props clobbered, the unguarded merge path, and class-level refs landing on the shared directory node) were verified exactly as you described, and your 3-part reproduction (exact source set / 5-run stability / >50-file parallel) was model reproduce-first work. We carried it over the line as 896aa91 (PR #844) with you credited as co-author — the only changes were adding DCO sign-off (your commits lacked the trailer) and one clang-format fix. Closes #787. Closing in favor of the distill — thanks again for the excellent write-up! |
… directory-module collisions Follow-up to #787/#828, disclosed during #828's review: the same clobber class also affects two call sites #828 deliberately left out of scope. create_env_configures_for_file (pass_definitions.c) and resolve_file_throws (pass_parallel.c) both resolved their edge source via a direct cbm_gbuf_find_by_qn(enclosing_func_qn) lookup with no guard against the lookup landing on the shared Folder/Project node for directory-module languages (Java/Go) -- the exact #787 root cause, just unguarded at two more sites. A class-level construct (no enclosing function) has its enclosing_func_qn fall back to the MODULE QN, which for a directory-based module collides with the package's Folder node, conflating every same-package file's CONFIGURES/THROWS edges onto one arbitrary source. - create_env_configures_for_file: added the same cbm_pipeline_node_is_dir_container guard already used by the three #828-fixed finders, falling back to the per-file File node on a directory-container hit. - resolve_file_throws: was missing the `rel` parameter its sibling resolve_file_calls/usages/rw already carry, so it inlined a raw lookup instead of calling the shared, already-guarded find_source_node. Added `rel` and switched to find_source_node, matching its siblings exactly. Regression guard: tests/repro/repro_issue842.c, reproduce-first per the issue's request for "an #828-style multi-order red fixture": - CONFIGURES reproduces on a small 5-file sequential fixture (Go, since Java's System.getenv detection is separately broken -- extract_env_key_from_call reads a "function" field Java's method_invocation node doesn't have; noted in the test, out of scope here). - THROWS only reproduces past MIN_FILES_FOR_PARALLEL=50 (the worker-merge path) -- confirmed empirically that the small sequential fixture already attributes correctly even on unfixed HEAD, mirroring #787's own two-faced (sequential/parallel) bug. Padded with 60 filler classes, following repro_issue787.c's build_parallel_fixture precedent. RED on unfixed HEAD (both tests): 0 passed, 2 failed. GREEN with this fix: 2 passed, 0 failed. Full scripts/test.sh (5958 cases) passes clean with this change; scripts/lint.sh's two pre-existing failures (pass_lsp_cross.h parameter-name mismatch, extract_defs.c cppcheck AST parse) are unrelated and present identically on unfixed main. Fixes #842 Signed-off-by: Dhruvy0804 <dhruvgarg553@gmail.com>
What does this PR do?
Fixes #787: since #667, fresh indexes of the same Java tree attached
USAGEedges to different (wrong) source files on every run.Reproduce-first, per the triage note on the issue: the new
tests/repro/repro_issue787.casserts stable USAGE source ownership across repeated indexes — exact distinct-source set, 5 sequential re-indexes, and a >50-file variant that forces the parallel merge path. All three cases fail on unfixed HEAD (the parallel case surfaces a random filler file as a USAGE source, which is the race made visible).Root cause
#667's directory-based Java/Go module QNs collide with the package's Folder node QN (or Project for root files), while every file still emits a
Moduledef with that QN. Three code paths then corrupted source attribution on the shared node:cbm_gbuf_upsert_node(graph_buffer.c) — the QA: reproduce-first bug suite + LSP/grammar extraction fixes (5-platform board green) #667 guard kept the Folder/Project label on a colliding Module def but still clobberedname/file_path/line-range, so the shared node'sfile_pathbecame whichever same-package file was processed last.merge_update_existing(graph_buffer.c) — the worker-gbuf merge had no guard at all: unconditional "src wins", including relabelling the Folder node toModule. Worker merge order varies per run — this is the nondeterminism.find_enclosing_node/find_source_node/calls_find_source) look up class-level references' enclosing QN directly, landing on that one shared directory node — so all same-package class-level USAGE edges (e.g.@Autowiredfields) collapsed onto a single source with an arbitraryfile_path.Fix
cbm_pipeline_node_is_dir_container()predicate; the three finders treat a directory-container hit as a miss and fall through to the per-file File node.Verification
OwnerRepositoryinbound USAGE sources = exactly the 7 ground-truth files, identical every run (HEAD gave 4–6 with run-varying false positives).Two adjacent direct enclosing-QN lookups were left out of scope (env-access CONFIGURES sources and
resolve_file_throws); noted here rather than folded in, happy to follow up if you want them handled the same way.