Skip to content

Test fork instrumentation output reproducibility across processes - #888

Closed
brandonpayton wants to merge 1 commit into
mainfrom
fix/fork-instrument-deterministic-output
Closed

Test fork instrumentation output reproducibility across processes#888
brandonpayton wants to merge 1 commit into
mainfrom
fix/fork-instrument-deterministic-output

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 11, 2026

Copy link
Copy Markdown
Member

Why

#907 already landed the production fix that sorts fork-instrumentation regions before assigning synthetic locals and instruction IDs. That makes the output deterministic, but the existing regression repeats the transform inside one process and does not vary Rust's randomized hash-map state.

This PR keeps the remaining test and documentation work:

  • run wasm-fork-instrument twelve times in fresh processes and compare every output byte;
  • use alternating i32/i64 nested regions so a changed allocation order is visible in the Wasm;
  • document the cross-process byte-reproducibility contract and the current ABI 39.

Removed from the old branch

The rewritten branch is one commit directly on current main. Its pre-rewrite head was f3c2c587c65ed7ed15507697efc2b381acd1a2d4.

ABI and packages

This adds a test, a WAT fixture, and documentation only. It does not change the fork format, Wasm exports, host/kernel ABI, ABI_VERSION, ABI snapshot, package inputs, package revisions, or built package bytes.

Validation

Passed through scripts/dev-shell.sh:

  • bash scripts/ci-run-test-suite.sh fork-instrument — 187 tests passed;
  • the exact fresh-process regression by itself — 1 test passed, exercising 12 separate CLI processes;
  • bash scripts/check-abi-version.sh — snapshot, C header, TypeScript bindings, and ABI version are consistent;
  • git diff --check.

Not run because this has no runtime, browser, libc, POSIX, VFS, or package-artifact changes:

  • host Vitest;
  • browser Playwright and ./run.sh browser;
  • libc, POSIX, and Sortix suites;
  • package rebuilds.

cargo fmt --check was attempted through the dev shell, but that shell does not currently provide cargo-fmt. The new Rust test follows the existing test style and compiled in both fork-instrument runs above.

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Phase B-1 matrix build status — pr-888-staging

ABI v18. 53 built, 0 failed, 53 total.

Package Arch Status Sha
bc wasm32 built f45f740f
bzip2 wasm32 built 19695174
coreutils wasm32 built 365366cf
curl wasm32 built 2ae673c2
dash wasm32 built 25bdf2cf
diffutils wasm32 built 85e7d2a9
dinit wasm32 built 177bfb90
fbdoom wasm32 built a61114ca
file wasm32 built 09380c0b
findutils wasm32 built 4a04839e
gawk wasm32 built 05a3ab3f
git wasm32 built 56ff3964
grep wasm32 built eb3c0049
gzip wasm32 built 9d578800
kandelo-sdk wasm32 built fe4bd1c9
kernel wasm32 built 355953e0
less wasm32 built 4dbb52f7
lsof wasm32 built c8a88920
m4 wasm32 built 2910bb31
make wasm32 built ccf6bc63
mariadb wasm32 built 518c5c54
mariadb wasm64 built 4fa41b94
modeset wasm32 built b8d78740
msmtpd wasm32 built 9ca2c667
nano wasm32 built ce3f0ed8
ncurses wasm32 built e5a0410b
netcat wasm32 built 0994f5e6
nginx wasm32 built 7882e5a9
php wasm32 built 8d635623
posix-utils-lite wasm32 built 4ce25d4e
ruby wasm32 built f70a1eea
sed wasm32 built 80aa0020
tar wasm32 built dbcd5b7a
tcl wasm32 built 55f6bfc5
unzip wasm32 built 7102a849
userspace wasm32 built 126d9bdd
vim wasm32 built d53d836a
wget wasm32 built b2ec19ac
xz wasm32 built 41ce04bc
zip wasm32 built f69e2537
zstd wasm32 built d8e76ecf
bash wasm32 built 7b9da8e5
mariadb-test wasm32 built 8645154d
mariadb-vfs wasm32 built c199b6fb
mariadb-vfs wasm64 built 2954778b
nethack wasm32 built dd04c9c8
vim-browser-bundle wasm32 built 2c34f0fd
nethack-browser-bundle wasm32 built 3e635d0c
rootfs wasm32 built 8e7ae770
shell wasm32 built 6b23d674
lamp wasm32 built 04258e42
node-vfs wasm32 built 07e99d2c
wordpress wasm32 built bd6dfe9d

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

@brandonpayton

Copy link
Copy Markdown
Member Author

Homebrew Sharutils exact consumer validation

I reproduced this PR's failure mode and fix against the identical linked, pre-instrumentation GNU Sharutils 4.15.2 shar input.

Input:

  • 230,576 bytes
  • SHA-256 43be9de13cc1e81d256d5d379fd7482c2524384dd349a2e14e291f59c6847466

Pre-fix instrumenter:

  • 24 fresh processes produced 24 distinct outputs
  • every output was 306,664 bytes
  • two saved outputs differ at exactly 328 bytes: 164 synthetic local.set operands and their 164 matching local.get operands
  • the variance comes from the regions.keys() walk that collects carryover plans before allocating per-subregion spill locals

Exact PR head f3c2c587c65ed7ed15507697efc2b381acd1a2d4:

  • 24 fresh processes produced 24 byte-identical outputs
  • output SHA-256 236b32fbe2122c1acf6e307bb17112f6083410fea312e4ed5c9ca0697e544e15
  • wasm-validate --enable-all passed
  • a reduced diagnostic that sorted only the carryover-plan region walk produced the same exact SHA as this PR

This independently confirms the randomized nested-region allocation cause and the canonical-order fix on a real Homebrew consumer. Sharutils bottle publication will remain gated on this PR and a post-merge rebuild. No ABI change is implicated by this evidence.

@brandonpayton

Copy link
Copy Markdown
Member Author

Recording the pre-rewrite head before focusing this PR: f3c2c587c65ed7ed15507697efc2b381acd1a2d4 (currently stacked on fix/teardown-reclaim-readiness at bcd704a6aa7678f8ef326cb7f65003db971b8d02). I’m rebuilding the branch directly on current main, keeping only the fresh-process determinism regression, fixture, and current documentation. The production sorting change already landed through #907, so it will not be replayed.

@brandonpayton
brandonpayton force-pushed the fix/fork-instrument-deterministic-output branch from f3c2c58 to 14848c0 Compare July 13, 2026 17:33
@brandonpayton brandonpayton changed the title fork-instrument: make instrumented Wasm byte-reproducible Test fork instrumentation output reproducibility across processes Jul 13, 2026
@brandonpayton
brandonpayton changed the base branch from fix/teardown-reclaim-readiness to main July 13, 2026 17:34
The deterministic region ordering already landed through #907, but its in-process regression does not vary Rust HashMap random state. Run the CLI in twelve fresh processes against an alternating-type nested-region fixture and compare the emitted bytes.

Document the cross-process byte-reproducibility contract and update the fork-instrumentation reference to the current ABI 39. This adds no ABI or package-artifact change.

Validation: the full fork-instrument suite passed 187 tests; the focused fresh-process test passed; and the ABI snapshot check passed through scripts/dev-shell.sh.
@brandonpayton

Copy link
Copy Markdown
Member Author

Closing this focused source PR because #934 now carries the unique fresh-process fork-output reproducibility fixture and current ABI-39 documentation at exact green head 260d75d5adc31b327f5591229dbfbb287d395fd6. The implementation already on main through #907 and stale ABI wording were deliberately excluded. The full staging and fork-instrument gates passed. This is not a claim that this branch was merged verbatim.

@brandonpayton
brandonpayton deleted the fix/fork-instrument-deterministic-output branch July 14, 2026 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant