Skip to content

test(runner/toolexec): carry the spike's corpus and criteria over as acceptance tests - #582

Merged
hmdsefi merged 1 commit into
mainfrom
plan/2.5-task-11
Nov 1, 2026
Merged

hmdsefi merged 1 commit into
mainfrom
plan/2.5-task-11

Conversation

@hmdsefi

@hmdsefi hmdsefi commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Carries the spike's acceptance corpus and its criteria over as permanent tests in runner/toolexec/acceptance. Stacked on #581.

  • AT-XTE-33: C2 (gkv), C3 (httpcorpus), C4 and C5 (order) give the same trace hash and status for seeds 0 to 19 in two runs. C3 is built with net/http rewritten (FAULTLINE_TOOLEXEC_STD=net/http). With the default rewrite set, net/http's own goroutines and timers run outside sched, and C3 differed between two runs of a seed in about 1 of 2,500 comparisons.
  • AT-XTE-34 and 35: C4, C5 and C7 follow the seed, with enough distinct outcomes over seeds 0 to 99 and equal outcomes per seed.
  • AT-XTE-36: the root package, shims/netsim, shims/grpcsim and the compat fixture pass when built with the wrapper, under FAULTLINE_EXACT=1 and FAULTLINE_CHECK_DETERMINISM=1.
  • AT-XTE-37: C3 built with net/http rewritten logs compile net/http rewritten and stays exact.
  • AT-XTE-38: the wrapper build of C2 is at most 5 times slower than the plain build (median of 3 runs, plain and wrapped runs alternating).
  • AT-XTE-39 and 40: runner/toolexec depends only on the root module and gograph, and runner/toolexec plus internal/gosched stay within 2,900 counted lines.
  • AT-XTE-41 and 42: the spike's exb08 and exb09 fixtures keep their rewritten forms and their seed sensitivity.

Closes #347

@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Base automatically changed from plan/2.5-task-10b to main November 1, 2026 02:58
@hmdsefi
hmdsefi merged commit f30a77e into main Nov 1, 2026
18 checks passed
@hmdsefi
hmdsefi deleted the plan/2.5-task-11 branch November 1, 2026 03:11
@hmdsefi hmdsefi mentioned this pull request Nov 1, 2026
14 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[2.5-11] Exact backend: acceptance corpus and the EXB criteria as permanent tests

1 participant