Run test configs on parallel stacks, two slots by default - #282
Merged
Merged
Conversation
Configs ran serially, so a full local pass cost the sum of all eleven; they now partition across isolated stacks and cost roughly the largest slot. Each slot gets its own compose project (sheaf-test-<s>) and port triplet, so databases, volumes, and networks never overlap; slot 1 uses the exact serial defaults. Two slots by default - conservative for any dev box but still roughly halves a full pass; --jobs N widens, --jobs 1 forces the classic serial path (byte-identical output, banners included), and single-config runs (CI's shape) clamp to one slot so the Actions matrix is untouched. The image builds once before slots fork, per-config output is captured and replayed serially in canonical order, the machine-wide flock still covers the whole run, and slot traps tear down their own stacks with a master trap for interrupt cleanup. selfhosted/none dominates wall clock and gets slot 1 alone. The test compose interpolates host ports and SHEAF_BASE_URL from SHEAF_TEST_*_PORT (defaults unchanged) and pins the image tag the serial build already produced. Parallel verification flushed out one real coupling: the import-runner test helpers hardcoded the sheaf-test compose project for docker exec and now honour SHEAF_TEST_COMPOSE_PROJECT (default unchanged). Verified live: the default two-slot path and explicit --jobs 2 both ran concurrent stacks to green with clean teardown; --jobs 1 matches serial exactly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local-dev quality of life: run_tests.sh now spreads the eleven configs across parallel stacks, two slots by default - conservative for any dev box but still roughly halving a full pass.
--jobs Nwidens it,--jobs 1forces the classic serial path (byte-identical output, banners included), and single-config runs clamp to one slot, so CI's per-config matrix jobs are untouched.Isolation comes from per-slot compose projects (
sheaf-test-<s>) with offset port triplets: each slot has its own postgres, redis, app, volumes, and network, and slot 1 keeps the serial defaults (8001/5433/6380). The image builds once before slots fork; per-config output is captured and replayed serially in canonical order so nothing interleaves; the machine-wide flock still covers the whole run; slot traps tear down their own stacks with a master trap for interrupt cleanup.selfhosted/nonedominates wall clock and gets slot 1 to itself.The test compose interpolates its host ports and
SHEAF_BASE_URLfromSHEAF_TEST_*_PORT(defaults unchanged) and pins the image tag the serial build already produces. Parallel verification flushed out one real coupling: the import-runner test helpers hardcoded thesheaf-testcompose project for theirdocker execand now honourSHEAF_TEST_COMPOSE_PROJECT(default unchanged, so direct pytest use and CI need nothing).Verified live: the no-flag default ran two concurrent stacks to green (both project prefixes in
docker ps, zero container/volume residue after teardown), explicit--jobs 2likewise, and--jobs 1matches serial output exactly. Incidental fixes: all eleven banners carry the mode suffix (previously four did), and duplicate config args dedupe instead of miscounting the progress prefix.Remaining soak: one timed full pass of all eleven configs to size headroom before habitually widening past two.