Repository navigation
ci: build installer & appliance in parallel matrix legs - #52
Closed
phorcys420 wants to merge 1 commit into
Closed
phorcys420 wants to merge 1 commit into
phorcys420 wants to merge 1 commit into
Conversation
The test Images job previously ran a single matrix leg per system that built the installer and appliance ISOs sequentially on one runner, doubling wall-clock when both kinds build full. Restructure so the plan job emits a JSON matrix of one leg per (kind x system); each Images leg builds exactly one kind on its own runner, so installer and appliance build in parallel. - plan: output a compact JSON matrix (kind, system, runner, full, suffix) instead of installer_full/appliance_full/kinds. - Images: matrix: fromJSON(needs.plan.outputs.matrix); per-leg name 'Build <kind> <ISO|DRV> (<system>)'; single-kind build + one ISO artifact per leg (still bundling the ISO with its .sha256 sidecar).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Splits the test workflow's
Imagesjob so the installer and appliance ISOs build on separate runners in parallel, instead of sequentially on a single runner.Why
Previously the
Imagesmatrix had one leg per system (x86_64 / aarch64), and each leg ranbuild_kind installerthenbuild_kind appliancesequentially on the same runner. When both kinds build full, that roughly doubles wall-clock — and because each ISO's squashfs (full system closure at the chosen compression) is an inherently per-host derivation, the two kinds can't dedup that work even with a warm store. Running them on separate runners makes the doubled cost parallel, so wall-clock stays ~1×.How
planjob: now emits a compact JSON build matrix (jq -c) of one entry perkind × system— each carryingkind,system,runner,full(ISO vs drv-only), and anISO/DRVsuffix— instead of theinstaller_full/appliance_full/kindsoutputs. The label/event logic that decides full-vs-drv is unchanged.Imagesjob:matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}(4 legs: installer/appliance × 2 arches). Each leg builds exactly one kind (make <kind>/isoor<kind>/drv), names itselfBuild <kind> <ISO|DRV> (<system>), and uploads a single per-kind artifact still bundling the ISO with its.sha256sidecar.Net behavior (which kinds build full, per-kind artifacts, fast verification compression) is unchanged; only the job topology changes from per-system (sequential kinds) to per-kind×system (parallel).
Notes / trade-offs
os/arch(not per-kind), so installer and appliance legs on the same arch still share one cache lane and stay within the repo's 10G cache budget.Validation
test.ymlYAML parses.planmatrix shell/jqfor both-full and mixed (installer ISO / appliance DRV) cases — emits the expected 4-leg JSON with correctfull/suffix.INSTALLER_FULL/APPLIANCE_FULLjob env oroutputs.kinds.nix buildin the sandbox (unwritable store); runtime build behavior will be confirmed by this PR's CI.