[CI/Packaging] Publish each PR staging snapshot with one finalizer - #1089
Closed
brandonpayton wants to merge 4 commits into
Closed
[CI/Packaging] Publish each PR staging snapshot with one finalizer#1089brandonpayton wants to merge 4 commits into
brandonpayton wants to merge 4 commits into
Conversation
Keep matrix builders read-only and collect their immutable archives before any release mutation. Validate a complete target-relative snapshot offline, acquire the staging lock once, publish the index only after every referenced archive is present, and re-read the published transaction. Preserve last-green failure evidence and distinguish build success from publication success.
Let the test gate consume only exact-current last-green fallbacks after the single writer has published and re-read a complete snapshot, while a separate package result remains red for any failed build. Resolve the release ID once, read every asset page through retry-safe stable snapshots, reject duplicate or changing inventories, and cover first-ABI, page-two, partial-response, conflict, and index-only publication paths.
Freeze every valid expected package/architecture key available in canonical state without requiring absent keys at the baseline-only step. Matrix artifacts can then add or repair those keys, while the post-composition validator still requires exact full coverage. Cover package additions, architecture additions, partial archive materialization, unexpected entries, and falsely complete snapshots.
Base automatically changed from
homebrew/formula-source-reference-contract-qk044
to
main
July 24, 2026 21:16
Member
Author
|
Superseded by merged #1160. The current design replaces this draft |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PR package staging currently makes every package worker take the same release
lock, download release state, upload one archive, and replace
index.toml.That turns a large parallel package wave back into a mostly serial publication
queue. It also lets an interrupted first run leave a sparse release that later
runs cannot safely reuse.
This change keeps package compilation parallel and gives release mutation to
one auditable writer after the full build wave is known. Package identity,
dependency ordering, last-green behavior, and consumer testing remain
fail-closed.
What this changes
dependency order and upload immutable workflow artifacts, with up to ten
workers running at once.
canonical state, trims it to the current managed package set, and applies
every successful matrix artifact offline. Missing, stale, transient, and
failed-without-fallback keys become matrix-owned gaps. An exact-current
failed entry may contribute its fully verified last-green fallback.
Final validation still requires exact full coverage.
complete first snapshot. Missing or invalid artifacts are recorded as
truthful failed entries; the finalizer never invents a fallback.
coverage, version, revision, cache key, immutable Git provenance, safe
target-relative URLs, and archive manifest/hash/size agreement.
index. Disabled, historical, and unrelated canonical assets are not copied.
retry-safe stable snapshots, rejects duplicate or changing inventories,
uploads or verifies immutable archives, replaces the complete index as the
final mutable write, and re-reads all referenced bytes.
fallback has the same version, revision, cache key, and immutable Git
provenance. A separate package result remains red, so testing a fallback
never hides the failed current build.
and consumer test results.
GitHub implements
gh release upload --clobber index.tomlas delete followedby upload, not an availability-atomic swap. Publishing archives before the
index keeps every visible index referentially safe, but interruption can leave
the index temporarily absent until a retry repairs it. Canonical activation's
journaled publisher is intentionally unchanged.
Validation
bash scripts/dev-shell.sh env AR=llvm-ar bash tests/scripts/package-publish-flow.shbash scripts/dev-shell.sh cargo test -p xtask --target aarch64-apple-darwin staging_reuse --no-fail-fast— 31 passed/opt/homebrew/bin/actionlint -color .github/workflows/staging-build.ymlSC2034excluded only for thepre-existing unused variable in
test-merge-candidate-workflows.sh)git diff --checkThe local tests include package addition and architecture addition against an
existing partial baseline; repair of canonical failed, stale, transient, and
missing keys; exact failed-fallback materialization and later success cleanup;
partial or malformed fallback rejection; a library-to-program package-kind
change; complete and failed first-ABI composition; an index-only all-failed
publication; a later-page asset on a release with more than 100 assets;
partial-response retry; idempotent retry; duplicate metadata; conflicting
immutable bytes; and the workflow outcome truth table.
Hosted proof and current external gate
This PR is stacked on #1085's frozen head so it can be reviewed independently
from the active product rollout. Its own CI-control-only diff does not select a
package matrix, and the canonical
binaries-abi-v42release does not existyet, so its current hosted test preparation fails before exercising the new
finalizer.
After this exact head passes independent review, a separate
lsofpackage-input canary will be rebased onto it. That canary isnever-merge test scaffolding: it intentionally triggers the complete
no-baseline ABI 42 matrix and this single-writer finalizer.