Skip to content

[Homebrew/Product] Validate the complete bottle-backed shell lifecycle - #1087

Closed
brandonpayton wants to merge 21 commits into
mainfrom
integration/homebrew-post-frozen-product-batch-qk044
Closed

[Homebrew/Product] Validate the complete bottle-backed shell lifecycle#1087
brandonpayton wants to merge 21 commits into
mainfrom
integration/homebrew-post-frozen-product-batch-qk044

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 24, 2026

Copy link
Copy Markdown
Member

Goal

Ship one coherent ABI 42 product cutover:

  • a mostly-lazy shell with Bash and its startup closure embedded;
  • ordinary /usr/bin/brew, whose Homebrew runtime remains lazy;
  • first-party bottle installation from Kandelo-dev/homebrew-tap-core;
  • installation from an independently owned third-party tap; and
  • the same exact image and package identities tested in Node.js and Chromium,
    including root-filesystem export and reboot.

Why this is one integration PR

The ABI, publisher, package, lazy-VFS, and browser-lifecycle work was developed
on related branches. Running the expensive package, Node.js, and Chromium gates
on each branch separately would not prove that their final trees work together.
This PR preserves the reviewed source commits as merge ancestors and pays for
one complete product-level gate on their combined tree.

Important ABI boundary

The current main tree still runs ABI 41. PR #1092 preserved the exact ABI 42
producer source in Git history, but intentionally did not activate that tree.
This PR therefore includes the reviewed ABI 42 kernel, shared ABI, libc, and
host-runtime activation required by the ABI 42 bottles. It is not only a
packaging change, and it must not merge without fresh explicit kernel approval.

Product behavior under test

  • Generate an exact lock from the final public tap metadata for 22 requested
    roots and their complete dependency closure.
  • Embed only libcxx, ncurses, and Bash; keep optional bottles lazy.
  • Keep the Homebrew bootstrap tree lazy until brew is first used.
  • Install, execute, reinstall, and uninstall a first-party bottle.
  • Tap an independent repository and install its package plus its first-party
    dependency.
  • Preserve installed state across exported-image reboot.
  • Run exact-byte shell, lifecycle, and graphics checks on both supported hosts.

Integrity and CI design

  • Bottle, tap, package, image, and bootstrap inputs are bound to exact commits,
    digests, byte counts, and ABI values.
  • The durable package-generation base is downloaded anonymously and validated
    with current code; changed package identities use the normal source fallback.
  • Closed CI mirrors already-verified bytes without changing their canonical
    transport identities.
  • Non-obvious security and artifact-identity boundaries have inline WHY
    comments and focused regression tests.

Current status

This PR remains a draft while the final Git bottle, exact generated lock,
mostly-lazy image, and complete Node.js/Chromium lifecycle are being finished.
The description will be updated with exact run and artifact evidence before the
PR is marked ready.

Embed Bash and its always-needed runtime closure, but leave Homebrew and the selected Formula trees as verified lazy sources. Derive eager and lazy variants from the same package identities so product policy does not create a second recipe.

Record the exact migration locks, Node and Chromium runtime probes, browser download accounting, and the current rollout plan. Ruby-backed brew commands remain intentionally blocked on the ABI 42 dynamic fork-continuation transition and final tap repin before this candidate can ship.
The cutover intentionally turns homebrew-bootstrap into a two-output package and makes it the shell package’s sole registry dependency. Update the combined packaging-contract tests to assert that complete atomic generation and the current shell revision, while continuing to reject every retired registry bundle.
Exercise /usr/bin/brew itself rather than only the canonical target. The test proves symlink resolution finds the registered package tree, fetches the complete verified archive once, materializes every member atomically, and does not refetch on later use.
Regenerate the canonical program package index after the frozen lazy-shell stack changed the closure keys used by lamp, nginx-php-vfs, and WordPress. This restores the package projection freshness contract used before browser assets may resolve.
Add one durable root-image export contract to the Node and browser kernel hosts. The owning worker closes a snapshot gate, drains earlier mutations, rejects concurrent rootfs changes and live processes, and transfers only the root image so lazy descriptors survive a reboot without capturing boot-scoped mounts.

Cover pre-init and non-VFS failures, exact-byte transfer, lazy-state restore, live/tearing-down process exclusion, mutation ordering, and real Chromium/Firefox/WebKit reboot behavior. Keep Vite module-worker reloads on a normal 200 response because WebKit rejects conditional worker revalidation under COEP even when the original response has matching CORP headers.
Exercise the browser worker through its public API so rootfs snapshots are proven to reject live and tearing-down processes, wait for an active lazy-file materialization, and exclude overlapping exports and writes. Reboot the exported bytes and verify the completed mutation is durable while the rejected write is absent.
…romium

Move the generated guest scripts, export/reboot orchestration, and exact VFS input checks into browser-safe shared contracts so both hosts prove the same lifecycle instead of maintaining parallel semantics.

Add an explicitly gated Chromium fixture with immutable URL, byte-length, SHA-256, tap-revision, embedded-plan, and closed-payload bindings. Wire offline admission coverage into CI and document that live execution still requires one coherent published ABI-42 fixture.

Validated with the 38-test focused contract suite, host declaration typecheck, targeted TypeScript checks, Node and browser bundles, the selected production Vite build, and a real Chromium offline admission run.
Regenerate the authoritative package projection after the bridge changes the shell-owned deferred package tree. This keeps the bridge head independently buildable before the browser ownership follow-up adds its separate shell input change.
Replace the obsolete total invocation count with named structural checks for offline lifecycle rejection, shell acceptance, and MODESET. The WHY comment records that process isolation belongs to each heavyweight proof rather than to an incidental command count.
The shell imports the split deferred-tree contract through package-deferred-tree.ts, so that contract file contributes to reproducible shell output and must be an explicit build input. Regenerate the authoritative projection in the same commit so this bridge head remains internally exact.
…ication

Transfer the exported lifecycle filesystem directly into its worker-owned phase-two runtime, enforce one bounded deadline across fixture and script work, and prove the exact ownership path in Node and Chromium. The bridge owns its deferred-tree package inputs and browser-process contract; this commit carries the lifecycle ownership changes and their closed-lazy-assets identity delta.
A package archive can be present but unusable, including when a same-run staging overlay has a stale cache identity. In that case the resolver must be able to run the shell recipe from any direct or transitive caller without caller-specific npm setup.

Create a resolver-owned Git source snapshot, install both locked JavaScript dependency trees there with a scrubbed public-registry npm environment, and run the composer only from that private snapshot. Isolate every snapshot Git command from ambient attributes, replacement refs, selectors, and external diff configuration.

Remove predictive setup from CI and run.sh, bind the preparer into the package cache identity, and cover stale-overlay fallback, concurrent builds, path substitution, Git/npm configuration, and credential isolation.

Regenerate the authoritative program-package projection so shell and every composite package that depends on it carry the new manifest and cache identities.
…ages

Preserve the exact publisher-integrity and browser lifecycle histories in one downstream integration point after the immutable ABI 42 source. Regenerate the authoritative package projection from the combined package manifests so the shared generated file represents both lineages.
…cle batch

Preserve the exact source-fallback fix and current-main ancestry in the post-publication product integration point. Combine both shell-closure test contracts and regenerate the package projection from the resulting manifests.
… snapshot

The combined lazy-shell and self-contained source-fallback contracts add one Git-owned package-tree input. Read it from the resolver-owned source snapshot, not the shared checkout, and make the concurrency test prove that isolation.
@github-actions

Copy link
Copy Markdown
Contributor

Phase B-1 matrix build status — pr-1087-staging

ABI v42. 39 built, 37 failed, 76 total.

Package Arch Status Sha
icu wasm32 built 48a3ee79
libcurl wasm32 built 8273da44
libcxx wasm32 built 53612e1c
libcxx wasm64 built 986b4977
libiconv wasm32 built b494e26a
libpng wasm32 built e31da77e
libxml2 wasm32 built f8e7709d
libzip wasm32 built 0e020e8d
openssl wasm32 built bdef2279
openssl wasm64 built 11532ba6
sqlite wasm32 built a21f5097
sqlite wasm64 built 1f7a45fb
zlib wasm32 built a826c5a1
zlib wasm64 built 09bd3336
bc wasm32 built 87d9afde
bzip2 wasm32 built 09122fb0
coreutils wasm32 built 72765db8
cpython wasm32 built 314d4763
curl wasm32 built 63cca65e
dash wasm32 built 0b2035e2
diffutils wasm32 built 5b829e0f
dinit wasm32 built 28167e43
erlang wasm32 built fb022926
fbdoom wasm32 built ad115c1c
file wasm32 built 41f1cbea
findutils wasm32 built 9cf821c8
gawk wasm32 built f25ff8e1
git wasm32 built 956b61b3
grep wasm32 built 2b95ab13
gzip wasm32 built c5fea74e
homebrew-bootstrap wasm32 built 8230aaca
kandelo-sdk wasm32 built 5ec342f9
kernel wasm32 built d0e03c4b
less wasm32 built 69fd5aea
lsof wasm32 built 0cbea55d
m4 wasm32 built 746a3cb4
make wasm32 built 8bbaab34
mariadb wasm32 failed
mariadb wasm64 failed
modeset wasm32 built 8a5f567e
msmtpd wasm32 built 94c0ffb6
nano wasm32 failed
ncurses wasm32 failed
netcat wasm32 failed
nginx wasm32 failed
php wasm32 failed
posix-utils-lite wasm32 failed
ruby wasm32 failed
sed wasm32 failed
spidermonkey wasm32 failed
tar wasm32 failed
tcl wasm32 failed
unzip wasm32 failed
userspace wasm32 failed
vim wasm32 failed
wget wasm32 failed
xz wasm32 failed
zip wasm32 failed
zstd wasm32 failed
bash wasm32 failed
mariadb-test wasm32 failed
mariadb-vfs wasm32 failed
mariadb-vfs wasm64 failed
nethack wasm32 failed
node wasm32 failed
redis-vfs wasm32 failed
shell wasm32 failed
spidermonkey-node wasm32 failed
vim-browser-bundle wasm32 failed
lamp wasm32 failed
nethack-browser-bundle wasm32 failed
nginx-php-vfs wasm32 failed
nginx-vfs wasm32 failed
node-vfs wasm32 failed
rootfs wasm32 failed
wordpress wasm32 failed

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

Record the narrow critical-path acceptance for an eager-Bash, lazy-brew shell with real first-party and independent-tap installs on Node.js and Chromium. Keep Erlang, Dinit, failed noncritical formulae, catalog parity, manuals, activation groups, and the wider migration as visible follow-up work without allowing ABI-41 fallback.
brandonpayton added a commit that referenced this pull request Jul 24, 2026
…1092)

Keep the immutable ABI 42 bottle producer commits reachable from main without replaying their stale product tree. The PR diff documents the bounded ancestry-only procedure; #1087 owns the current shell and guest Homebrew cutover.
Base automatically changed from homebrew/formula-source-reference-contract-qk044 to main July 24, 2026 21:16
@brandonpayton brandonpayton added the skip-staging-tests Opt into skipping the staging tests. This does not skip tests before merge. label Jul 24, 2026
@brandonpayton

Copy link
Copy Markdown
Member Author

Closing as superseded by exact public replacement #1109: ee45cb47e carries the sealed lifecycle substrate, d61a83b69 activates brew atomically, and 32c84abe7 gates the exact live Node/Chromium lifecycle. This older mega-stack must not be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-staging-tests Opt into skipping the staging tests. This does not skip tests before merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant