Skip to content

[Kernel/Homebrew] Land ABI 42 before activating the bottled shell - #1097

Merged
brandonpayton merged 34 commits into
mainfrom
homebrew/abi42-main-first-final-qk044
Jul 26, 2026
Merged

[Kernel/Homebrew] Land ABI 42 before activating the bottled shell#1097
brandonpayton merged 34 commits into
mainfrom
homebrew/abi42-main-first-final-qk044

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 25, 2026

Copy link
Copy Markdown
Member

Why

Kandelo needs ABI 42 before it can run the new ABI 42 Homebrew bottles. That
creates an ordering problem: the platform must land before the bottled shell can
become the default, but the browser demos must remain bootable during the
transition.

This PR lands the ABI 42 platform and keeps the current shell available through
a temporary source-built image. It deliberately does not switch the demos to
Homebrew bottles yet. That cutover follows after this exact change is on
main.

What changes

Make fork, catch, and spawn state activation-owned and bounded

  • make the kernel process table the source of truth for task identity;
  • replace fixed fork-continuation scratch storage with bounded linked frames;
  • keep ordinary WebAssembly catch state owned by the function invocation that
    created it, including recursive calls;
  • pass the exact saved continuation authority into child launch instead of
    deriving an address from global scratch state;
  • validate continuation ownership and wasm32/wasm64 pointer geometry before
    allocating a child; and
  • copy the complete posix_spawn input through bounded, reusable kernel-owned
    storage with pointer, range, and ARG_MAX checks.

These are platform fixes, not Bash- or package-specific workarounds.

Strengthen package and VFS boundaries

  • reject stale or structurally invalid fork-instrumented Wasm;
  • validate package archives and extract requested members through one bounded,
    path-safe implementation;
  • keep lazy and eager package-tree behavior on the same validated VFS path;
  • bind package generations, indexes, and publication writes to explicit source
    and authority identities; and
  • fail closed when release state, package provenance, or retry state cannot be
    proven.

Keep the shell bootable while bottles are finalized

  • build the current rootfs shell closure from source through the normal package
    resolver;
  • keep the always-needed shell eager and preserve the current shell product
    contract, including lazy utilities and supporting trees; and
  • continue exercising the same image through Node.js and browser hosts.

This source-shell bridge is intentionally temporary and is covered as such by
tests and documentation.

What happens to the pre-merge ABI 42 packages?

The exact 15-package rootfs closure produced by commit
748c2609954d2809bbcbbcb642fa7d257fc0dbc6 is preserved in a public,
content-addressed, application-sealed release. Its tag points directly to that
commit, all 18 release assets were anonymously read back and matched byte for
byte, and the manifest says admission: none.

That release is evidence, not a claim that the packages were built from
main.

After this PR merges, a separate packaging change will rederive the selected
rootfs build-input closure from the actual merge commit. It may create an
admitted generation only when the package projection, expected ledger, and
component inputs are exactly identical. Any difference requires a rebuild. The
larger browser package set is not covered by that equivalence and will not be
silently reused.

What this PR does not do

  • It does not activate the mostly-lazy bottled shell.
  • It does not make brew available inside the demo yet.
  • It does not publish or rotate the final tap configuration.
  • It does not label pre-merge package bytes as built from main.

Those are the next packaging and product steps after ABI 42 is on main.

Keep browser programs in one trusted cache

The exact source-built shell leaves browser programs in a temporary package
cache. Package fetching, the Vite build, and the sealed browser preview must all
use that same cache. If one phase silently falls back to another cache, Vite
correctly rejects the resulting symlink instead of serving a file from outside
its approved root.

The CI-only child commit f913f22b20b95de0687e308a5cea0f6fd477a5dd
passes one runner-local cache through every producer and browser consumer in
both source and bottle modes. It does not widen Vite access, change a package
identity, or change kernel behavior. The reviewed kernel/package head
f9bcfae2787c2687427bf1d41544637cdb4a9714 remains its direct parent.

Validation

Evidence already run on the combined candidate includes:

  • ABI snapshot and generated libc/TypeScript binding checks;
  • the complete fork-instrument suite and kernel-focused tests;
  • wasm32/wasm64 continuation geometry and malformed-authority regressions;
  • the real previously failing Nginx fork artifact, including HTTP 200 and 404
    child paths;
  • host-runtime tests plus focused Chromium continuation coverage;
  • all Sortix spawn cases, including repeated environments larger than 64 KiB;
  • package-generation, durable publisher, archive extraction, workflow trust,
    source-shell closure, and deployment contract suites; and
  • Node.js and browser consideration for every shared host-runtime change.

The final exact-head CI result will be recorded here before merge. Because this
changes the kernel and ABI, it will not merge without explicit approval of that
exact green head.

Final exact-head evidence

The final reviewed head is f913f22b20b95de0687e308a5cea0f6fd477a5dd; its direct parent is the reviewed kernel/package head f9bcfae2787c2687427bf1d41544637cdb4a9714. Current main is 4c0ec4a3629d00dd2f62aeef0b7177431ed5c2f7, and GitHub synthetic merge 2e080a5312fd9f56c1d4a45eab9c84b61522abdf has those exact ordered base/head parents.

Fresh evidence on that exact head/merge is green:

  • staging run 30188135451;
  • exact shell run 30188135455: forced source closure, one-cache browser inputs, exact installation, sealed product build, Node boot, Chromium shell boot, and fresh-process MODESET; image SHA-256 ec67d372822f8352cd18def1c4ab84229906bca3255166654ce99b01e9832b01;
  • Prepare run 30188160612: all 76 archive promotions, candidate Homebrew startup, libc, POSIX, Sortix, Vitest, browser, final test gate, and merge gate; and
  • scripts/check-abi-version.sh on both exact head and live synthetic merge: ABI snapshot, libc header, and generated TypeScript bindings are current, with the snapshot change correctly paired to the ABI 41 → 42 bump.

Prepare published the sealed 80-asset candidate release merge-candidate-abi-v42-pr-1097-run-30188160612-attempt-1 (release ID 359930157) targeted at the exact head. The CI-only child changes no package archive or cache-key identity.

Merging still requires explicit kernel approval and must use a merge commit so the exact reviewed head remains in main ancestry.

Apply the reviewed ABI 42 kernel, libc, host, package, VFS, and publisher foundation onto the current main tree, including activation-owned catch replay and bounded posix_spawn transport. This is an ordinary product-tree change: final archives and bottles will be rebuilt only after the resulting commit lands on main.
…e generations

Durable generations must be derived only after coherent package activation and must not preserve temporary PR identities or execute historical source tooling. Use the exact freshly queried main SHA as both package source and workflow authority, derive complete browser roots and typed closures with current declarative readers, and require every selected archive to carry that same build commit.

Share one strict semantic validator across preparation, publication, and materialization so the expected ledger, index, snapshot, asset inventory, archive manifests, and local bytes remain one fail-closed identity. Keep #1094 counts as selection regression evidence only; its pre-main artifacts are never eligible final inputs.
Require every archive producer to record a canonical source repository and exact commit in its embedded manifest. Restrict canonical force rebuilds to the live Automattic/kandelo main SHA, force each selected target through its source recipe, and recheck that authority before release and index mutations.

Keep tested PR and synthetic-merge archives available to the general resolver while making their distinct source identity visible to durable-generation admission. Cover source parsing, archive round trips, workflow plumbing, exact-main guards, and CI scope routing.
Incremental planning now trusts a bottle only when that architecture-specific sidecar records Automattic/kandelo at the admitted exact main SHA. This prevents newer top-level metadata from making a preserved older-commit bottle look current.
Regenerate the committed program-package projection after the package archive action became an authoritative global build input with source provenance. This keeps every projected cache key aligned with the exact build action used by CI.
Expand selected canonical rebuild roots to their complete buildable dependency graph and schedule that graph in explicit parallel topological levels. Each dependent consumes only same-run artifacts through an empty job-local cache, while missing artifacts fail immediately instead of falling back to an older cache-equivalent archive.

Bind sysroot and libcxx preparation to the admitted main SHA, expose a narrowly scoped source-build option for libcxx, and document and test the exact-main closure contract.
The credentialed ORAS transport now requires explicit exact-main authority and re-reads protected main immediately before copying either a bottle child or complete version index. Race coverage proves an intervening main advance stops before oras cp.
Canonical bottle builds need a single resolver view spanning independently sealed wasm32 and wasm64 browser inputs, without falling back to the mutable package index. Require an exact package-source SHA when requested, combine only validated local generation indexes and archives, and cover architecture-only packages and incompatible package identities.
The tap writer now owns the final exact-main check immediately before git push for success, failure-report, and rollback mutations. Workflow callers pass the same SHA as recorded provenance, and a mocked branch-advance regression proves origin remains unchanged.
The immutable GitHub release primitive now requires exact-main authority and revalidates it immediately before release creation, each asset upload, direct-tag creation, and the draft-to-public PATCH. A simulated main advance after tag creation leaves a complete draft unpublished and emits no receipt.
Carry exact-main authority into the journaled canonical index state machine and revalidate it immediately before every release asset upload, patch, and delete. Keep general resolver candidate activation as a separate non-Homebrew contract, and cover a live-main advance between transaction steps.
Fail before package builds unless the composite action is running at the workflow workspace root, the declared repository matches the workflow repository, HEAD equals the declared source commit, and the checkout is clean. This prevents caller-supplied provenance from certifying different source bytes.
Move exact-main validation into every canonical release creation and archive deletion attempt. Treat authority loss as definitive rather than reconcilable GitHub ambiguity, and cover main advancing before CREATE and between DELETE attempts.
@brandonpayton
brandonpayton force-pushed the homebrew/abi42-main-first-final-qk044 branch from 084b2aa to 33ead48 Compare July 25, 2026 10:54
Add a bounded streaming xtask command that copies one named regular member only after the complete package archive validates. Reject unsafe paths, duplicates, links, corrupt or oversized inputs, and occupied outputs without exposing partial destination bytes.
@brandonpayton
brandonpayton marked this pull request as ready for review July 25, 2026 14:20
@brandonpayton brandonpayton added the ready-to-ship Maintainer attests the exact tested head and requests merge preparation. label Jul 25, 2026
@brandonpayton
brandonpayton marked this pull request as draft July 26, 2026 02:38
@brandonpayton
brandonpayton marked this pull request as ready for review July 26, 2026 02:38
@brandonpayton
brandonpayton marked this pull request as draft July 26, 2026 02:39
@brandonpayton
brandonpayton marked this pull request as ready for review July 26, 2026 02:39
@brandonpayton brandonpayton added the skip-staging-tests Opt into skipping the staging tests. This does not skip tests before merge. label Jul 26, 2026
@brandonpayton
brandonpayton force-pushed the homebrew/abi42-main-first-final-qk044 branch from 7f4ac13 to f9bcfae Compare July 26, 2026 02:41
@brandonpayton brandonpayton added ready-to-ship Maintainer attests the exact tested head and requests merge preparation. and removed ready-to-ship Maintainer attests the exact tested head and requests merge preparation. labels Jul 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

prepare-merge: test-gate passed against the synthetic PR merge and sealed merge-candidate-abi-v42-pr-1097-run-30185008038-attempt-1. The canonical ABI index is unchanged. merge-gate=success was posted on PR HEAD. This PR is ready for a reviewer to squash merge; default-branch reconciliation will verify the exact merged tree before activation.

@brandonpayton brandonpayton added preserve-head-commit Preserve the exact PR head with a merge commit for publication-pinned artifacts. ready-to-ship Maintainer attests the exact tested head and requests merge preparation. and removed ready-to-ship Maintainer attests the exact tested head and requests merge preparation. labels Jul 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

prepare-merge: test-gate passed against the synthetic PR merge and sealed merge-candidate-abi-v42-pr-1097-run-30188160612-attempt-1. The canonical ABI index is unchanged. merge-gate=success was posted on PR HEAD. This PR is ready for a reviewer to merge merge; default-branch reconciliation will verify the exact merged tree before activation.

@brandonpayton
brandonpayton merged commit c7d0397 into main Jul 26, 2026
178 checks passed
@brandonpayton
brandonpayton deleted the homebrew/abi42-main-first-final-qk044 branch July 26, 2026 12:10
brandonpayton added a commit that referenced this pull request Jul 26, 2026
The #1097 cache bridge exists for one reviewed rootfs/wasm32 archive closure. Without explicit code-level bounds, generic v2 inputs could reuse that proof for another producer, source capture, root set, architecture, or ABI. A preservation run from an old clean checkout could also write release state after main had moved.

Hard-bind the bridge to the reviewed #1097 producer, pr-1097-staging capture, schema-1 rootfs/wasm32 selection, and ABI 42. Recheck the direct main ref before tag creation, draft creation, every asset upload, the application seal, and the public transition, while keeping an exact already-public retry read-only. Remove unused pull-request write permission.

Cover unauthorized selections and main movement before the first write, between uploads, and after the seal. Clarify v2 recovery and producer terminology, and remove stale comments that misstated the projection contract.
brandonpayton added a commit that referenced this pull request Aug 2, 2026
Distinguish the particular retired #1097 evidence from the supported,
versioned cache-projection method. Document the canonical v2 Force
preservation path and the separate current-main admission boundary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

preserve-head-commit Preserve the exact PR head with a merge commit for publication-pinned artifacts. ready-to-ship Maintainer attests the exact tested head and requests merge preparation. retain-package-staging Keep an exact merged PR package-staging release until post-merge validation and promotion finish. skip-staging-tests Opt into skipping the staging tests. This does not skip tests before merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant