[Homebrew/ABI] Keep the exact ABI 42 bottle source in main history - #1092
Merged
Conversation
Require shell-derived images to match the reviewed capacity profile as well as its data and inode reserves. Keep a deliberate expected-capacity override for a future larger product profile. Make host-tree composition fail on every read, unsupported-entry, and VFS-write error; intentional omissions remain explicit excludes. Cover the complete copy-option surface, ENOSPC propagation, capacity drift, and larger-profile escape path.
Validate the serialized image capacity before compression or output writes, require intentional symlink handling, and propagate MariaDB test source failures. Add contract coverage for masked capacity, failed host reads, and shell profile constraints.
Bind the Homebrew main-shell serializer to its encoded capacity contract before output writes. Require every declared MariaDB test and fixture tree instead of preserving best-effort omissions, and remove the stale simple_select entry that the pinned source archive does not contain. Add executable failure-path, selection-parity, and package-input coverage.
Allow the static Formula planner to recognize three canonical native-tool Requirement classes without evaluating Formula source. Preserve the existing sealed homebrew/core identity plan, and reject unknown, dynamic, forged, unloaded, or test-only Requirement declarations.
Bind allowlisted Requirement classes, Formula identities, sentinel executables, and tags into a closed schema-4 host dependency plan. Validate that plan in every producer and consumer, reconstruct only matched build-only Homebrew dependencies for Superenv, and expose sealed test tools through the normal Formula test lifecycle. Cover malformed plans and evaluated-object drift, plus an actual install/test against the exact pinned Homebrew source.
Provision an isolated Bundler copy before the publisher boundary, then seal it and run the real pinned Homebrew install and test commands behind an OS-enforced network sandbox. Probe that boundary with a reachable control socket and fail if the gem tree changes or Bundler activity appears.
Carry the dev-shell PATH explicitly through the passwordless-sudo network namespace used by GitHub runners. This keeps sealed native Requirement sentinels visible after sudo applies secure_path, while retaining the OS-enforced offline boundary.
Document the closed schema-4 native Requirement contract, the offline pinned publisher lifecycle, and the remaining canonical-package activation gate. Keep the trusted publisher Homebrew revision distinct from the dedicated guest homebrew-bootstrap package so staging URLs are not mistaken for durable consumer inputs.
Generate the real pinned Homebrew lifecycle plan through the same static Formula parser and closed-schema validator used by publication before staging it for Build and test. Bound every host dependency list to the platform's 128-entry limit in both validation layers, and cover oversized plans so protected control data cannot grow beyond the reviewed graph contract.
Make product WordPress and LAMP builds resolve only their SHA-pinned source archives instead of invoking the unpacked local-demo setup path. Centralize the reviewed WordPress core copy policy, materialize the SQLite plugin from its separate pinned source, and keep every unrelated source symlink fail-closed. Cover the package entrypoint boundary, local-demo setup ownership, exact exclusions, plugin guest placement, unexpected symlinks, and refreshed package identities.
Derive one canonical typed-tree descriptor from an exact declared package ZIP output, then let image builds either preserve it as one first-use group or directly materialize that same descriptor and payload. Record source-tree versus runtime-tree distribution meaning so a tool source tree cannot be mistaken for a Homebrew bottle. Give Node the same relative lazy-asset URL resolution contract as the browser, preserve closed exact-byte acceptance, and teach Homebrew materialization evidence to distinguish its bottle groups from coexisting package trees without overlooking unexpected bottles. This changes no kernel or guest ABI.
Propagate real namespace lookup errors during package-tree preflight instead of treating every lstat failure as a missing path. Add focused package-tree coverage proving a digest-mismatched first fetch commits no member, remains retryable, and coalesces concurrent access into one fetch on both the failed attempt and the successful retry.
Pre-publication browser checks may read an artifact from a local or staging source while the VFS stores its final immutable HTTPS URL. Callers previously had to fetch and assemble that binding themselves, which made it too easy to associate incomplete or changed bytes with the trusted URL. Add a bounded loader that validates the complete source list before I/O, omits credentials, rejects redirects, streams exactly the declared length, verifies SHA-256, and only then returns canonical closed-asset bindings. Export the primitive and cover invalid identities, truncation, overflow, digest changes, aggregate limits, concurrency, and result ordering.
Abort peer fetches and wait for their response bodies to close when one verified source fails or the caller cancels. Validate and snapshot the full manifest before I/O, keep decoded bytes as the size authority, redact source queries from loader errors, and reject noncanonical URL fragments. Cover the contract with focused host tests and a real Chromium transport test for gzip decoding, omitted credentials, redirects, bounded streams, stream failure, and caller abort.
Bind each VFS Formula ordinary dependency closure to a fixed, URL-free manifest and rootfs payload inside its keg. Validate canonical payload ownership and preflight selected layers before the existing atomic runtime-layer consumer sees them. Deduplicate a shared dependency only when both layers resolve the exact same immutable bottle, link projection, and provenance. Record the remaining direct-bottle versus derived-rootfs transport choice without claiming publication or browser support.
Regenerate the authoritative program-package projection once after replaying the packaging, lazy package-tree, and bottle-owned VFS layer work onto the exact post-#1078 package-generation head. This keeps every consumer bound to the same complete source graph instead of preserving intermediate cache identities from the prior stacked branches.
Allocate every top-level process, fork child, spawn child, and pthread TID from one monotonic Rust ProcessTable sequence. Remove host-selected identity APIs, validate exact task/channel bindings, and make process/thread identity construction capability-based. Bump the incompatible host/kernel contract to ABI 42, update libc fork-child state, host adapters, package harnesses, generated ABI evidence, documentation, and Node/browser regression coverage.
Replace the fixed fork continuation capacity with ABI 42 linked, page-backed frame chunks shared across main-process, pthread, browser, Node, and dynamic-linker paths. Preserve truthful fatal cleanup on allocation failure for now and document recoverable ABORT_UNWINDING as the next POSIX-correct step.
Add ABI 42 ABORT_UNWINDING replay so a failed linked-frame allocation reconstructs committed inner frames, releases continuation ownership, and returns the original errno without terminating the process. Negative SYS_FORK results now use the complete parent rewind path as well. Main, pthread, and supported dynamic-side-module paths share the recovery contract; integrity and cleanup failures remain fatal. The P-10 stress fixture grows from 52,052 to 58,370 instrumented bytes (+6,318, +12.14%); runtime performance was not measured. The instrumenter suite, focused host/V8 suite, host build, and ABI consistency check pass. Browser validation was attempted but blocked by 35 missing package assets; the broader host suite also remains blocked by missing wasm64/sysroot and stale package artifacts.
Route main-process, pthread, and side-module begin exports through one pointer-width-aware boundary. V8 requires BigInt for wasm64 i64 arguments even when the address fits in a Number, so a shared helper prevents host paths from drifting. Exercise real i32 and i64 WebAssembly exports and make the abort-unwind fixture independent of the caller's working directory.
…ctory tests Update fork, spawn, procfs, and directory-cookie fixtures to obtain their process identities from ProcessTable and call the caller-validating APIs. This keeps the unit suite aligned with ABI 42 instead of reintroducing caller-selected PID shortcuts.
The ABI 42 process-local kill and raise helpers no longer accept HostIO. Remove unused mock hosts so the kernel suite remains warning-clean for the changed signal surface.
Prebuild the exact host xtask under the trusted workflow identity and scope its path only to bottle build and verification steps. Require both Formula runners to independently select that host binary, then expose only its read-only source alias to isolated Formula tests. Recheck the original inode and bytes on every entry so stale or replaced policy code fails closed. Cover the full binary-resolver surface, workflow trust mutations, caller/path/access replacement cases, and Linux systemd bind-mount behavior.
Forward the exact workflow-prepared xtask through each dev-shell command invocation instead of preserving it globally in dev-shell.sh. This keeps Formula execution least-authority while restoring the global package-toolchain input byte-for-byte, so existing package cache keys remain current. Trust mutations and an exact-source projection regression protect both sides of the contract.
Problem: Cargo normally hard-links the Linux release xtask to its hashed deps artifact, while the Formula isolation boundary requires one inode with no alternate alias. Implementation: seal Cargo’s exact bytes into a fresh mode-0555, single-link inode before both Formula build and verification. Keep the launcher’s stronger nlink policy, bind both workflows to one helper, and make trust mutations reject either seal bypass. Validation: the canonical publisher suite, workflow trust self-test, exact-source program projection, helper regression, and an exact linux/amd64 Cargo reproduction passed. The reproduction changed the checker from nlink 2/mode 755 to byte-identical nlink 1/mode 555. Not run: kernel, POSIX conformance, and browser suites are outside this publisher-only change. Live publication remains gated on a newly pinned Asa canary. Residual boundary: a concurrent attacker already running as the trusted workflow UID could race the writable Cargo target; untrusted Formula code has not run when sealing occurs.
The hosted runner keeps the reviewed Kandelo checkout below a workflow-private home directory. Requiring the isolated Formula identity to read the original checker path rejected that secure layout before Formula execution, even though the launcher deliberately hides the checkout and exposes the exact checker through a root-created read-only bind alias. Keep the checker mode, single-link, owner, non-writability, ancestor-replacement, inode-state, and SHA-256 checks. Remove only the redundant original-path read/execute requirement, retain the isolated alias audit, split the remaining failures into precise diagnostics, and model a mode-0700 runner home in the privileged regression. Validated with the complete Homebrew publisher contract suite and a privileged Ubuntu bind-mount proof. The exact hosted-runner path remains for the Asa canary after sealing and tap repinning.
The checker sealer guarantees mode 0555, but the launcher’s historical bitmask also accepted owner-only mode 0700. That made an unsafe-mode regression reach the later alias audit instead of failing at the pre-bind boundary. Require the exact 0555 sealer contract and bind that predicate in the publisher trust checker. This keeps the private-original/read-only-alias design while making every accepted checker mode deterministic. Validated with shell/Ruby syntax, the publisher trust checker, and diff checks. The privileged Linux/systemd launcher regression is the required next gate before tap repinning.
The production-shaped checker case intentionally hides the original checkout below a mode-0700 runner home. That also hid the negative test’s deliberately Formula-owned inner directory, so the replacement test could no longer observe the unsafe ancestor it was meant to reject. Make the source parent traverse-only for that one negative assertion, then restore mode 0700 before the positive read-only-alias path. This preserves both independent contracts without weakening production isolation. Validated with shell syntax and diff checks. The privileged Linux/systemd launcher regression remains the required next gate.
Keep the privileged launcher test faithful to the production checker contract. The test now makes only its private checker copy writable while changing or restoring bytes, and restores the exact 0555 seal before every launcher invocation.
Homebrew rebuilds the ordinary environment before running Formula tests, which drops WASM_POSIX_XTASK_BIN. Carry the launcher-validated checker through the HOMEBREW_* namespace, keep caller-selected aliases excluded, and prove the behavior with the pinned real Homebrew lifecycle.
A read-only bind preserves the workflow user as inode owner, which prevents tap support from authenticating the checker with a self-contained root-owned rule. Stage the validated bytes as one root-owned 0555 inode, bind that exact inode over the authoritative release path, and verify its mount, state, and bytes at command entry, final verification, and cleanup.
…tests Carry complete content-addressed package generations into the read-only Formula runtime instead of flattening resolver mirrors into identityless files. Share the portable staging helper with prepared CI workspaces, bind tap runners to the authenticated cache root, and make materialization the verifier’s final binaries writer. Harden the replacement transaction against partial backup deletion and cover noncanonical links, escaping generation content, relocation, order changes, and rollback failures.
Treat exact static Requirement declarations as references into the already-bound Formula support tree instead of rejecting every additional support-module token. Keep authority over Requirement classes, tags, AST placement, and support definitions in the Ripper parser, and cover the exact Asa shape plus dynamic-reference rejection in the publication contract.
A package archive can be present but unusable, including when a same-run staging overlay has a stale cache identity. In that case the resolver must be able to run the shell recipe from any direct or transitive caller without caller-specific npm setup. Create a resolver-owned Git source snapshot, install both locked JavaScript dependency trees there with a scrubbed public-registry npm environment, and run the composer only from that private snapshot. Isolate every snapshot Git command from ambient attributes, replacement refs, selectors, and external diff configuration. Remove predictive setup from CI and run.sh, bind the preparer into the package cache identity, and cover stale-overlay fallback, concurrent builds, path substitution, Git/npm configuration, and credential isolation. Regenerate the authoritative program-package projection so shell and every composite package that depends on it carry the new manifest and cache identities.
Repository dispatch can be admitted before one protected-main update but instantiated after another, so the workflow execution SHA is not a stable record of the requested Formula source. Require publish and maintenance rebuild dispatches to carry one exact lowercase tap commit, verify the checkout matches it, and prove it remains on protected main before any package planning. Preserve the reviewed Kandelo main-or-exact-ABI-bootstrap source contract, leave rollback on current protected state, document the boundary, and extend behavioral and frozen-workflow tests for missing, mutable, detached, mismatched, and ancestry-unprovable tap sources.
MemoryFS already kept custom no-signal fetchers on the historical one-argument callback path, but its default wrapper forwarded an explicit undefined initializer to globalThis.fetch. That changed the observable callback shape and broke lazy archive consumers that distinguish initialized transports. Call the platform fetch with one argument when no initializer exists, retain the two-argument path for explicit cancellation, and document why the branch is part of the transport contract.
brandonpayton
force-pushed
the
packaging/producer-batch-abi42-v2-qk044
branch
from
July 24, 2026 18:31
29f976c to
7d8929e
Compare
brandonpayton
marked this pull request as ready for review
July 24, 2026 18:47
brandonpayton
added
the
preserve-head-commit
Contributor
Phase B-1 matrix build status —
|
| Package | Arch | Status | Sha |
|---|---|---|---|
| icu | wasm32 | built | 48a3ee79 |
| libcurl | wasm32 | built | 8273da44 |
| libcxx | wasm32 | built | 53612e1c |
| libcxx | wasm64 | built | 986b4977 |
| libiconv | wasm32 | built | b494e26a |
| libpng | wasm32 | built | e31da77e |
| libxml2 | wasm32 | built | f8e7709d |
| libzip | wasm32 | built | 0e020e8d |
| openssl | wasm32 | built | bdef2279 |
| openssl | wasm64 | built | 11532ba6 |
| sqlite | wasm32 | built | a21f5097 |
| sqlite | wasm64 | built | 1f7a45fb |
| zlib | wasm32 | built | a826c5a1 |
| zlib | wasm64 | built | 09bd3336 |
| bc | wasm32 | built | 87d9afde |
| bzip2 | wasm32 | built | 09122fb0 |
| coreutils | wasm32 | built | 72765db8 |
| cpython | wasm32 | built | 314d4763 |
| curl | wasm32 | built | 63cca65e |
| dash | wasm32 | built | 0b2035e2 |
| diffutils | wasm32 | built | 5b829e0f |
| dinit | wasm32 | built | 28167e43 |
| erlang | wasm32 | built | fb022926 |
| fbdoom | wasm32 | built | ad115c1c |
| file | wasm32 | built | 41f1cbea |
| findutils | wasm32 | built | 9cf821c8 |
| gawk | wasm32 | built | f25ff8e1 |
| git | wasm32 | built | 956b61b3 |
| grep | wasm32 | built | 2b95ab13 |
| gzip | wasm32 | built | c5fea74e |
| homebrew-bootstrap | wasm32 | built | 3f44ee7f |
| kandelo-sdk | wasm32 | built | 7678c7f7 |
| kernel | wasm32 | built | d0e03c4b |
| less | wasm32 | built | 69fd5aea |
| lsof | wasm32 | built | 0cbea55d |
| m4 | wasm32 | built | 746a3cb4 |
| make | wasm32 | built | 8bbaab34 |
| mariadb | wasm32 | built | 89cb71f5 |
| mariadb | wasm64 | built | f6668fed |
| modeset | wasm32 | built | 8a5f567e |
| msmtpd | wasm32 | built | 94c0ffb6 |
| nano | wasm32 | built | 48c6e796 |
| ncurses | wasm32 | built | e95f6769 |
| netcat | wasm32 | built | 3d57ef27 |
| nginx | wasm32 | built | cc1c9c86 |
| php | wasm32 | built | 54c56075 |
| posix-utils-lite | wasm32 | built | 81206dd7 |
| ruby | wasm32 | built | b9554136 |
| sed | wasm32 | built | 6265486b |
| shell | wasm32 | failed | — |
| spidermonkey | wasm32 | built | 2c2b1f59 |
| tar | wasm32 | built | d842e794 |
| tcl | wasm32 | built | edae6679 |
| unzip | wasm32 | built | f4ee34b5 |
| userspace | wasm32 | built | 6538ad13 |
| vim | wasm32 | built | 39d28fe5 |
| wget | wasm32 | built | 222061a6 |
| xz | wasm32 | built | 71047565 |
| zip | wasm32 | built | a73d281e |
| zstd | wasm32 | failed | — |
| bash | wasm32 | failed | — |
| lamp | wasm32 | failed | — |
| mariadb-test | wasm32 | failed | — |
| mariadb-vfs | wasm32 | failed | — |
| mariadb-vfs | wasm64 | failed | — |
| nethack | wasm32 | failed | — |
| nginx-php-vfs | wasm32 | failed | — |
| nginx-vfs | wasm32 | failed | — |
| node | wasm32 | failed | — |
| redis-vfs | wasm32 | failed | — |
| spidermonkey-node | wasm32 | failed | — |
| vim-browser-bundle | wasm32 | failed | — |
| wordpress | wasm32 | failed | — |
| nethack-browser-bundle | wasm32 | failed | — |
| node-vfs | wasm32 | failed | — |
| rootfs | wasm32 | failed | — |
Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.
Join the reviewed bottle-producing branch to current main without replaying its already-landed tree. This keeps the exact published source commits reachable while the consumer shell cutover remains in its dedicated product batch.
brandonpayton
changed the title
Member
Author
|
Superseded publication guidance (2026-07-25): retaining a producer commit in the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Kandelo's public ABI 42 bottles record the exact source commit
d3805721…. That source is still reachable from this pull request, but it is not yet part ofmain. A normal squash or rebase would rewrite the history and leave the published provenance pointing outside the durable main-line history.What this PR does
This PR now has one narrow purpose: keep the exact published producer lineage in
mainwithout restoring stale product files.maincommit to the previous exact PR head.main; it changes history, not product behavior.437fde252…,d3805721…, and3545bfd34…remain ancestors of the new PR head.The prior version of this PR also carried an older shell snapshot. That product state is intentionally not replayed. The current Homebrew consumer configuration, mostly-lazy shell, and in-guest
brewproof remain in #1087.Validation
09ed47951… 3545bfd34….maintree.437fde252…,d3805721…, and3545bfd34…are ancestors of the PR head.docs/binary-releases.md.git diff --checkpassed.This is a documentation-and-history change. It does not claim fresh runtime, browser, package-build, or POSIX validation.
Merge requirement
Keep the
preserve-head-commitlabel and merge this PR with a merge commit. Before merging,mainmust still be09ed47951…; if it advances, the ancestry join and merge preparation must be refreshed. After merging, we will verify the exact ordered parents, tree, and producer ancestry before rotating the trusted tap callers.