homebrew: publish tap-native formulas without registry duplicates

[brandonpayton]

Purpose

Make tap-owned Homebrew Formulae first-class Kandelo packages: derive durable Kandelo sidecars from Homebrew bottle/archive facts, remove the need for duplicate packages/registry/<name> recipes, and serialize only the final lossless composition.

This is a main-repository platform PR for Brandon's review. Do not auto-merge it. Formulae remain owned by Automattic/kandelo-homebrew.

Stack And Commit Boundary

This PR is one commit on the reviewed publication-security stack:

• ci: isolate and verify Homebrew bottle publication #881 (bca9393a34abb786419fbbdcea032d3c46e7b7c5): fresh-runner credential/data handoffs and cache isolation.
• homebrew: preserve bottle compatibility for host build dependencies #887 (1e819fa6c02129698f9fd6fb0b528dca135bea86): prefix-preserving patched Homebrew launcher.
• homebrew: trust same-tap dependencies during bottle builds #890 (0a23dfbc8d2752d5ecfb37d46ed5d0f9f0530649): isolated XDG state and explicit selected-tap trust.
• homebrew: publish tap-native formulas without registry duplicates #885 head: d2696657d0a75a66c1b6bacbf7e346a2769f27a2.
• Exact parent/base: 0a23dfbc8d2752d5ecfb37d46ed5d0f9f0530649 (fix/homebrew-tap-trust).

Root Cause

The sidecar generator previously required duplicate registry metadata for identity, dependencies, links, cache identity, and fork disposition even though Homebrew's reviewed Formula, rich bottle JSON, archived Formula, receipt, and bottle archive already own those facts.

Parallel matrix jobs also generated full tap payloads before taking the state lock. A later job could overwrite refreshed peer packages or sibling architectures with a stale aggregate, and carried provenance could be rewritten as if old bottles came from the newest build.

Fresh-Runner Roles

After read-only planning, four execution domains use fresh runners and strict artifacts:

1. build-and-test has read-only contents/actions access. It runs Formula/upstream code and emits only the strict build handoff.
2. upload-bottle has packages: write on a fresh runner. It validates inert build data, uploads with isolated ORAS credentials, and emits a strict receipt.
3. verify-bottle is read-only on a fresh runner. It anonymously reads back the published digest, runs runtime/browser verification, evaluates reviewed Formula/Homebrew only here, and emits package-scoped composition data.
4. finalize-tap has contents: write on a fresh runner. It validates inert handoffs before credentials, then performs only trusted static composition. It never evaluates Homebrew, Formula, or package code.

Changes

• Derive Formula identity, version/revision, direct runtime dependencies and receipt versions, link plans, fork instrumentation, and cache identity from tap-native Homebrew facts.
• Bind built_from.formula_sha256 to the exact archived .brew/<formula>.rb; reject archive/source drift and preserve carried bottles' original build provenance.
• Use Homebrew bottle --merge --keep-old only in the read-only verifier. The credentialed finalizer does not consume Homebrew output as commands.
• Publish the strict data-only handoff: build/{manifest.json,bottle.json,bottle.tar.gz}, receipt.json, and composition/sidecars-input.json.
• Under the tap lock, refresh main, prove planned ancestry and bottle-excluded Formula source equivalence, statically compose the selected tag, regenerate aggregate sidecars, validate, then push.
• Preserve a sibling architecture only when refreshed metadata proves the exact same ABI, package name, version, Formula revision, and bottle rebuild. Identity transitions discard old tags; a later same-identity architecture preserves the newly published peer.
• Require Formula root/rebuild/tag/digest parity with sidecars. The root is exactly https://ghcr.io/v2/<lowercase tap_repository>.
• Reuse the trusted Ripper structural guard so extra or executable/noncanonical bottle calls fail without evaluating Formula Ruby.
• Reject symlinks and special objects throughout refreshed/staged Formula/ and Kandelo/ trees, and stage replacements before modifying the credentialed tap checkout.
• Preserve unrelated peer packages and original carried provenance while refreshing aggregate cross-file hashes.
• Remove registry duplication from the tap-native path and document the new ownership, composition, and validation contracts.

Regression Coverage

scripts/test-homebrew-tap-native-sidecars.sh builds tap-native dependency/consumer fixtures with no registry entries. It seeds an old version/revision/rebuild wasm64 bottle, proves the first wasm32 publication drops that stale sibling and validates a one-architecture aggregate, then proves the second wasm64 publication from the same original plan preserves the new wasm32 bottle and validates both architectures. It also covers dependency derivation, source drift rejection, and a Node-built VFS pour.

Workflow tests cover strict handoff grammar, fresh-runner trust topology, static preserve/discard composition, and tap symlink rejection without writing through the external target. Rust tests cover archived Formula hashing, immutable carried provenance, Formula root/rebuild/tag/digest parity, and rejection of extra noncanonical bottle calls.

Validation

Run through scripts/dev-shell.sh on aarch64-apple-darwin at the exact head above:

• cargo test -p xtask --target aarch64-apple-darwin homebrew: 23 passed.
• bash scripts/test-homebrew-publish-workflow.sh: passed.
• bash scripts/test-homebrew-tap-native-sidecars.sh: passed.
• trust checker, Bash syntax, Ruby syntax, workflow YAML parse, and git diff --check: passed.
• Independent devil's-advocate re-audit: no blocking findings; it independently reran the focused gates and rechecked the exact clean head.

The broader cargo test -p xtask --target aarch64-apple-darwin run was 311/318. Seven unrelated existing fixture failures remain in archive_stage_cli::cli_produces_archive_with_canonical_filename and build_deps binaries-dir/resolve tests; every Homebrew-focused test passed. shellcheck and cargo-fmt are unavailable in the repository dev shell.

Contract Scope

This changes trusted Homebrew publication and metadata semantics. It does not change the Kandelo ABI, syscall surface, host/browser runtime, package archive format, or any tap Formula. No ABI bump, snapshot update, package revision, bottle upload, or generated tap state is included.

[brandonpayton]

Devil-review blockers from the first real ABI 18 trusted bottle dry run:

1. declared_directly in Homebrew's bottle receipt is not the formula-declaration boundary this code assumes. Run 29166185779 built/tested/bottled tap-main SQLite at 78ab4e2 against merged Kandelo 2c083e1; bottle SHA-256 is 345f84dc0d25352890a1c92710f9bdc46e1996a1744f790bb659755c8b8ce92f. The authoritative SQLite formula and package.toml declare no dependencies, but Homebrew 6.0.9 injected Linux sandbox dependencies into INSTALL_RECEIPT.json:

[
  {"full_name":"libcap","declared_directly":false},
  {"full_name":"bubblewrap","declared_directly":true}
]

Executing this PR's exact homebrew-generate-sidecars-from-env.sh at b6be88870 against the downloaded CI bottle fails:

direct runtime dependency 'bubblewrap' is not a formula in automattic/kandelo-homebrew

Please derive the formula's declared runtime dependencies through Homebrew's structured formula API and use the receipt to resolve/verify those declarations. Do not special-case bubblewrap, and do not treat every receipt entry marked declared_directly as a package dependency.

2. The same exact script can exit 141 before generation on a developer Homebrew checkout because it runs brew --version | head -n 1 under set -o pipefail; Homebrew emits its version plus tap revisions and receives SIGPIPE. I reproduced this through scripts/dev-shell.sh with Homebrew 6.0.9-94. Use a first-line extraction that consumes the complete producer output.

The CI bottle itself has exactly the declared SQLite library payload (archive, headers, pkg-config, receipts); the current main publisher's bin/sqlite rejection is the stale registry-derived link-plan bug this PR is meant to replace.

[github-actions]

Phase B-1 matrix build status — pr-885-staging

ABI v18. 69 built, 0 failed, 69 total.

Package	Arch	Status	Sha
libcurl	wasm32	built	b16dfa55
libcxx	wasm32	built	a6a226bf
libcxx	wasm64	built	05cc04c6
libpng	wasm32	built	fb0072b7
libxml2	wasm32	built	7693dffe
libxml2	wasm64	built	0e9bdba6
openssl	wasm32	built	6079e850
openssl	wasm64	built	96e12f2f
sqlite	wasm32	built	5079ecc6
sqlite	wasm64	built	41e589af
zlib	wasm32	built	02809e96
zlib	wasm64	built	74a9b521
bc	wasm32	built	e10365d5
bzip2	wasm32	built	78f873f3
coreutils	wasm32	built	198092d9
curl	wasm32	built	db51ebb0
dash	wasm32	built	01910f46
diffutils	wasm32	built	c730ca1d
dinit	wasm32	built	7fa2cfc1
fbdoom	wasm32	built	0560d7dc
file	wasm32	built	0285fc03
findutils	wasm32	built	7d4a601c
gawk	wasm32	built	69d7773b
git	wasm32	built	de2da6a2
grep	wasm32	built	41d5f22d
gzip	wasm32	built	cf0f7b2f
hello	wasm32	built	0e7165f9
kandelo-sdk	wasm32	built	911d8907
kernel	wasm32	built	3a942356
less	wasm32	built	a926fd81
lsof	wasm32	built	9415a3e3
m4	wasm32	built	f653ae19
make	wasm32	built	9cba934d
mariadb	wasm32	built	efe83b04
mariadb	wasm64	built	8238b3ea
modeset	wasm32	built	54395757
msmtpd	wasm32	built	418b5b1b
nano	wasm32	built	35094cd6
ncurses	wasm32	built	95f1807e
netcat	wasm32	built	bc14b930
nginx	wasm32	built	b728d91a
php	wasm32	built	1100861d
posix-utils-lite	wasm32	built	2ba7af8b
ruby	wasm32	built	646b371d
sed	wasm32	built	4bb4d852
spidermonkey	wasm32	built	5e0492a5
tar	wasm32	built	b41c7bad
tcl	wasm32	built	889ee16d
unzip	wasm32	built	54d97756
userspace	wasm32	built	325ab8bf
vim	wasm32	built	8e11f39b
wget	wasm32	built	88792aab
xz	wasm32	built	f9fecf97
zip	wasm32	built	c26954c3
zstd	wasm32	built	8fed2df5
bash	wasm32	built	28a7bea0
mariadb-test	wasm32	built	02fd8177
mariadb-vfs	wasm32	built	9cc3a58b
mariadb-vfs	wasm64	built	f5340c57
nethack	wasm32	built	86fd6eea
node	wasm32	built	abc37eda
spidermonkey-node	wasm32	built	121e7aa3
vim-browser-bundle	wasm32	built	abedb1b1
nethack-browser-bundle	wasm32	built	2b1fabb6
rootfs	wasm32	built	3cb8e7f3
shell	wasm32	built	fe1e6492
lamp	wasm32	built	8f3681cb
node-vfs	wasm32	built	929cf9bb
wordpress	wasm32	built	1bb5f5a5

_{Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.}

[brandonpayton]

Superseded by #936, which retains this PR as patch-equivalent purpose commits in the consolidated Homebrew publisher batch and reruns the combined validation on current main.