Skip to content

[Homebrew/Python] Build CPython and its standard library inside the core tap - #124

Draft
brandonpayton wants to merge 9 commits into
mainfrom
packaging/python-schema3-qk044
Draft

[Homebrew/Python] Build CPython and its standard library inside the core tap#124
brandonpayton wants to merge 9 commits into
mainfrom
packaging/python-schema3-qk044

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Why

Python still depends on Kandelo's legacy package registry for its source
build. That prevents the core tap from owning the complete recipe and
blocks retirement of the registry entry.

Cross-compiling CPython also requires a native Python from the same
major/minor release. That interpreter must be a declared, sealed build
input. An ambient host Python would make the bottle depend on whichever
tool happened to be installed on the CI runner.

What changed

  • Build the target CPython 3.13.3 interpreter from the Formula's
    checksum-verified upstream source.
  • Declare Homebrew python@3.13 as the native build interpreter, using
    CPython's supported --with-build-python contract.
  • Require that interpreter to resolve inside the declared
    Cellar/python@3.13/<version> keg, be read-only, and report version
    3.13.
  • Link the target interpreter against the core tap's zlib Formula.
  • Instrument and validate the interpreter for Kandelo's fork and ABI
    contracts.
  • Package the standard library deterministically beside python,
    python3, and python3.13.
  • Exercise the installed interpreter, standard library, and zlib in
    both Node and Chromium.

Python remains wasm32-only. This change does not add a language-specific
VFS image; lazy image composition can select the Python bottle.

Closed build boundary

The Kandelo runner seals the native Homebrew keg and its complete
dependency closure before the recipe starts. The recipe then verifies
that python3.13 did not fall back to /usr or another ambient path.

The runner also supplies target configuration and fork instrumentation
from the exact Kandelo revision chosen by the publisher. Formula or
recipe code cannot substitute a checkout or a different tool.

The recipe copies the read-only sysroot into private working storage and
opens only that copy before adding CPython's required WASI emulation
archives.

Inline WHY comments preserve these non-obvious authority and
reproducibility decisions.

Validation

  • The Python migration contract passes.
  • brew style Formula/python.rb reports no offenses.
  • Ruby and shell syntax checks pass.
  • ShellCheck reports no error-level findings.
  • git diff --check passes.
  • Formula support passes 128 tests and 2,285 assertions.
  • Publisher trust checks pass.
  • All 19 trust-rotation tests pass.

Dry-run investigation established three separate packaging boundaries:

  1. A Homebrew LLVM keg contained a link outside its sealed closure.
  2. The private sysroot copy retained the source's read-only mode.
  3. Using target LLVM as a native compiler selected an unavailable host
    loader.

The recipe now uses the declared native python@3.13 dependency instead
of attempting that invalid compiler bootstrap. The first attempt of dry
run 30469839479, on runner image 20260726.254.1, rejected the
runner's unconditional /usr projection before the recipe started.
That is runner-instance evidence, not a stable runner contract; this PR
does not weaken the rejection or expose more host state.

The exact retry and exact-head control 30471174839, both on runner
image 20260720.247.2, crossed that boundary. The control installed the
declared Cellar/python@3.13/3.13.14_1 keg, passed the keg-path
assertion, and built the target bottle: 750 files and 28 MB in 3 minutes
39 seconds.

Both builds then reached the shared Formula runtime and failed before
Python executed because programs/dash.wasm had no matching selected
package projection. They used Kandelo commit 85d2efc, which predates
the central resolver fix in Automattic/kandelo#1136. Further expensive
retries are parked until Automattic/kandelo#1138 lands and the tap
rotates trust to its new exact publisher commits.

The existing bottle block remains last-green evidence until an exact
protected build replaces it. This draft must still pass the live Node
and Chromium Formula lifecycle before merge.

@brandonpayton
brandonpayton changed the base branch from main to homebrew/close-shell-demo-recipes-qk044 July 29, 2026 04:08
@brandonpayton
brandonpayton force-pushed the packaging/python-schema3-qk044 branch from 84c1c5b to bfd066e Compare July 29, 2026 04:08
@brandonpayton
brandonpayton force-pushed the homebrew/close-shell-demo-recipes-qk044 branch 3 times, most recently from e705a19 to a9bddb5 Compare July 29, 2026 12:13
@brandonpayton
brandonpayton force-pushed the packaging/python-schema3-qk044 branch from bfd066e to fe690f8 Compare July 29, 2026 15:03
@brandonpayton
brandonpayton changed the base branch from homebrew/close-shell-demo-recipes-qk044 to main July 29, 2026 15:03
@brandonpayton
brandonpayton force-pushed the packaging/python-schema3-qk044 branch from fe690f8 to 848fe93 Compare July 29, 2026 15:04
Consume the SDK config-site path that the privileged Kandelo runner
derives from its sealed platform projection. Reject any build-script
assignment so Formula-controlled recipe code cannot redirect shared
target facts back to broad repository authority.

Refresh the attested recipe records and Formula manifest identity.
The current recipe runner projects fork instrumentation directly and
withholds the broad Kandelo checkout. Use that executable, remove the
obsolete checkout-derived prefix map, and lock the expanded compiler
wrapper checks into Python's authenticated recipe manifest.
The protected recipe runner already supplies an attested LLVM projection. Use its exact paths for CPython native generators instead of adding Homebrew LLVM, whose prefix-config symlink correctly falls outside the closed native dependency projection.
cp -a preserves the sealed sysroot directory modes. Make the recipe-owned lib copy writable before adding CPython’s required WASI emulation archives, while leaving the publisher-owned source untouched.
CPython supports --with-build-python from any interpreter with the same
major and minor version. Use the sealed python@3.13 dependency for that
role instead of treating the Kandelo target LLVM as a native compiler.
@brandonpayton
brandonpayton force-pushed the packaging/python-schema3-qk044 branch from 6026b83 to 21a1b07 Compare July 29, 2026 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant