Skip to content

sdk: centralize cross-compile function availability - #874

Closed
brandonpayton wants to merge 3 commits into
mainfrom
fix/sdk-config-absent-functions
Closed

sdk: centralize cross-compile function availability#874
brandonpayton wants to merge 3 commits into
mainfrom
fix/sdk-config-absent-functions

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 11, 2026

Copy link
Copy Markdown
Member

Why

The software development kit (SDK) tells package build tools what Kandelo supports. Autoconf is the common configure-time tool that tests those capabilities, and musl is Kandelo's C standard library.

Make the Kandelo SDK's Autoconf target description authoritative enough for dependency-first Homebrew builds, instead of requiring each formula to repeat target availability overrides.

Kandelo's Wasm link path permits undefined imports. A conventional AC_CHECK_FUNC link probe can therefore report yes for a function absent from musl and the SDK archives; legacy probes can also report false negatives for exported functions by calling them with invalid signatures. GNU tar, Coreutils, Dash, Bash, Diffutils, Gettext, Bison, and the dependency roots exposed both directions during the tap migration.

Dynamic loading has another false-positive path: musl's weak dlfcn stubs link but only report "Dynamic loading not supported." An uncached AC_SEARCH_LIBS(dlopen) therefore returns none required, omits -ldl, and prevents the SDK compiler wrapper from selecting Kandelo's functional Wasm dlopen glue.

What changed

  • record three verified-present musl functions: bsd_signal, isalpha, and stpcpy
  • record 39 verified-absent extension functions used by dependency and leaf package waves
  • include the final formula-surface audit results: Windows GetSystemTimeAsFileTime, FreeBSD cap_rights_limit, and glibc feenableexcept/obstack_printf
  • record verified-working strerror(0) and strcasecmp behavior so gnulib consumers neither select unnecessary replacements nor abort while cross-configuring
  • route AC_SEARCH_LIBS checks for dlopen, dlsym, dlclose, and dlerror through -ldl, which the SDK translates to libc/glue/dlopen.c
  • keep unadorned ac_cv_func_dlopen=no, report the explicit SDK library as ac_cv_lib_dl_dlopen=yes, and remove contradictory duplicate defaults
  • preserve caller overrides with ${var=default}, with automated coverage for defaults and overrides
  • document that reusable sysroot/toolchain availability belongs in sdk/config.site; package recipes own only package-specific runtime, dependency-library, or semantic probes

No syscall behavior, host behavior, Wasm layout, ABI surface, generated artifact, Node path, or browser path changes.

Validation

Current stacked-head validation through scripts/dev-shell.sh on exact head 128a3f714fe5583cd25210f4e899b12258b3eb7f:

  • SDK Vitest suite: 7 files, 73 tests passed, including sdk: preserve static linker argument order #913 linker ordering, config-site sourcing, caller-override coverage, -ldl selection, and compile integration
  • scripts/check-abi-version.sh: ABI snapshot, C header, generated TypeScript bindings, and ABI version remained in sync
  • git diff --check c6849bde6..HEAD: passed

The target-fact changes are unchanged by the #913 restack. Consumer and symbol-audit evidence recorded on the prior reviewed head includes:

  • every added absent function was compared with all wasm32 sysroot archives; none exports the named symbol
  • the final tap/open-formula audit classified library-dependent probes such as OpenSSL/ncurses and package-semantic probes such as PHP syslog as formula-owned rather than moving them into the SDK
  • GNU Bash 5.2.37 configure/link integration using only package-specific facts produced the asserted target configuration
  • GNU Wget 1.25.0 built with Gnulib's real group_member replacement instead of its obsolete always-false package stub
  • GNU Make 4.4.1 selected the SDK -ldl glue and invoked a side-module callback
  • GNU Coreutils 9.6, GNU tar 1.35, and Dash 0.5.12 built without formula-local reusable availability overrides and passed focused runtime coverage
  • pristine GNU Diffutils 3.12 resolves both string checks as cached yes and configures without package-local function facts
  • the current tap Cflow formula also built without this PR by truthfully selecting Gnulib's strerror replacement, proving these facts optimize and centralize target identity rather than masking a required replacement

Kernel, libc-conformance, POSIX, and browser suites were not rerun because this supplies build-time Autoconf cache facts only. Consumer executables were exercised through Kandelo; no bottles or package archives are published by this PR.

Merge order

This PR is stacked directly on fully-green #913. Merge #913 first, then retarget this PR to main; its two purpose commits remain the complete review diff.

Review state

Current head: 128a3f714fe5583cd25210f4e899b12258b3eb7f, based on exact #913 head c6849bde6cfbe51837227789840ecf2b05791070, on ABI 39. This is a main-repository PR for Brandon's review and must not be auto-merged.

Formula source acceptance can proceed in parallel, but final bottles must be rebuilt after this PR lands so they consume SDK-owned facts rather than duplicated formula overrides.

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Phase B-1 matrix build status — pr-874-staging

ABI v39. 71 built, 0 failed, 71 total.

Package Arch Status Sha
icu wasm32 built 7adac0d6
libcurl wasm32 built 09364632
libcxx wasm32 built eb8b0f23
libcxx wasm64 built 98408792
libiconv wasm32 built 33ebf32a
libpng wasm32 built f6b060eb
libxml2 wasm32 built ed004619
libzip wasm32 built 3a18fa37
openssl wasm32 built 2534f50d
openssl wasm64 built b1f606da
sqlite wasm32 built 38d1e709
sqlite wasm64 built 74ba7df5
zlib wasm32 built ed1f0382
zlib wasm64 built fe65fab5
bc wasm32 built 74fe9422
bzip2 wasm32 built fbf3db6e
coreutils wasm32 built 4ad93277
curl wasm32 built 8d9826d0
dash wasm32 built 6aa80963
diffutils wasm32 built 157e6de9
dinit wasm32 built 32700f6b
fbdoom wasm32 built adfb0c3d
file wasm32 built dd352e11
findutils wasm32 built 7c850a20
gawk wasm32 built 97df82fe
git wasm32 built 6c1ba9dd
grep wasm32 built 16fdbb01
gzip wasm32 built 21778374
hello wasm32 built 7f3a5ebd
kandelo-sdk wasm32 built 2e05e690
kernel wasm32 built 5ad0bc5c
less wasm32 built c1d48473
lsof wasm32 built 141d3a95
m4 wasm32 built 8fb73304
make wasm32 built 8255da12
mariadb wasm32 built d7742568
mariadb wasm64 built c844b652
modeset wasm32 built bc6ee39f
msmtpd wasm32 built 6eecbf15
nano wasm32 built 9a4f539b
ncurses wasm32 built 0ad43612
netcat wasm32 built 04b88dfd
nginx wasm32 built 8b0eaf1d
php wasm32 built 65dc7d82
posix-utils-lite wasm32 built fe0864d3
ruby wasm32 built 7b786f57
sed wasm32 built 290ef527
spidermonkey wasm32 built 3934d30d
tar wasm32 built c18feaf9
tcl wasm32 built 50406646
unzip wasm32 built 007b2902
userspace wasm32 built a0479d3e
vim wasm32 built 116c55b3
wget wasm32 built 4aa9cc71
xz wasm32 built 540b9422
zip wasm32 built 59ba947e
zstd wasm32 built 5b01d1d5
bash wasm32 built 3d825116
mariadb-test wasm32 built 5f58870f
mariadb-vfs wasm32 built 8b85364a
mariadb-vfs wasm64 built 67f214cb
nethack wasm32 built c9baab43
node wasm32 built 64a3eb03
spidermonkey-node wasm32 built 4c31f9e3
vim-browser-bundle wasm32 built 655462a3
nethack-browser-bundle wasm32 built 1200fdd1
rootfs wasm32 built f73d1843
shell wasm32 built 37b5964d
lamp wasm32 built 729d141a
node-vfs wasm32 built 5cfb8cd8
wordpress wasm32 built 1252eb49

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

@brandonpayton

Copy link
Copy Markdown
Member Author

Wget 1.25.0 provides concrete downstream evidence for the new group_member target fact in this PR.

Without the shared ac_cv_func_group_member=no fact, Autoconf links a probe successfully because the Kandelo linker permits unresolved Wasm imports, even though musl provides neither the declaration nor the symbol. Wget then suppresses its bundled gnulib replacement and fails to compile at the undeclared group_member call.

With this PR stacked unchanged in the ABI 18 integration:

  • configure reports checking for group_member... (cached) no
  • generated config has HAVE_GROUP_MEMBER 0
  • the build compiles libgnu_a-group-member.o
  • the unmodified upstream Wget source completes its Homebrew source build
  • the final fork-instrumented artifact passes the literal Node Homebrew test and exact Chromium HTTPS retrieval

This is a target capability fact, not a Wget-specific cache override; the Wget formula retains only its package-specific strerror(0) runtime probe result.

brandonpayton added a commit to Automattic/kandelo-homebrew that referenced this pull request Jul 11, 2026
GNU Make's loadable-object API was disabled even though Kandelo supports Wasm side modules. Install the public header, mark the documented gmk API as visible, and exercise a compiled module that registers and runs a custom Make function.

The runtime path depends on the SDK's explicit -ldl configure fact and the host's main-program handle support, so the tap PR remains gated on Automattic/kandelo#874 and #879.
@brandonpayton

Copy link
Copy Markdown
Member Author

Libmagic supplied a focused regression probe for the existing fmtcheck target fact.

The file 5.45 cross configure uses AC_REPLACE_FUNCS(fmtcheck). Without the canonical ac_cv_func_fmtcheck=no, Kandelo's unresolved-import link behavior can make that probe appear successful even though musl provides neither a declaration nor a symbol; compilation then fails in softmagic.c instead of selecting upstream's replacement implementation.

Commit 7e0c99881 adds focused coverage that:

  • reads ac_cv_func_fmtcheck=no from sdk/config.site;
  • proves a caller-provided result still overrides the default.

Validation through the repository dev shell:

  • npx vitest run sdk/test/configure.test.ts: 7/7 passed
  • npm --prefix sdk test: 57/57 passed
  • git diff --check: passed

No ABI or generated artifact changes.

brandonpayton added a commit to Automattic/kandelo-homebrew that referenced this pull request Jul 11, 2026
GNU Make's loadable-object API was disabled even though Kandelo supports Wasm side modules. Install the public header, mark the documented gmk API as visible, and exercise a compiled module that registers and runs a custom Make function.

The runtime path depends on the SDK's explicit -ldl configure fact and the host's main-program handle support, so the tap PR remains gated on Automattic/kandelo#874 and #879.
brandonpayton added a commit to Automattic/kandelo-homebrew that referenced this pull request Jul 12, 2026
GNU Make's loadable-object API was disabled even though Kandelo supports Wasm side modules. Install the public header, mark the documented gmk API as visible, and exercise a compiled module that registers and runs a custom Make function.

The runtime path depends on the SDK's explicit -ldl configure fact and the host's main-program handle support, so the tap PR remains gated on Automattic/kandelo#874 and #879.
@brandonpayton
brandonpayton force-pushed the fix/sdk-config-absent-functions branch from 7e0c998 to b23f9f3 Compare July 12, 2026 01:15
brandonpayton added a commit to Automattic/kandelo-homebrew that referenced this pull request Jul 12, 2026
* make: add GNU build automation formula

* make: enable loadable object support

GNU Make's loadable-object API was disabled even though Kandelo supports Wasm side modules. Install the public header, mark the documented gmk API as visible, and exercise a compiled module that registers and runs a custom Make function.

The runtime path depends on the SDK's explicit -ldl configure fact and the host's main-program handle support, so the tap PR remains gated on Automattic/kandelo#874 and #879.

* make: load support from the installed tap
@brandonpayton

Copy link
Copy Markdown
Member Author

Exact-head CI exposed one integration failure after ac_cv_func_group_member=no became authoritative: the legacy registry Wget recipe still injected a file-local always-false group_member stub after Gnulib's generated unistd.h declaration. Wget itself failed with a linkage-declaration conflict, and the same failure propagated into shell, lamp, node-vfs, and wordpress.

Fixed at 4a21076a0: remove the obsolete source injection, let Gnulib provide the configured replacement, and bump Wget's build revision from 2 to 3 because archive bytes change.

Fresh local evidence through scripts/dev-shell.sh:

  • worktree-local musl SDK build completed;
  • Wget 1.25.0 normal registry build completed;
  • generated lib/Makefile records HAVE_GROUP_MEMBER = 0;
  • libgnu_a-group-member.o was built and the final module contains 0026c91e t group_member;
  • full SDK suite: 7 files, 59 tests;
  • build-script syntax and git diff --check passed.

The five failed staging jobs were all manifestations of this one Wget build failure and should rebuild from revision 3 on this head.

Keep parser classifications for SDK mode and thread inference, but forward non-SDK arguments in their original sequence so objects, -l libraries, archives, and linker group controls retain normal static-link semantics. Add unit and real wasm-ld coverage for mixed input ordering and linker controls.
Kandelo permits undefined Wasm imports, so ordinary link probes can report unavailable libc functions as present. Centralize verified musl availability, gnulib string behavior, and the libdl search exception in config.site while preserving caller overrides. Document and test the shared target contract.
Remove Wget's obsolete package-local group_member stub now that the SDK config.site reports the target function truthfully. Bump the package revision because the source and resulting archive change.
@brandonpayton
brandonpayton force-pushed the fix/sdk-config-absent-functions branch from 04a72d4 to 128a3f7 Compare July 14, 2026 02:59
@brandonpayton
brandonpayton changed the base branch from main to fix/sdk-preserve-linker-order July 14, 2026 03:00
Base automatically changed from fix/sdk-preserve-linker-order to main July 14, 2026 18:42
@brandonpayton

Copy link
Copy Markdown
Member Author

Absorbed by rebase-merged #967. Main is now 6efb411 with the sealed tree b133bfa018f7e2bc84c4845edf2f945b2f844954, and ABI 41 activation completed successfully in run 29550069085. The absorption audit confirmed this PR\x27s behavior and tests are present in that merged tree, so this source PR is closed and its branch is deleted.

@brandonpayton
brandonpayton deleted the fix/sdk-config-absent-functions branch July 17, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant