Support cross-compiling to WebAssembly and Android with Swift SDKs - #1809
Closed
AttilaTheFun wants to merge 17 commits into
Closed
Support cross-compiling to WebAssembly and Android with Swift SDKs#1809AttilaTheFun wants to merge 17 commits into
AttilaTheFun wants to merge 17 commits into
Conversation
Extends the `swift` module extension with `wasm_sdk` and `android_sdk`
tags that download the official Swift SDK artifact bundles published by
swift.org (the bundles consumed by `swift sdk install`), pinned by
SHA-256, and define toolchains so that plain `swift_library` and
`swift_binary` targets build for `wasm32-unknown-wasip1` and
`{aarch64,x86_64}-unknown-linux-android` under `--platforms`.
Each generated repository pairs one Swift SDK with one standalone host
toolchain of exactly the same Swift release (the Swift module format is
not stable across compiler versions) and defines:
* a `swift_toolchain` that compiles against the SDK's sysroot and Swift
resource directory, and statically links the SDK's Swift runtime via
new generic `linkopts`/`linker_inputs` attributes; and
* a rules_cc `cc_toolchain` driving the matching clang for the target
(the host toolchain's clang for WebAssembly, the hermetically fetched
Android NDK's clang for Android), following the same shape as the
embedded toolchain.
The Android NDK is only fetched when an Android target is actually
built; WebAssembly-only builds do not download it.
Also adds the `swift.no_entry_point_rename` feature: wasm-ld has no
`--defsym`, so the WebAssembly toolchain cannot alias a renamed
`swift_binary` entry point back to the symbol wasi-libc's startup code
expects and instead skips the rename.
The new `examples/cross_compilation` package builds a library and
binary for all three targets through platform transitions; the
resulting WebAssembly binary runs under wasmtime and the Android
binaries are correctly formed PIE executables (16 KiB max page size,
linked against the NDK's `libc++_shared.so`, which the NDK repository
exposes for app packaging).
…ation
Addresses the gaps found while migrating a consumer onto the Swift SDK
toolchains: plain executables are not what WebAssembly and Android actually
load.
* Add a `linkshared` attribute to `swift_binary` (mirroring `cc_binary`):
- Android and other ELF/Mach-O targets get a `lib<name>.so` / `.dylib`
dynamic library, loadable via `System.loadLibrary` / `dlopen` (e.g. a JNI
library; export entry points with `@_cdecl`).
- WebAssembly gets a `<name>.wasm` "reactor" module linked with
`-mexec-model=reactor`: no `main`, initializers run via the exported
`_initialize`, and functions are exposed for a JS host to call. Exports
are retained with `-Xlinker --export=<symbol>` in `linkopts`.
The target is detected with `ctx.target_platform_has_constraint`, and
`linkshared` disables the entry-point rename (no `main`). WebAssembly
outputs also get the conventional `.wasm` extension.
* Enable the rules_cc `shared_flag` feature in the generated Android/wasm C++
toolchains so the dynamic-library link passes `-shared`.
* Expose the NDK's `libc++_shared.so` at a host-independent label
(`@<toolchain>//:libcxx_shared_<arch>`) that selects the NDK for the build
host, so an APK rule can bundle it without naming the host.
* Document a one-line `register_toolchains("@<toolchain>//:all")` for
single-host setups, and the `rules_apple` coexistence story (shared
`compatibility_level = 3`).
The `examples/cross_compilation` example is reworked to build a WebAssembly
reactor and an Android JNI shared library, both from `swift_binary` targets
depending on a shared `Greeter` `swift_library`. `android_app/` adds the
Kotlin app (and a documented `rules_android` packaging recipe) that loads the
JNI library, completing the Kotlin -> Swift (.so) -> Swift library chain. The
JNI entry point is written in Swift using the SDK's `Android` module, so no C
shim is needed.
Verified locally: the reactor runs under wasmtime (exported functions call
into the Swift library), and the Android `.so` is a shared object that exports
the `Java_..._greetingFromSwift` JNI symbol and links `libc++_shared.so`.
Adds a static site that embeds the `swift_binary(linkshared = True)` wasm reactor and drives it from JavaScript end-to-end: it instantiates the module with a minimal WASI shim, runs the reactor's `_initialize`, calls the exported `greeting_length`/`greeting_into`, reads the string Swift wrote into linear memory, and shows it. - `web/index.html` — the page (served via `:web_app`, which assembles index.html + Reactor.wasm into one directory). - `web/verify.mjs` — the same flow under Node for a headless check. - `web/README.md` + README/table entries. Verified in headless Chrome (shows "Hello from Swift, WebAssembly!") and via the Node/wasmtime headless flow.
…rash) The Swift SDK wasm toolchain linked binaries without the `--global-base` / `--table-base` flags that `swiftc` always passes to wasm-ld for its own wasm links. Optimized (`-O`) Swift relies on the indirect function table starting at index 4096 where the runtime/codegen expects it; without `--table-base=4096` generic-metadata instantiation reads out of bounds at runtime (a `memory access out of bounds` fault inside `__swift_instantiateGenericMetadata` the moment a generic type's metadata is instantiated). `-Onone` happens to tolerate the default table base, which masked the bug. Add `-Wl,--global-base=4096 -Wl,--table-base=4096` to the wasm toolchain's linkopts so cc-driven links reproduce swiftc's memory/table layout. Verified an optimized (`-c opt`) SwiftUI app that previously crashed on its first generic metadata access now boots and renders; the reactor example is unaffected (`greeting_length` still returns via wasmtime).
The //examples/cross_compilation targets are tagged `manual` (they download the Swift SDK bundles and the Android NDK), so the `//examples/...` wildcard the other tasks build skips them and the Swift-SDK cross-compilation toolchains were never exercised in CI. Add a dedicated macOS task that builds the wasm reactor, the web app, and the Android JNI shared library explicitly, so a break in the toolchain wiring or link flags is caught in presubmit.
Capture how building Swift on Windows works in rules_swift (the existing host autoconfiguration toolchain, discovered from an installed Swift + Visual Studio, in the same vein as the Xcode/apple_support model), the prerequisites, a verification checklist, and the known gaps (CI Windows task is commented out, so the path needs verifying rather than implementing). Orthogonal to the Swift-SDK cross-compilation in this branch; recorded here to pick the work up on an actual Windows machine.
Work through the WINDOWS.md checklist on a native Windows host (Swift 6.3.2 + MSVC) and fix the bit-rot and missing Windows code paths it surfaced, so that swift_binary, swift_library, swift_binary(linkshared) and swift_test all build and run. Autoconfiguration: - Skip the Microsoft Store python3.exe execution-alias stub and probe for a working interpreter when reading the SDK Info.plist. - Normalize SDKROOT (forward slashes, no trailing separator) so it is valid inside the Python snippet that reads XCTEST_VERSION. - Detect the host CPU instead of hardcoding x86_64. Toolchain: - Don't require a clang CC toolchain on Windows; MSVC (msvc-cl) is expected. - Use MSVC /ALTERNATENAME instead of GNU ld --defsym for the entry point. - Emit the -msvc target-triple environment so swift-symbolgraph-extract can load modules built for *-windows-msvc. - Pass the XCTest include paths to the symbol-graph-extract action. - Suppress LNK4217 (benign for statically linked dllimport symbols). - Understand the aarch64 library / bin64a layout and register an aarch64 Windows toolchain. swift_test / test discovery: - Port tools/test_observer to Windows: SRWLOCK locking, GetProcAddress-based swift-testing entry point lookup, and a swift-corelibs XCTest runner shared with Linux (renamed from LinuxXCTestRunner). - Run the test discovery tool with the Swift runtime on PATH. Worker / general: - Make the persistent worker's filesystem operations long-path (\?\) aware; the _swift_incremental storage area exceeds MAX_PATH. - Sanitize spaces out of derived object paths so the MSVC archiver/linker response files parse (e.g. swift-argument-parser's "Parsable Properties"). - Disable worker sandboxing on Windows (build:windows in .bazelrc). Examples / CI / docs: - Add a shared_library linkshared -> .dll example. - Re-enable a Windows CI task that builds the examples and runs the xctest. - Update WINDOWS.md with the verified status.
…rified on this branch)
…ols)
The Android Swift-SDK toolchain mirrored swiftc's static-stdlib-args.lnk, which
passes -Wl,--exclude-libs,ALL to hide the static Swift runtime's symbols. That
works for swiftc because the user's code is compiled into the main object files
and only the runtime arrives via static archives. In the Bazel model a
swift_binary's deps (swift_library) are themselves static archives, so
--exclude-libs,ALL also demotes the user's own exported symbols to local --
including @_cdecl("Java_...") JNI entry points defined in a library. They drop
out of .dynsym and System.loadLibrary can't bind them (UnsatisfiedLinkError on
the first native call).
Omit --exclude-libs,ALL so a linkshared library exports its symbols. A consumer
that wants to hide the runtime can pass a linker version script listing the
symbols to export, which is the standard way to control a JNI .so's exports.
Verified: a swift_binary(linkshared) Android JNI library with @_cdecl JNI
functions in a swift_library dep now exports all 15 Java_ symbols in .dynsym and
the app binds them and launches on an emulator; a consumer version script
(global: Java_*; local: *) cleanly exports only the JNI symbols with no runtime
leakage.
AttilaTheFun
requested review from
aaronsky,
adincebic,
brentleyjones,
keith,
luispadron and
mattrobmattrob
as code owners
June 15, 2026 05:49
buildifier 8.5.1's `external-path` lint flags the literal "/external/" in `_execroot_relative_path`, but that helper exists precisely to turn an absolute output-base path into an execroot-relative one, so the substring is intentional. Annotate the two lines with `# buildifier: disable=external-path` so the buildifier CI check passes.
…nds) The Windows task installed Swift under a `batch_commands:` key, which BazelCI doesn't recognize, so the "Setup (Batch Commands)" step ran empty — Swift was never installed, the autoconfiguration found no `swiftc.exe`, declared no `windows-toolchain`, and every `swift_*` target failed to resolve a toolchain. BazelCI runs a task's `shell_commands` as a batch script on Windows (that is the "Setup (Batch Commands)" step), so move the install there.
batch_commands is the correct Windows key (bazelci.py runs it on Windows; shell_commands is ignored there). The installer runs but the machine-wide Path/SDKROOT it sets don't reach the already-running CI process, so the Swift autoconfiguration finds no swiftc.exe. Add temporary DIAG lines to print the install location and the Path/SDKROOT the installer set, so they can be exposed to the build via the task's `environment:` block.
The swift.org Windows installer puts swiftc.exe under %LOCALAPPDATA%\Programs\Swift\Toolchains\<ver>+Asserts\usr\bin and sets SDKROOT to the bundled Windows.sdk, but it records these on the user/machine env, which the already-running CI process never picks up — so the Swift autoconfiguration found no swiftc.exe and declared no windows-toolchain. Set Path (Toolchains + Runtimes + Tools bins) and SDKROOT in the task's environment block, which BazelCI applies (with %VAR% expansion) to the build. Drop the diagnostics now that the layout is known.
The Windows autoconfiguration read repository_ctx.os.environ["ProgramData"] unguarded, but that variable isn't always present in the build's environment (e.g. the service-account Buildkite CI agent doesn't set it), which failed toolchain configuration with `key "ProgramData" not found in dictionary`. Fall back to its conventional value `C:\ProgramData`, mirroring the existing defensive handling of `Path`/`PATH`.
swiftc's clang can't find the UCRT/MSVC C headers (errno.h) because the build doesn't run inside a Visual Studio developer environment, so INCLUDE/LIB aren't set. Print the vcvars-provided INCLUDE/LIB/LIBPATH (and the SDK/toolset versions) so they can be set in the task's environment block (the swift compile inherits os.environ). Temporary; removed once the values are wired in.
swiftc's clang could not find the C headers (errno.h) because the build does not run inside a Visual Studio developer environment, so INCLUDE/LIB were unset. Set them (and add the MSVC/SDK tool bins to Path for link.exe) from the image's VS 2022 BuildTools + Windows SDK, mirroring what vcvars64.bat exports; the swift compile inherits these via os.environ. Versions are pinned to the Bazel CI image. Drop the diagnostic now that the values are known.
…ied host) Swift 6.0.3's clang module setup hit a cyclic dependency (ucrt -> _Builtin_intrinsics -> ucrt) against the CI image's recent Windows SDK (10.0.26100). 6.3.2 -- the version the Windows host support was verified on, and the one this PR's Swift SDK cross-compilation already uses -- carries the clang fixes for those Windows module cycles. Bump only the Windows task; Linux stays on its pinned version.
This was referenced Jun 16, 2026
keith
requested changes
Jun 23, 2026
keith
left a comment
Member
There was a problem hiding this comment.
This is great, some high level comments:
- can you split this up into multiple PRs? as is there are a lot of separate things in here, linkshared support, windows support, android support, and wasm support
- can we use @androidndk instead of downloading it ourselves? i imagine for the most likely users of this having to maintain the version bumps and stuff between those 2 versions would be a bit annoying
- can you edit down the LLM comments / docs to what you think is useful long term
- would it be possible to avoid some of the wasm specific stuff in swift_binary? i guess ideally that stuff would live in the toolchain somehow?
| # The file prefix map would make the worker resolve the Xcode | ||
| # developer directory on macOS hosts, which this toolchain | ||
| # does not depend on. | ||
| "-swift.file_prefix_map", |
Member
There was a problem hiding this comment.
I think disabling this is probably the wrong one, we might want to disable the xcode specific one? or i believe there was one added for embedded stuff specifically to say it doesn't support developer dir that maybe we should use instead?
This was referenced Jun 23, 2026
This was referenced Jun 23, 2026
Contributor
Author
|
thanks @keith ! yes I can try to split this up, though Github doesn't handle cross-repo stacked PRs well. we might have to land them one at a time and I can rebase. I have broken them up like this: main ─┬─ linkshared |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds first-class support for cross-compiling
swift_library/swift_binaryto WebAssembly (wasm32-unknown-wasip1) and Android (aarch64/x86_64) using the official Swift SDK artifact bundles published by swift.org -- the same bundlesswift sdk installconsumes.A target is selected purely by
--platformsand produces the artifact each ecosystem actually consumes: a WASI executable or reactor module, or an Android executable or shared library. This builds directly on the existing hermetic standalone-toolchain infrastructure (theswiftmodule extension and the embedded toolchain shape) rather than introducing a parallel mechanism -- no custom rules, no manualswiftcplumbing.Usage
Then build any
swift_library/swift_binarywith a platform carrying@platforms//os:wasi + cpu:wasm32or@platforms//os:android + cpu:{aarch64,x86_64}:See the new section in
doc/standalone_toolchain.mdandexamples/cross_compilation/.Output kinds
swift_binaryproduces the conventional artifact for each platform:linksharedFalse(default).wasm, has_start/main)True.wasm, nomain,-mexec-model=reactor)lib<name>.so).so/.dylib)A wasm reactor / shared library has no
main; it exports functions for a host to call (linkopts = ["-Xlinker", "--export=<sym>"]). On wasm,linksharedmaps to the reactor execution model, not a-shareddynamic library.Design
swift.toolchaintag by name and derive the version, URLs, and bundled checksums (swift_sdk_releases.bzl) from it.swift_toolchain(compile:-sdk,-resource-dir, clang-importer builtin headers) plus a rules_cccc_toolchainfor linking -- the host toolchain's clang for wasm, the NDK's clang for Android. Everything is fetched lazily: host-only builds fetch nothing new, and wasm-only builds do not download the NDK.swiftcdoes. The toolchains link the SDK's static stdlib, mirroring the flagsswiftcuses with these SDKs (static-executable-args.lnk/static-stdlib-args.lnk). For wasm this includes placing linear-memory data and the indirect function table at the same basesswiftcpasses to wasm-ld (--global-base=4096,--table-base=4096);--table-basein particular is required for correctness -- optimized (-O) Swift relies on the indirect function table starting where the runtime/codegen expects it, and without it generic-metadata instantiation faults at runtime (see the dedicated commit). Android binaries use the NDK'slibc++_shared.so(exposed by the NDK repo as:libcxx_shared_{arch}for app packaging) and a 16 KiB max page size as required by Android 15+.swift_toolchainattributes.linkopts/linker_inputslet a toolchain supply the runtime link flags and inputs explicitly, replacing the hardcoded per-OS defaults that don't apply to SDK-provided runtimes. The runtime start object (swiftrt.o) is modeled as an executable-only input so reactor / shared-library links omit it automatically.swift.no_entry_point_renamefeature. wasm-ld has no--defsym, so the entry-point rename thatswift_binaryperforms (to allow linking binaries into tests) cannot be aliased back to the symbol wasi-libc's startup expects; the wasm toolchain opts out of the rename instead.swiftc -emit-executable, the Swift driver's requirement thatswiftrt.olive inside the-sdksysroot never applies; the start object is passed explicitly, and the plain NDK sysroot is used for compilation.Testing
examples/cross_compilation/builds a sharedswift_library(Greeter) reused by every entry point, plus a wasm reactor (Reactor.wasm), a browser web app embedding it (web_app--index.html+ a Nodeverify.mjsheadless check, seeweb/README.md), and an Android JNI shared library (libSwiftJNI.so) with a documented Kotlin packaging recipe (android_app/). A new dedicated CI task builds all of these on macOS (they're taggedmanualbecause they fetch the SDK bundles / NDK, so they were previously skipped by the//examples/...wildcard).ET_DYNAArch64 object (verified JNI symbols,liblog/libc++_sharedNEEDED entries, and 16 KiB LOAD alignment withllvm-readelf); it loads on-device with noUnsatisfiedLinkError.rules_apple, which coexists with thisrules_swift). All targets build and the app boots and renders in the browser in bothfastbuildand-c opt. The-Ometadata crash fixed by the--table-basecommit was found and reproduced through this consumer.bazel test //test/... //examples/... //doc/...: identical results to upstreammainon the same machine (the only failures are pre-existing local XCTest/Xcode-version environment issues, byte-identical failure sets verified against a pristinemainworktree).bazel run //doc:gazelleis a no-op; buildifier clean.MODULE.bazeltoolchain was bumped 6.3 → 6.3.2 (the release the SDK checksums are bundled for); all examples includingexamples/embeddedbuild with it.Windows host support
This branch also takes building Swift natively on Windows (host toolchain + MSVC, orthogonal to the Swift-SDK cross-compilation above) from "scaffolding present but never exercised in CI" to verified end-to-end on a real Windows 11 host (Swift 6.3.2 for Windows + Visual Studio 2022 / MSVC 14.44, Bazel 9.1.1).
swift_binary,swift_library,swift_binary(linkshared = True)→.dll, andswift_testall build and run. Verifying theWINDOWS.mdchecklist surfaced a mix of bit-rot and genuinely missing Windows code paths; the fixes, by area:swift_autoconfiguration.bzl): skip the Microsoft Storepython3.exeexecution-alias stub and probe for a working interpreter; normalizeSDKROOT(forward slashes, no trailing separator) so it is valid inside the Python snippet that readsXCTEST_VERSION— both previously leftxctest_versionempty; detect the host CPU instead of hardcodingx86_64.swift_toolchain.bzl): don't require a clang CC toolchain on Windows (MSVCmsvc-clis expected); use MSVC/ALTERNATENAMEinstead of GNUld's--defsymfor the entry-point alias; emit the-msvctarget-triple environment soswift-symbolgraph-extractcan load*-windows-msvcmodules; pass the XCTest include paths to the symbol-graph-extract action; suppress the benignLNK4217that statically linkingdllimportsymbols produces; understand theaarch64library /bin64alayout.swift_test/ test discovery: porttools/test_observerto Windows —SRWLOCKlocking,GetProcAddress-based swift-testing entry-point lookup, and a swift-corelibs XCTest runner shared with Linux (LinuxXCTestRunner→SwiftCorelibsXCTestRunner); run the discovery tool with the Swift runtime onPATH.\\?\) aware (the_swift_incrementalstorage area exceedsMAX_PATH); sanitize spaces out of derived object paths so the MSVC archiver/linker response files parse (e.g. swift-argument-parser'sParsable Properties/); disable worker sandboxing on Windows (build:windowsin.bazelrc).A new
//examples/xplatform/shared_library(linkshared→.dll) example is added, and a Windows CI task that builds the examples and runs thexctesttest is re-enabled — the Swift-install prologue / BazelCI Windows image provisioning is the one piece not validated on a local host.aarch64Windows is implemented (host-CPU detection + a registered toolchain) but unverified for lack of arm64 hardware. SeeWINDOWS.mdfor the full status and prerequisites.Notes for review
armv7resources are present but not wired up;ANDROID_ARCHScan be extended on demand.register_toolchainslist is currently explicit per (host, target); auto-registration / an:allconvenience could be a follow-up (the devMODULE.bazelnotes why:allis ambiguous across Linux distros today).