Skip to content

Support cross-compiling to WebAssembly and Android with Swift SDKs - #1809

Closed
AttilaTheFun wants to merge 17 commits into
bazelbuild:mainfrom
AttilaTheFun:lshire-android-wasm-support
Closed

Support cross-compiling to WebAssembly and Android with Swift SDKs#1809
AttilaTheFun wants to merge 17 commits into
bazelbuild:mainfrom
AttilaTheFun:lshire-android-wasm-support

Conversation

@AttilaTheFun

Copy link
Copy Markdown
Contributor

Summary

Adds first-class support for cross-compiling swift_library / swift_binary to WebAssembly (wasm32-unknown-wasip1) and Android (aarch64 / x86_64) using the official Swift SDK artifact bundles published by swift.org -- the same bundles swift sdk install consumes.

A target is selected purely by --platforms and produces the artifact each ecosystem actually consumes: a WASI executable or reactor module, or an Android executable or shared library. This builds directly on the existing hermetic standalone-toolchain infrastructure (the swift module extension and the embedded toolchain shape) rather than introducing a parallel mechanism -- no custom rules, no manual swiftc plumbing.

Usage

swift = use_extension("@rules_swift//swift:extensions.bzl", "swift")
swift.toolchain(name = "swift_toolchain", swift_version = "6.3.2")
swift.wasm_sdk(toolchain_name = "swift_toolchain")
swift.android_sdk(toolchain_name = "swift_toolchain")  # api_level = 28 by default

register_toolchains(
    "@swift_toolchain//:swift_toolchain_wasm32_xcode",
    "@swift_toolchain//:cc_toolchain_wasm32_xcode",
    "@swift_toolchain//:swift_toolchain_android_aarch64_xcode",
    "@swift_toolchain//:cc_toolchain_android_aarch64_xcode",
    # ... per host platform / architecture
)

Then build any swift_library / swift_binary with a platform carrying @platforms//os:wasi + cpu:wasm32 or @platforms//os:android + cpu:{aarch64,x86_64}:

# A WASI reactor (no `main`; exports functions for a JS host) or an Android
# JNI shared library -- same attribute as cc_binary's `linkshared`:
swift_binary(name = "App", srcs = [...], linkshared = True)

See the new section in doc/standalone_toolchain.md and examples/cross_compilation/.

Output kinds

swift_binary produces the conventional artifact for each platform:

linkshared wasm Android host (unchanged)
False (default) WASI command module (.wasm, has _start/main) executable executable
True WASI reactor (.wasm, no main, -mexec-model=reactor) shared library (lib<name>.so) shared library (.so/.dylib)

A wasm reactor / shared library has no main; it exports functions for a host to call (linkopts = ["-Xlinker", "--export=<sym>"]). On wasm, linkshared maps to the reactor execution model, not a -shared dynamic library.

Design

  • One Swift release, everywhere. A Swift SDK only works with the host compiler from the same release tag (the module format is unstable across versions), so the SDK tags reference a swift.toolchain tag by name and derive the version, URLs, and bundled checksums (swift_sdk_releases.bzl) from it.
  • Per-(host, target) repositories. Each generated repo pairs one SDK with one standalone host toolchain and defines a swift_toolchain (compile: -sdk, -resource-dir, clang-importer builtin headers) plus a rules_cc cc_toolchain for linking -- the host toolchain's clang for wasm, the NDK's clang for Android. Everything is fetched lazily: host-only builds fetch nothing new, and wasm-only builds do not download the NDK.
  • Static Swift runtime, linked the way swiftc does. The toolchains link the SDK's static stdlib, mirroring the flags swiftc uses with these SDKs (static-executable-args.lnk / static-stdlib-args.lnk). For wasm this includes placing linear-memory data and the indirect function table at the same bases swiftc passes to wasm-ld (--global-base=4096, --table-base=4096); --table-base in particular is required for correctness -- optimized (-O) Swift relies on the indirect function table starting where the runtime/codegen expects it, and without it generic-metadata instantiation faults at runtime (see the dedicated commit). Android binaries use the NDK's libc++_shared.so (exposed by the NDK repo as :libcxx_shared_{arch} for app packaging) and a 16 KiB max page size as required by Android 15+.
  • New generic swift_toolchain attributes. linkopts / linker_inputs let a toolchain supply the runtime link flags and inputs explicitly, replacing the hardcoded per-OS defaults that don't apply to SDK-provided runtimes. The runtime start object (swiftrt.o) is modeled as an executable-only input so reactor / shared-library links omit it automatically.
  • swift.no_entry_point_rename feature. wasm-ld has no --defsym, so the entry-point rename that swift_binary performs (to allow linking binaries into tests) cannot be aliased back to the symbol wasi-libc's startup expects; the wasm toolchain opts out of the rename instead.
  • No merged NDK sysroot. Because linking goes through the CC toolchain rather than swiftc -emit-executable, the Swift driver's requirement that swiftrt.o live inside the -sdk sysroot never applies; the start object is passed explicitly, and the plain NDK sysroot is used for compilation.

Testing

  • examples/cross_compilation/ builds a shared swift_library (Greeter) reused by every entry point, plus a wasm reactor (Reactor.wasm), a browser web app embedding it (web_app -- index.html + a Node verify.mjs headless check, see web/README.md), and an Android JNI shared library (libSwiftJNI.so) with a documented Kotlin packaging recipe (android_app/). A new dedicated CI task builds all of these on macOS (they're tagged manual because they fetch the SDK bundles / NDK, so they were previously skipped by the //examples/... wildcard).
  • The wasm reactor runs under wasmtime and in the browser, returning the greeting Swift wrote into linear memory ("Hello from Swift, WebAssembly!").
  • The Android shared library is a well-formed ET_DYN AArch64 object (verified JNI symbols, liblog/libc++_shared NEEDED entries, and 16 KiB LOAD alignment with llvm-readelf); it loads on-device with no UnsatisfiedLinkError.
  • Validated end-to-end against a real consumer: a full SwiftUI-subset application cross-compiled -- unchanged source -- to wasm and Android (alongside the real-SwiftUI Apple build via a recent rules_apple, which coexists with this rules_swift). All targets build and the app boots and renders in the browser in both fastbuild and -c opt. The -O metadata crash fixed by the --table-base commit was found and reproduced through this consumer.
  • bazel test //test/... //examples/... //doc/...: identical results to upstream main on the same machine (the only failures are pre-existing local XCTest/Xcode-version environment issues, byte-identical failure sets verified against a pristine main worktree).
  • bazel run //doc:gazelle is a no-op; buildifier clean.
  • The dev MODULE.bazel toolchain was bumped 6.3 → 6.3.2 (the release the SDK checksums are bundled for); all examples including examples/embedded build with it.

Windows host support

This branch also takes building Swift natively on Windows (host toolchain + MSVC, orthogonal to the Swift-SDK cross-compilation above) from "scaffolding present but never exercised in CI" to verified end-to-end on a real Windows 11 host (Swift 6.3.2 for Windows + Visual Studio 2022 / MSVC 14.44, Bazel 9.1.1). swift_binary, swift_library, swift_binary(linkshared = True).dll, and swift_test all build and run. Verifying the WINDOWS.md checklist surfaced a mix of bit-rot and genuinely missing Windows code paths; the fixes, by area:

  • Autoconfiguration (swift_autoconfiguration.bzl): skip the Microsoft Store python3.exe execution-alias stub and probe for a working interpreter; normalize SDKROOT (forward slashes, no trailing separator) so it is valid inside the Python snippet that reads XCTEST_VERSION — both previously left xctest_version empty; detect the host CPU instead of hardcoding x86_64.
  • Toolchain (swift_toolchain.bzl): don't require a clang CC toolchain on Windows (MSVC msvc-cl is expected); use MSVC /ALTERNATENAME instead of GNU ld's --defsym for the entry-point alias; emit the -msvc target-triple environment so swift-symbolgraph-extract can load *-windows-msvc modules; pass the XCTest include paths to the symbol-graph-extract action; suppress the benign LNK4217 that statically linking dllimport symbols produces; understand the aarch64 library / bin64a layout.
  • swift_test / test discovery: port tools/test_observer to Windows — SRWLOCK locking, GetProcAddress-based swift-testing entry-point lookup, and a swift-corelibs XCTest runner shared with Linux (LinuxXCTestRunnerSwiftCorelibsXCTestRunner); run the discovery tool with the Swift runtime on PATH.
  • Worker / general: make the persistent worker's filesystem operations long-path (\\?\) aware (the _swift_incremental storage area exceeds MAX_PATH); sanitize spaces out of derived object paths so the MSVC archiver/linker response files parse (e.g. swift-argument-parser's Parsable Properties/); disable worker sandboxing on Windows (build:windows in .bazelrc).

A new //examples/xplatform/shared_library (linkshared.dll) example is added, and a Windows CI task that builds the examples and runs the xctest test is re-enabled — the Swift-install prologue / BazelCI Windows image provisioning is the one piece not validated on a local host. aarch64 Windows is implemented (host-CPU detection + a registered toolchain) but unverified for lack of arm64 hardware. See WINDOWS.md for the full status and prerequisites.

Notes for review

  • CI cost. The new cross-compilation CI task downloads the Swift SDK bundles (~400 MB) and the Android NDK (~600 MB), on top of the standalone toolchain the embedded example already fetches. It is a single dedicated macOS task (build-only) so it can be tuned or split independently; Linux coverage can be added the same way if desired.
  • The SDK's armv7 resources are present but not wired up; ANDROID_ARCHS can be extended on demand.
  • The register_toolchains list is currently explicit per (host, target); auto-registration / an :all convenience could be a follow-up (the dev MODULE.bazel notes why :all is ambiguous across Linux distros today).

Extends the `swift` module extension with `wasm_sdk` and `android_sdk`
tags that download the official Swift SDK artifact bundles published by
swift.org (the bundles consumed by `swift sdk install`), pinned by
SHA-256, and define toolchains so that plain `swift_library` and
`swift_binary` targets build for `wasm32-unknown-wasip1` and
`{aarch64,x86_64}-unknown-linux-android` under `--platforms`.

Each generated repository pairs one Swift SDK with one standalone host
toolchain of exactly the same Swift release (the Swift module format is
not stable across compiler versions) and defines:

* a `swift_toolchain` that compiles against the SDK's sysroot and Swift
  resource directory, and statically links the SDK's Swift runtime via
  new generic `linkopts`/`linker_inputs` attributes; and
* a rules_cc `cc_toolchain` driving the matching clang for the target
  (the host toolchain's clang for WebAssembly, the hermetically fetched
  Android NDK's clang for Android), following the same shape as the
  embedded toolchain.

The Android NDK is only fetched when an Android target is actually
built; WebAssembly-only builds do not download it.

Also adds the `swift.no_entry_point_rename` feature: wasm-ld has no
`--defsym`, so the WebAssembly toolchain cannot alias a renamed
`swift_binary` entry point back to the symbol wasi-libc's startup code
expects and instead skips the rename.

The new `examples/cross_compilation` package builds a library and
binary for all three targets through platform transitions; the
resulting WebAssembly binary runs under wasmtime and the Android
binaries are correctly formed PIE executables (16 KiB max page size,
linked against the NDK's `libc++_shared.so`, which the NDK repository
exposes for app packaging).
…ation

Addresses the gaps found while migrating a consumer onto the Swift SDK
toolchains: plain executables are not what WebAssembly and Android actually
load.

* Add a `linkshared` attribute to `swift_binary` (mirroring `cc_binary`):
  - Android and other ELF/Mach-O targets get a `lib<name>.so` / `.dylib`
    dynamic library, loadable via `System.loadLibrary` / `dlopen` (e.g. a JNI
    library; export entry points with `@_cdecl`).
  - WebAssembly gets a `<name>.wasm` "reactor" module linked with
    `-mexec-model=reactor`: no `main`, initializers run via the exported
    `_initialize`, and functions are exposed for a JS host to call. Exports
    are retained with `-Xlinker --export=<symbol>` in `linkopts`.
  The target is detected with `ctx.target_platform_has_constraint`, and
  `linkshared` disables the entry-point rename (no `main`). WebAssembly
  outputs also get the conventional `.wasm` extension.

* Enable the rules_cc `shared_flag` feature in the generated Android/wasm C++
  toolchains so the dynamic-library link passes `-shared`.

* Expose the NDK's `libc++_shared.so` at a host-independent label
  (`@<toolchain>//:libcxx_shared_<arch>`) that selects the NDK for the build
  host, so an APK rule can bundle it without naming the host.

* Document a one-line `register_toolchains("@<toolchain>//:all")` for
  single-host setups, and the `rules_apple` coexistence story (shared
  `compatibility_level = 3`).

The `examples/cross_compilation` example is reworked to build a WebAssembly
reactor and an Android JNI shared library, both from `swift_binary` targets
depending on a shared `Greeter` `swift_library`. `android_app/` adds the
Kotlin app (and a documented `rules_android` packaging recipe) that loads the
JNI library, completing the Kotlin -> Swift (.so) -> Swift library chain. The
JNI entry point is written in Swift using the SDK's `Android` module, so no C
shim is needed.

Verified locally: the reactor runs under wasmtime (exported functions call
into the Swift library), and the Android `.so` is a shared object that exports
the `Java_..._greetingFromSwift` JNI symbol and links `libc++_shared.so`.
Adds a static site that embeds the `swift_binary(linkshared = True)` wasm
reactor and drives it from JavaScript end-to-end: it instantiates the module
with a minimal WASI shim, runs the reactor's `_initialize`, calls the exported
`greeting_length`/`greeting_into`, reads the string Swift wrote into linear
memory, and shows it.

- `web/index.html` — the page (served via `:web_app`, which assembles
  index.html + Reactor.wasm into one directory).
- `web/verify.mjs` — the same flow under Node for a headless check.
- `web/README.md` + README/table entries.

Verified in headless Chrome (shows "Hello from Swift, WebAssembly!") and via
the Node/wasmtime headless flow.
…rash)

The Swift SDK wasm toolchain linked binaries without the `--global-base` /
`--table-base` flags that `swiftc` always passes to wasm-ld for its own wasm
links. Optimized (`-O`) Swift relies on the indirect function table starting
at index 4096 where the runtime/codegen expects it; without `--table-base=4096`
generic-metadata instantiation reads out of bounds at runtime (a
`memory access out of bounds` fault inside `__swift_instantiateGenericMetadata`
the moment a generic type's metadata is instantiated). `-Onone` happens to
tolerate the default table base, which masked the bug.

Add `-Wl,--global-base=4096 -Wl,--table-base=4096` to the wasm toolchain's
linkopts so cc-driven links reproduce swiftc's memory/table layout. Verified an
optimized (`-c opt`) SwiftUI app that previously crashed on its first generic
metadata access now boots and renders; the reactor example is unaffected
(`greeting_length` still returns via wasmtime).
The //examples/cross_compilation targets are tagged `manual` (they download
the Swift SDK bundles and the Android NDK), so the `//examples/...` wildcard
the other tasks build skips them and the Swift-SDK cross-compilation
toolchains were never exercised in CI. Add a dedicated macOS task that builds
the wasm reactor, the web app, and the Android JNI shared library explicitly,
so a break in the toolchain wiring or link flags is caught in presubmit.
Capture how building Swift on Windows works in rules_swift (the existing host
autoconfiguration toolchain, discovered from an installed Swift + Visual Studio,
in the same vein as the Xcode/apple_support model), the prerequisites, a
verification checklist, and the known gaps (CI Windows task is commented out, so
the path needs verifying rather than implementing). Orthogonal to the Swift-SDK
cross-compilation in this branch; recorded here to pick the work up on an actual
Windows machine.
Work through the WINDOWS.md checklist on a native Windows host (Swift 6.3.2
+ MSVC) and fix the bit-rot and missing Windows code paths it surfaced, so
that swift_binary, swift_library, swift_binary(linkshared) and swift_test all
build and run.

Autoconfiguration:
- Skip the Microsoft Store python3.exe execution-alias stub and probe for a
  working interpreter when reading the SDK Info.plist.
- Normalize SDKROOT (forward slashes, no trailing separator) so it is valid
  inside the Python snippet that reads XCTEST_VERSION.
- Detect the host CPU instead of hardcoding x86_64.

Toolchain:
- Don't require a clang CC toolchain on Windows; MSVC (msvc-cl) is expected.
- Use MSVC /ALTERNATENAME instead of GNU ld --defsym for the entry point.
- Emit the -msvc target-triple environment so swift-symbolgraph-extract can
  load modules built for *-windows-msvc.
- Pass the XCTest include paths to the symbol-graph-extract action.
- Suppress LNK4217 (benign for statically linked dllimport symbols).
- Understand the aarch64 library / bin64a layout and register an aarch64
  Windows toolchain.

swift_test / test discovery:
- Port tools/test_observer to Windows: SRWLOCK locking, GetProcAddress-based
  swift-testing entry point lookup, and a swift-corelibs XCTest runner shared
  with Linux (renamed from LinuxXCTestRunner).
- Run the test discovery tool with the Swift runtime on PATH.

Worker / general:
- Make the persistent worker's filesystem operations long-path (\?\) aware;
  the _swift_incremental storage area exceeds MAX_PATH.
- Sanitize spaces out of derived object paths so the MSVC archiver/linker
  response files parse (e.g. swift-argument-parser's "Parsable Properties").
- Disable worker sandboxing on Windows (build:windows in .bazelrc).

Examples / CI / docs:
- Add a shared_library linkshared -> .dll example.
- Re-enable a Windows CI task that builds the examples and runs the xctest.
- Update WINDOWS.md with the verified status.
…ols)

The Android Swift-SDK toolchain mirrored swiftc's static-stdlib-args.lnk, which
passes -Wl,--exclude-libs,ALL to hide the static Swift runtime's symbols. That
works for swiftc because the user's code is compiled into the main object files
and only the runtime arrives via static archives. In the Bazel model a
swift_binary's deps (swift_library) are themselves static archives, so
--exclude-libs,ALL also demotes the user's own exported symbols to local --
including @_cdecl("Java_...") JNI entry points defined in a library. They drop
out of .dynsym and System.loadLibrary can't bind them (UnsatisfiedLinkError on
the first native call).

Omit --exclude-libs,ALL so a linkshared library exports its symbols. A consumer
that wants to hide the runtime can pass a linker version script listing the
symbols to export, which is the standard way to control a JNI .so's exports.

Verified: a swift_binary(linkshared) Android JNI library with @_cdecl JNI
functions in a swift_library dep now exports all 15 Java_ symbols in .dynsym and
the app binds them and launches on an emulator; a consumer version script
(global: Java_*; local: *) cleanly exports only the JNI symbols with no runtime
leakage.
buildifier 8.5.1's `external-path` lint flags the literal "/external/" in
`_execroot_relative_path`, but that helper exists precisely to turn an absolute
output-base path into an execroot-relative one, so the substring is intentional.
Annotate the two lines with `# buildifier: disable=external-path` so the
buildifier CI check passes.
…nds)

The Windows task installed Swift under a `batch_commands:` key, which BazelCI
doesn't recognize, so the "Setup (Batch Commands)" step ran empty — Swift was
never installed, the autoconfiguration found no `swiftc.exe`, declared no
`windows-toolchain`, and every `swift_*` target failed to resolve a toolchain.
BazelCI runs a task's `shell_commands` as a batch script on Windows (that is the
"Setup (Batch Commands)" step), so move the install there.
batch_commands is the correct Windows key (bazelci.py runs it on Windows;
shell_commands is ignored there). The installer runs but the machine-wide
Path/SDKROOT it sets don't reach the already-running CI process, so the Swift
autoconfiguration finds no swiftc.exe. Add temporary DIAG lines to print the
install location and the Path/SDKROOT the installer set, so they can be exposed
to the build via the task's `environment:` block.
The swift.org Windows installer puts swiftc.exe under
%LOCALAPPDATA%\Programs\Swift\Toolchains\<ver>+Asserts\usr\bin and sets SDKROOT
to the bundled Windows.sdk, but it records these on the user/machine env, which
the already-running CI process never picks up — so the Swift autoconfiguration
found no swiftc.exe and declared no windows-toolchain. Set Path (Toolchains +
Runtimes + Tools bins) and SDKROOT in the task's environment block, which
BazelCI applies (with %VAR% expansion) to the build. Drop the diagnostics now
that the layout is known.
The Windows autoconfiguration read repository_ctx.os.environ["ProgramData"]
unguarded, but that variable isn't always present in the build's environment
(e.g. the service-account Buildkite CI agent doesn't set it), which failed
toolchain configuration with `key "ProgramData" not found in dictionary`. Fall
back to its conventional value `C:\ProgramData`, mirroring the existing
defensive handling of `Path`/`PATH`.
swiftc's clang can't find the UCRT/MSVC C headers (errno.h) because the build
doesn't run inside a Visual Studio developer environment, so INCLUDE/LIB aren't
set. Print the vcvars-provided INCLUDE/LIB/LIBPATH (and the SDK/toolset
versions) so they can be set in the task's environment block (the swift compile
inherits os.environ). Temporary; removed once the values are wired in.
swiftc's clang could not find the C headers (errno.h) because the build does
not run inside a Visual Studio developer environment, so INCLUDE/LIB were
unset. Set them (and add the MSVC/SDK tool bins to Path for link.exe) from the
image's VS 2022 BuildTools + Windows SDK, mirroring what vcvars64.bat exports;
the swift compile inherits these via os.environ. Versions are pinned to the
Bazel CI image. Drop the diagnostic now that the values are known.
…ied host)

Swift 6.0.3's clang module setup hit a cyclic dependency
(ucrt -> _Builtin_intrinsics -> ucrt) against the CI image's recent Windows SDK
(10.0.26100). 6.3.2 -- the version the Windows host support was verified on, and
the one this PR's Swift SDK cross-compilation already uses -- carries the clang
fixes for those Windows module cycles. Bump only the Windows task; Linux stays
on its pinned version.

@keith keith left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, some high level comments:

  • can you split this up into multiple PRs? as is there are a lot of separate things in here, linkshared support, windows support, android support, and wasm support
  • can we use @androidndk instead of downloading it ourselves? i imagine for the most likely users of this having to maintain the version bumps and stuff between those 2 versions would be a bit annoying
  • can you edit down the LLM comments / docs to what you think is useful long term
  • would it be possible to avoid some of the wasm specific stuff in swift_binary? i guess ideally that stuff would live in the toolchain somehow?

# The file prefix map would make the worker resolve the Xcode
# developer directory on macOS hosts, which this toolchain
# does not depend on.
"-swift.file_prefix_map",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think disabling this is probably the wrong one, we might want to disable the xcode specific one? or i believe there was one added for embedded stuff specifically to say it doesn't support developer dir that maybe we should use instead?

@AttilaTheFun

AttilaTheFun commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

thanks @keith ! yes I can try to split this up, though Github doesn't handle cross-repo stacked PRs well. we might have to land them one at a time and I can rebase.

I have broken them up like this:

main ─┬─ linkshared
└─ Swift SDK framework
├─ WebAssembly
├─ Android
└─ Windows host

#1820
#1821
#1817
#1818
#1819

@AttilaTheFun
AttilaTheFun deleted the lshire-android-wasm-support branch July 6, 2026 19:59
@AttilaTheFun

Copy link
Copy Markdown
Contributor Author

Superseded — this was split into #1817 (wasm), #1818 (Android), #1819 (Windows), #1820 (linkshared), and #1821 (Swift SDK framework), all now merged. Thanks for the reviews!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants