Add wasm32-unknown-wasip1-threads variant to swift.wasm_sdk - #1860
Add wasm32-unknown-wasip1-threads variant to swift.wasm_sdk#1860AttilaTheFun wants to merge 3 commits into
Conversation
End-to-end verification ✅Built and ran a Swift Concurrency program through the # MODULE.bazel
swift.toolchain(name = "swift_toolchain", swift_version = "6.3") # match the SDK's swiftmodule format
swift.wasm_sdk(
toolchain_name = "swift_toolchain",
threads = True,
url = "…/swift-wasm-6.3-RELEASE-…-wasip1-threads.artifactbundle.tar.gz",
sha256 = "…",
)// Hello.swift
@main struct Hello {
static func main() async {
await withTaskGroup(of: Int.self) { group in
for i in 0..<4 { group.addTask { i * i } }
var total = 0
for await v in group { total += v }
print("sum of squares:", total)
}
}
}The compile resolves the Note on the triple fix ( One gotcha worth flagging for users: the host |
|
Pushed 03a6f45, which replaces the hand-mirrored flag lists and the bundle-layout scan with parsing of the bundle's own metadata ( The parsing helpers are deliberately SDK-kind-agnostic (and |
Add a `threads` option to the `swift.wasm_sdk` module extension so a consumer can target `wasm32-unknown-wasip1-threads` (WebAssembly with shared memory, atomics, and wasi-threads) in addition to the single-threaded `wasm32-unknown-wasip1`. swift.org does not publish a threads SDK bundle, so the extension also gains `url` + `sha256` attrs that override the computed swift.org URL; these are required for the threads variant (point it at a swiftwasm release). The swiftwasm bundle nests its target directory under a differently-named inner directory than the swift.org bundle, so the threads path discovers it by scanning for the target triple. When threads is enabled, the generated Swift and cc toolchains gain the atomics/bulk-memory/pthread compile flags and the shared-memory linker flags. The single-threaded path is unchanged (default threads=False renders byte-for-byte identical toolchains).
The threads variant injected the atomics/shared-memory build flags but the
generated cc + Swift toolchains still compiled with
`--target=wasm32-unknown-wasip1` (the single-threaded triple). The swiftwasm
threads SDK ships its swiftmodules under `wasm32-unknown-wasip1-threads`, so
swiftc could not resolve `Swift`/`_Concurrency` for the single-threaded triple
("could not find module Swift for target wasm32-unknown-wasip1; found:
wasm32-unknown-wasip1-threads").
Thread the already-computed `triple` through a `{target_triple}` substitution so
both the clang `--target` and the `CC_TARGET_TRIPLE` make variable (which the
Swift toolchain parses into swiftc `-target`) use the correct triple. Byte-for-
byte identical for the default single-threaded path.
Caught by locally building a threaded wasm binary against a swiftwasm threads
SDK bundle; the existing CI matrix does not exercise the threads path.
Instead of hardcoding the threads variant's compiler/linker flags and scanning the archive for the target-triple directory, parse the metadata that every Swift SDK artifact bundle ships: * `info.json` names the default (non-embedded) artifact's `swift-sdk.json`, which replaces the layout scan. * `swift-sdk.json` provides the target triple (validated against the `threads` attribute), `sdkRootPath`, and `swiftStaticResourcesPath`, which replace the hardcoded `WASI.sdk`/`swift_static` paths. * `toolset.json` provides the per-tool `extraCLIOptions`, which replace the hardcoded threads flag lists. Compiler options are forwarded verbatim; linker options are raw wasm-ld flags (SwiftPM invokes the linker directly), so they are wrapped in `-Wl,` for the clang driver the generated toolchains link through. For the single-threaded swift.org bundle the only behavior change is that the toolset's `-static-stdlib` is now passed to swiftc (inert for compile actions); for the threads bundle the generated toolchains are identical to before, now sourced from the bundle instead of mirrored constants. The parsing helpers (`_relative_metadata_path`, `merged_toolset_options`, `linker_options_to_clang_args`, `_swift_sdk_json_path`, `_swift_sdk_target_settings`) are SDK-kind-agnostic so the Android and Static Linux (bazelbuild#1813) repository rules can share them.
03a6f45 to
308114d
Compare
What
Adds a
threadsoption to theswift.wasm_sdkmodule extension so a consumercan target
wasm32-unknown-wasip1-threads(WebAssembly with shared memory,atomics, and wasi-threads) in addition to the single-threaded
wasm32-unknown-wasip1. This unblocks building threaded Swift-wasm reactorsthat run with real multicore parallelism (e.g. under WAMR with wasi-threads) via
bazel build.To support the second bundle flavor without a second set of hardcoded flags and
paths, the wasm repository rule now derives the target triple, SDK paths, and
compiler/linker flags from the metadata the bundle itself ships
(
info.json/swift-sdk.json/toolset.json) — details below.Why an explicit
url+sha256swift.org does not publish a threads bundle — the computed
https://download.swift.org/.../swift-<v>-RELEASE_wasm.artifactbundle.tar.gzURL 404s for the threads variant. The threads SDK comes from
swiftwasm instead, e.g.
swift-wasm-6.3-RELEASE-wasm32-unknown-wasip1-threads.artifactbundle.zip. So theextension gains
urlandsha256attrs that override the computed swift.org URL(and are required when
threads = True).The
url/sha256override is also usable on the non-threads path if you want topin a mirror.
The repository rule is driven by the bundle's own metadata
Rather than hardcoding per-variant flags and directory layouts, the repository
rule now parses the metadata every Swift SDK artifact bundle ships (both the
swift.org and swiftwasm bundles follow the same scheme):
info.jsonnames the bundle's artifacts; the default (non-embedded)artifact's
swift-sdk.jsonpath replaces the previousbundle-layout computation/scan. (Embedded Swift variants —
embedded-swift-sdk.json— are ignored.)swift-sdk.jsonprovides the target triple (validated against thethreadsattribute, so pointingthreads = Trueat a single-threadedbundle or vice versa fails with a clear message), plus
sdkRootPathandswiftStaticResourcesPath, which replace the previously hardcodedWASI.sdk/swift.xctoolchain/usr/lib/swift_staticpaths.toolset.json(viatoolsetPaths, merged in order) provides theper-tool
extraCLIOptions:cCompiler/cxxCompileroptions → the generatedcc_args(compile);for the threads bundle this is
-matomics -mbulk-memory -mthread-model posix -pthread -ftls-model=local-exec.swiftCompileroptions →swift_toolchaincopts (for the threads bundle:-static-stdlibplus the clang flags via-Xcc; for the swift.orgbundle:
-static-stdlib).linkeroptions are rawwasm-ldflags (SwiftPM invokes the linkerdirectly), so they are wrapped in
-Wl,for the clang driver thegenerated toolchains link through (threads bundle:
--import-memory --export-memory --shared-memory --max-memory=1073741824).rootPathand per-tool executable overrides are intentionally ignored:the generated toolchains always drive the paired standalone toolchain's
own
swiftc/clang.The parsing helpers (
_relative_metadata_path,_swift_sdk_json_path,_swift_sdk_target_settings,merged_toolset_options,linker_options_to_clang_args) are SDK-kind-agnostic, so the Androidrepository rule and the Static Linux SDK proposed in #1813 can share them —
_relative_metadata_pathis taken verbatim from #1813 so whichever landssecond rebases cleanly onto the other's copy.
Backwards compatibility
The single-threaded path now reads the same metadata. For the swift.org
wasm32-unknown-wasip1bundle the resolved paths are identical and the onlyflag delta is that the bundle toolset's
-static-stdlibis now passed toswiftc— inert for compile actions: rebuilding the wasm example producedbit-identical objects (all downstream actions cache-hit) and
//examples/cross_compilation/wasm:reactor_testpasses.Verification
//examples/cross_compilation/wasm:reactor_testpasses; the recompiledobjects are bit-identical to before, so everything downstream cache-hits.
withTaskGroupprogram compilesagainst the
wasm32-unknown-wasip1-threadsswiftmodules, links ashared-memory module, and runs under
wasmtime -W threads=y,shared-memory=y -S threads=y. A peak-concurrencyprobe confirms tasks actually run in parallel (8 workers, peak 4 concurrent
tasks). After the metadata rework the generated toolchains are byte-identical
to the previously hand-mirrored flags. Full transcript in the PR comments.
merged_toolset_options(accumulation across multipletoolsets, both bundle shapes) and
linker_options_to_clang_argsintest/utils_tests.bzl.Files changed
swift/extensions.bzl—threads(bool),url(string) attrs on thewasm_sdktag class;_setup_wasm_sdkvalidation + URL/sha256 selection;threading
threads/urlthrough to the repository rule.swift/internal/extensions/swift_sdks.bzl— metadata parsing(
info.json/swift-sdk.json/toolset.json) shared helpers; triplevalidation against
threads; toolset-driven template substitutions.swift/internal/extensions/wasmsdk.BUILD— resource-dir/sysroot andcompile/link flag placeholders filled from the SDK metadata.
test/utils_tests.bzl— unit tests for the toolset helpers.