From 3139db71f165f06bf66fb44636063e7a54c540cc Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 00:49:42 +0300 Subject: [PATCH 1/7] =?UTF-8?q?Phase=202.6:=20Part=202=20closeout=20?= =?UTF-8?q?=E2=80=94=20consolidation=20decision=20memo,=20modules=20docs,?= =?UTF-8?q?=20final=20trace?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - MODERNIZATION.md: Phase 2.6 rewritten as a DECISION MEMO. The planned consolidation (code into purview, internal headers deleted) is BLOCKED by the 2.5 Android finding: the NDK's clang cannot build the modules and Android compiles the libraries textually from exactly the headers consolidation would delete. Preconditions documented (NDK clang >= 22 or Android parked; clangd purview-lint maturity; dht partition DAG), along with what consolidation buys (the unmet incremental targets) and the recommended leaf-first measured path when unblocked. The facade stage is a stable resting point. - Final ClangBuildAnalyzer trace recorded: DhtRpc.pb.h fell from #1 (96s, 33x) to #9 (15.7s, 6x); SLogger/Logger left the expensive list entirely; new #1 is gtest.h (stays #include by design); total parse CPU 859s -> 723s. The third success metric is met. - README: C++ named modules developer documentation (import rules, sibling-import rule, direct-link rule, Android exception). - Stale 'No C++ modules in the codebase yet' comments updated in all 9 CMakeLists. - 2.5 section extended with the Android saga (folly supports overlay; STREAMR_MODULES_SUPPORTED fallback; Android workflow lint removal). Co-Authored-By: Claude Fable 5 --- CMakeLists.txt | 7 +- MODERNIZATION.md | 67 ++++++++++++++++++- README.md | 34 ++++++++++ packages/streamr-dht/CMakeLists.txt | 7 +- packages/streamr-eventemitter/CMakeLists.txt | 7 +- packages/streamr-json/CMakeLists.txt | 7 +- .../CMakeLists.txt | 7 +- packages/streamr-logger/CMakeLists.txt | 7 +- packages/streamr-proto-rpc/CMakeLists.txt | 7 +- .../CMakeLists.txt | 7 +- packages/streamr-utils/CMakeLists.txt | 7 +- 11 files changed, 127 insertions(+), 37 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index de953210..a3eeac7c 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -7,10 +7,9 @@ set(VCPKG_INSTALLED_DIR ${CMAKE_CURRENT_SOURCE_DIR}/build/vcpkg_installed) set (VCPKG_OVERLAY_TRIPLETS ENV{VCPKG_OVERLAY_TRIPLETS}) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) include(cmake/homebrewClang.cmake) diff --git a/MODERNIZATION.md b/MODERNIZATION.md index ef96998d..8976660b 100644 --- a/MODERNIZATION.md +++ b/MODERNIZATION.md @@ -632,7 +632,7 @@ before any consolidation gains. trackerless-network + proxyclient downstream builds unchanged; root tree 307/307; full lint green. -## Phase 2.5 — trackerless-network + proxyclient, FINAL METRICS (PR pending) +## Phase 2.5 — trackerless-network + proxyclient, FINAL METRICS ✅ (PR #33, merged) The migration surface is complete: every C++ package that exports an API now ships a module. - **streamr-trackerless-network**: `:protos` over the NetworkRpc trio @@ -651,6 +651,26 @@ now ships a module. - New standing gate applied (2.4 lesson): a LOCAL RELEASE build+test of the migrated package (caught nothing this time — the folly/protobuf overlay patches hold). +- **The Android smoke gate (first Android build since the 1.3 dependency + wave) earned its keep twice**: + 1. Upstream's 2026 folly port silently dropped Android from its + `supports` expression → `./install.sh --android` failed outright at + dependency install. The folly overlay re-allows Android + (JUSTIFICATION.md updated); **folly 2026 builds fine on the NDK**. + 2. **The NDK's clang (18/19) cannot build our modules** — observed: + `import streamr.json` fails to provide the `toJson` overload set + (known pre-clang-19/20 `export using` overload bugs). Resolution: + `STREAMR_MODULES_SUPPORTED=OFF` on Android in StreamrModules.cmake — + Android builds libraries textually from headers (the façade keeps + them as source of truth), module units are skipped + (`streamr_add_module_library` falls back to STATIC + stub source so + PUBLIC usage requirements stay identical), and test/example targets + (which import, and are never executed on Android) are not built. + The Android workflow also stops running lint (host-independent; + covered by validate.yml; import-using test files have no compile + commands on an Android tree). + **⇒ Android now has a hard dependency on headers — see the Phase 2.6 + decision memo below.** ### FINAL FAÇADE-STAGE METRICS (vs Phase 2.0 baselines, macOS, idle) | Metric | Baseline | Final (7 packages migrated) | Target | Verdict | @@ -658,6 +678,8 @@ now ships a module. | Clean root build | 102 s | **78 s (−24%)** | −25% | ✅ effectively met (78–88 s across repeats) | | `SLogger.hpp` touch | 62–70 s | 58 s (−6…−12%) | −40% | ❌ not yet — consolidation work | | `StreamID.hpp` touch | 19 s | 32 s (+68%) | −40% | ❌ regression — BMI-chain latency | +| Expensive headers (ClangBuildAnalyzer) | `DhtRpc.pb.h` #1 (96 s, 33×); SLogger/Logger top-4 | `DhtRpc.pb.h` #9 (15.7 s, 6×); SLogger/Logger GONE from list; #1 = `gtest.h` (stays `#include` by design) | pb.h/folly leave the top | ✅ met | +| Total frontend parse CPU | 859 s | 723 s (−16%, incl. the module units) | — | ✅ | **Honest reading**: the clean-build target is effectively met already at the façade stage (test TUs load BMIs instead of re-parsing header @@ -677,6 +699,49 @@ sequencing; the numbers now quantify exactly why consolidation matters. exclude import-using files from clangd-tidy until clangd modules support stabilizes. +## Phase 2.6 — Consolidation: DECISION MEMO (owner decision required) +The planned end state (user-confirmed 2026-07): NO internal headers — +all code in `.cppm` module units, `#include` only for third-party, +generated proto, and the C API header. Consolidation is what delivers the +so-far-unmet incremental-rebuild targets (a partition edit stops +invalidating whole-package BMIs). + +**Consolidation is currently BLOCKED by a platform constraint discovered +at the 2.5 Android gate**: the NDK's clang cannot build the modules, so +Android compiles the libraries textually FROM THE HEADERS that +consolidation would delete. Deleting internal headers breaks the Android +build outright — including the shipping artifact (proxyclient `.so` / +Kotlin wrapper). + +### Preconditions for starting consolidation +1. **NDK ships a modules-capable clang** (>= 22, matching the rest of the + toolchain), or Android support is explicitly parked/dropped. +2. clangd modules support matures enough to lint purview code (today it + mis-unifies preamble/BMI types; with code in partitions there is no + header fallback for the linter — coverage would regress from "full" + to "none" for consolidated code). +3. The dht intra-package include cycles (connection/endpoint cluster) + must become a partition DAG — untangle or use coarse per-cluster + partitions (deferred from 2.4 by design). + +### What consolidation buys (quantified at the 2.4/2.5 checkpoints) +- The −40% incremental-rebuild targets (currently: SLogger touch −6…−12%, + StreamID touch REGRESSED +68% from BMI-chain invalidation). +- The final IWYU/ODR hygiene of the no-headers end state. + +### Recommended path (when unblocked) +Leaf-first and interleaved as originally planned — eventemitter as the +consolidation canary, one package per PR, `bench.sh` measured at every +step (the 2.4 lesson: measure before generalizing), headers deleted +per-package with a grep-enforced "nothing includes them" gate. + +### Interim posture (adopted now) +The façade stage is COMPLETE and delivers: uniform `import streamr.` +consumption, −24% clean builds, 250+ tests through import, and module +infrastructure exercised on macOS/Linux/iOS. Headers remain the linted +source of truth; Android remains textual. This is a stable resting point +— nothing rots by waiting for the NDK. + ## Success metrics (measured macOS + Linux, end of Phase 2.5) - ≥25% clean-build wall-clock reduction (dev build with tests). - ≥40% reduction on incremental rebuild after touching a mid-stack header diff --git a/README.md b/README.md index dde30b41..a7071fa9 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,40 @@ The codebase is built as C++26 (`CMAKE_CXX_STANDARD 26`, required) everywhere, b Practical rule: header-only features (ranges, `contains`, concepts, formatting of standard types, etc.) work everywhere; features needing runtime-library symbols (e.g. new locale facets, some `std::print`/filesystem edges) are gated by the iOS deployment target — the iOS build will tell you. +#### C++ named modules + +Every API-exporting package ships a C++ named module (`streamr.json`, +`streamr.logger`, `streamr.eventemitter`, `streamr.utils`, +`streamr.protorpc`, `streamr.dht`, `streamr.trackerlessnetwork`). The +package test suites consume them with a single `import streamr.;`. + +The modules are currently *façades*: each package's `modules/` directory +holds a coarse `:all` interface unit that includes the public headers in +its global module fragment and re-exports the public names (plus a +`:protos` unit over generated protobuf code where applicable). The +headers remain the source of truth — `#include` and `import` consumers +can be mixed freely. + +Rules of thumb when writing import-using code: + +* Include what you use: `import` does not leak transitive std/third-party + headers the way textual inclusion did. +* Third-party libraries (folly, boost, gtest, protobuf) stay `#include`. +* Do **not** also textually include monorepo headers that reach the same + third-party stacks in an import-using file — import the sibling module + instead (clangd's experimental modules support mis-diagnoses the mix). +* A target whose sources `import` a module must link that module's target + directly (`streamr::streamr-`) and call + `streamr_enable_imports()` — see `cmake/StreamrModules.cmake`. +* **Android builds do not use modules** (the NDK's clang is too old): + libraries build textually from the headers and test targets are + skipped there. Handled automatically by `STREAMR_MODULES_SUPPORTED` + in `StreamrModules.cmake`. + +The plan for consolidating code out of headers into the module units +(and the platform preconditions for it) is in +[MODERNIZATION.md](MODERNIZATION.md). + ### Install all the dependencies and build the SDK for MacOS and Linux ```bash diff --git a/packages/streamr-dht/CMakeLists.txt b/packages/streamr-dht/CMakeLists.txt index 29e16f31..b6f5aca0 100644 --- a/packages/streamr-dht/CMakeLists.txt +++ b/packages/streamr-dht/CMakeLists.txt @@ -5,10 +5,9 @@ include(homebrewClang.cmake) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) set(CMAKE_EXPORT_COMPILE_COMMANDS ON) diff --git a/packages/streamr-eventemitter/CMakeLists.txt b/packages/streamr-eventemitter/CMakeLists.txt index 0611fd8f..dd1242d5 100644 --- a/packages/streamr-eventemitter/CMakeLists.txt +++ b/packages/streamr-eventemitter/CMakeLists.txt @@ -3,10 +3,9 @@ set(CMAKE_POLICY_DEFAULT_CMP0077 NEW) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) include(homebrewClang.cmake) diff --git a/packages/streamr-json/CMakeLists.txt b/packages/streamr-json/CMakeLists.txt index 6479b45c..0a5f220f 100644 --- a/packages/streamr-json/CMakeLists.txt +++ b/packages/streamr-json/CMakeLists.txt @@ -3,10 +3,9 @@ set(CMAKE_POLICY_DEFAULT_CMP0077 NEW) include(homebrewClang.cmake) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) set(CMAKE_CXX_USE_RESPONSE_FILE_FOR_INCLUDES Off) diff --git a/packages/streamr-libstreamrproxyclient/CMakeLists.txt b/packages/streamr-libstreamrproxyclient/CMakeLists.txt index 9e539935..7fa0c87c 100644 --- a/packages/streamr-libstreamrproxyclient/CMakeLists.txt +++ b/packages/streamr-libstreamrproxyclient/CMakeLists.txt @@ -4,10 +4,9 @@ set(CMAKE_POLICY_DEFAULT_CMP0077 NEW) include(homebrewClang.cmake) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) set(CMAKE_CXX_USE_RESPONSE_FILE_FOR_INCLUDES Off) diff --git a/packages/streamr-logger/CMakeLists.txt b/packages/streamr-logger/CMakeLists.txt index 9b318fbf..c5f267f1 100644 --- a/packages/streamr-logger/CMakeLists.txt +++ b/packages/streamr-logger/CMakeLists.txt @@ -3,10 +3,9 @@ include(homebrewClang.cmake) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) set(CMAKE_CXX_USE_RESPONSE_FILE_FOR_INCLUDES Off) diff --git a/packages/streamr-proto-rpc/CMakeLists.txt b/packages/streamr-proto-rpc/CMakeLists.txt index 69311528..a28942ce 100644 --- a/packages/streamr-proto-rpc/CMakeLists.txt +++ b/packages/streamr-proto-rpc/CMakeLists.txt @@ -5,10 +5,9 @@ include(homebrewClang.cmake) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) set(CMAKE_EXPORT_COMPILE_COMMANDS ON) diff --git a/packages/streamr-trackerless-network/CMakeLists.txt b/packages/streamr-trackerless-network/CMakeLists.txt index e88f6818..d2df5851 100644 --- a/packages/streamr-trackerless-network/CMakeLists.txt +++ b/packages/streamr-trackerless-network/CMakeLists.txt @@ -5,10 +5,9 @@ include(${CMAKE_CURRENT_LIST_DIR}/homebrewClang.cmake) set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) set(CMAKE_EXPORT_COMPILE_COMMANDS ON) diff --git a/packages/streamr-utils/CMakeLists.txt b/packages/streamr-utils/CMakeLists.txt index 3f5a20bb..6e2499a8 100644 --- a/packages/streamr-utils/CMakeLists.txt +++ b/packages/streamr-utils/CMakeLists.txt @@ -10,10 +10,9 @@ set(CMAKE_TOOLCHAIN_FILE "$ENV{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake") set(CMAKE_CXX_STANDARD 26) set(CMAKE_CXX_STANDARD_REQUIRED ON) -# No C++ modules in the codebase yet: CMake >= 3.28 (CMP0155) would otherwise -# scan all C++20+ sources for module imports, injecting GCC scanning flags -# (-fmodules-ts, -fmodule-mapper, -fdeps-format) into compile_commands.json, -# which clangd rejects. The modules migration will turn scanning back on. +# Module scanning stays OFF globally so that non-module TUs keep clean +# compile commands for clangd; targets with module units or import-using +# sources opt back in via the StreamrModules.cmake helpers. set(CMAKE_CXX_SCAN_FOR_MODULES OFF) project(streamr-utils CXX) From 76af151a7db96b07b50baddc39846925349004d2 Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 00:58:48 +0300 Subject: [PATCH 2/7] Android: use NDK r29 (clang 21) and gate modules on NDK clang version (androidbuild) Owner review catch: the 2.5 Android modules failure was on the CI runner's DEFAULT NDK r27.3 (clang 18.0.2), not the latest stable. NDK r29 (Oct 2025) ships clang 21, whose modules support should handle the facades. The Android workflow now selects ANDROID_NDK_LATEST_HOME (r29, preinstalled on macos-26 runners), and STREAMR_MODULES_SUPPORTED on Android is version-gated (NDK clang >= 21 -> modules ON, incl. building the import-using test targets as a compile smoke; older NDKs fall back to the textual build). This PR's androidbuild leg is the validation run; if green, the consolidation memo's Android precondition is already satisfiable today. Co-Authored-By: Claude Fable 5 --- .github/workflows/validateandroid.yml | 11 ++++++++ MODERNIZATION.md | 13 +++++++--- README.md | 6 ++--- cmake/StreamrModules.cmake | 26 ++++++++++++------- packages/streamr-dht/StreamrModules.cmake | 26 ++++++++++++------- .../streamr-eventemitter/StreamrModules.cmake | 26 ++++++++++++------- packages/streamr-json/StreamrModules.cmake | 26 ++++++++++++------- .../StreamrModules.cmake | 26 ++++++++++++------- packages/streamr-logger/StreamrModules.cmake | 26 ++++++++++++------- .../streamr-proto-rpc/StreamrModules.cmake | 26 ++++++++++++------- .../StreamrModules.cmake | 26 ++++++++++++------- packages/streamr-utils/StreamrModules.cmake | 26 ++++++++++++------- 12 files changed, 176 insertions(+), 88 deletions(-) diff --git a/.github/workflows/validateandroid.yml b/.github/workflows/validateandroid.yml index 09dafb3d..13433047 100644 --- a/.github/workflows/validateandroid.yml +++ b/.github/workflows/validateandroid.yml @@ -24,6 +24,17 @@ jobs: fetch-depth: 0 submodules: true token: ${{ secrets.GITHUB_TOKEN }} + # The runner's default ANDROID_NDK is an older LTS (r27 = clang 18, + # whose C++ modules support is too immature). Use the latest stable + # NDK preinstalled on the image (r29 = clang 21) — StreamrModules.cmake + # enables the modules on Android when the NDK clang is >= 21. + - name: use latest stable NDK + run: | + echo "ANDROID_NDK=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" + echo "ANDROID_NDK_HOME=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" + echo "ANDROID_NDK_ROOT=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" + "$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/darwin-arm64/bin/clang" --version | head -1 + shell: bash - name: install uses: ./.github/workflows/reusable/cached-install # No lint step: linting is host-independent and runs on the diff --git a/MODERNIZATION.md b/MODERNIZATION.md index 8976660b..52ca012d 100644 --- a/MODERNIZATION.md +++ b/MODERNIZATION.md @@ -669,8 +669,9 @@ now ships a module. The Android workflow also stops running lint (host-independent; covered by validate.yml; import-using test files have no compile commands on an Android tree). - **⇒ Android now has a hard dependency on headers — see the Phase 2.6 - decision memo below.** + UPDATE (2.6): the failing NDK was the CI runner's default r27 + (clang 18.0.2); the gate is now version-based and Android CI uses + NDK r29 (clang 21) — see the 2.6 memo. ### FINAL FAÇADE-STAGE METRICS (vs Phase 2.0 baselines, macOS, idle) | Metric | Baseline | Final (7 packages migrated) | Target | Verdict | @@ -714,8 +715,12 @@ build outright — including the shipping artifact (proxyclient `.so` / Kotlin wrapper). ### Preconditions for starting consolidation -1. **NDK ships a modules-capable clang** (>= 22, matching the rest of the - toolchain), or Android support is explicitly parked/dropped. +1. **A modules-capable NDK clang on Android.** The 2.5 failure was on the + CI runner's default NDK r27 (clang 18.0.2). NDK r29 (latest stable, + Oct 2025) ships clang 21 — the Android CI now uses r29 and + StreamrModules.cmake gates Android modules on NDK clang >= 21 + (validated on the Phase 2.6 PR's androidbuild leg). Older NDKs fall + back to the textual build automatically. 2. clangd modules support matures enough to lint purview code (today it mis-unifies preamble/BMI types; with code in partitions there is no header fallback for the linter — coverage would regress from "full" diff --git a/README.md b/README.md index a7071fa9..6f843325 100644 --- a/README.md +++ b/README.md @@ -88,9 +88,9 @@ Rules of thumb when writing import-using code: * A target whose sources `import` a module must link that module's target directly (`streamr::streamr-`) and call `streamr_enable_imports()` — see `cmake/StreamrModules.cmake`. -* **Android builds do not use modules** (the NDK's clang is too old): - libraries build textually from the headers and test targets are - skipped there. Handled automatically by `STREAMR_MODULES_SUPPORTED` +* **Android needs NDK r29+** (clang 21) for the modules. With an older + NDK the libraries build textually from the headers and test targets + are skipped — handled automatically by `STREAMR_MODULES_SUPPORTED` in `StreamrModules.cmake`. The plan for consolidating code out of headers into the module units diff --git a/cmake/StreamrModules.cmake b/cmake/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/cmake/StreamrModules.cmake +++ b/cmake/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-dht/StreamrModules.cmake b/packages/streamr-dht/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-dht/StreamrModules.cmake +++ b/packages/streamr-dht/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-eventemitter/StreamrModules.cmake b/packages/streamr-eventemitter/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-eventemitter/StreamrModules.cmake +++ b/packages/streamr-eventemitter/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-json/StreamrModules.cmake b/packages/streamr-json/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-json/StreamrModules.cmake +++ b/packages/streamr-json/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-libstreamrproxyclient/StreamrModules.cmake b/packages/streamr-libstreamrproxyclient/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-libstreamrproxyclient/StreamrModules.cmake +++ b/packages/streamr-libstreamrproxyclient/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-logger/StreamrModules.cmake b/packages/streamr-logger/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-logger/StreamrModules.cmake +++ b/packages/streamr-logger/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-proto-rpc/StreamrModules.cmake b/packages/streamr-proto-rpc/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-proto-rpc/StreamrModules.cmake +++ b/packages/streamr-proto-rpc/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-trackerless-network/StreamrModules.cmake b/packages/streamr-trackerless-network/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-trackerless-network/StreamrModules.cmake +++ b/packages/streamr-trackerless-network/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() diff --git a/packages/streamr-utils/StreamrModules.cmake b/packages/streamr-utils/StreamrModules.cmake index 906e96fe..c312fc98 100644 --- a/packages/streamr-utils/StreamrModules.cmake +++ b/packages/streamr-utils/StreamrModules.cmake @@ -10,16 +10,24 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang (18/19), whose C++ modules support is -# too immature for the façade modules (known `export using` overload-set -# bugs — observed: `import streamr.json;` fails to provide the toJson -# overloads). Android consumes the ordinary headers instead (they remain the -# source of truth during the façade stage), the module units are skipped, -# and packages must not build their import-using test targets -# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships -# clang >= 22 — consolidation (2.6) requires it, as headers disappear then. +# Android builds use the NDK's clang, and modules support depends on its +# version: NDK r27 ships clang 18, whose C++ modules support is too +# immature for the façade modules (known `export using` overload-set bugs — +# observed: `import streamr.json;` failed to provide the toJson overloads). +# NDK r29 ships clang 21, which is expected to handle them. Gate on the +# compiler version: with an older NDK, Android consumes the ordinary +# headers instead (they remain the source of truth during the façade +# stage), the module units are skipped, and import-using test/example +# targets are not built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - set(STREAMR_MODULES_SUPPORTED OFF) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + set(STREAMR_MODULES_SUPPORTED ON) + else() + set(STREAMR_MODULES_SUPPORTED OFF) + message(STATUS + "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " + "< 21 (use NDK r29+ for modules on Android)") + endif() else() set(STREAMR_MODULES_SUPPORTED ON) endif() From 92d4fa8b109c2f303e8a9782378cf28a1f7a5625 Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 00:59:49 +0300 Subject: [PATCH 3/7] Retrigger CI with androidbuild title in effect (androidbuild) Co-Authored-By: Claude Fable 5 From f9a76f1698bf136b84debe2563759e83b11b983e Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 01:05:04 +0300 Subject: [PATCH 4/7] Fix NDK version-print path: prebuilt dir is darwin-x86_64 on all Macs (androidbuild) Co-Authored-By: Claude Fable 5 --- .github/workflows/validateandroid.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/validateandroid.yml b/.github/workflows/validateandroid.yml index 13433047..374a6058 100644 --- a/.github/workflows/validateandroid.yml +++ b/.github/workflows/validateandroid.yml @@ -33,7 +33,9 @@ jobs: echo "ANDROID_NDK=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" echo "ANDROID_NDK_HOME=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" echo "ANDROID_NDK_ROOT=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" - "$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/darwin-arm64/bin/clang" --version | head -1 + # NB: the prebuilt dir is named darwin-x86_64 even on ARM Macs + # (universal binaries under the legacy name). + "$ANDROID_NDK_LATEST_HOME"/toolchains/llvm/prebuilt/darwin-*/bin/clang --version | head -1 shell: bash - name: install uses: ./.github/workflows/reusable/cached-install From 4501815a26d8d86a540c1fd2ee947af5c01268d3 Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 02:04:37 +0300 Subject: [PATCH 5/7] Fix Android modules: -pthread BMI config mismatch, not compiler maturity The Phase 2.5 Android failure (import streamr.json -> every imported name "undeclared") was root-caused by reproducing the CI build locally against NDK r27/r28/r29: the import-using test TUs compile with -pthread (inherited from GTest's Threads::Threads) while the module libraries' BMIs were built without it. On Android -pthread toggles a language option clang validates when loading a module file (-Wmodule-file-config-mismatch), so the BMI silently fails to load and every imported name cascades to "use of undeclared identifier". The raw NDK compilers (clang 18/19/21) all build and import the facade modules correctly. - StreamrModules.cmake helpers: make -pthread a PUBLIC compile option on Android (genexpr, no-op elsewhere) so BMIs and importers agree. - Lower the Android STREAMR_MODULES_SUPPORTED floor from clang 21 to clang 18 (= NDK r27, oldest verified); textual fallback below that. - Verified locally: streamr-json + streamr-eventemitter full package builds (tests + examples) green for arm64-android on NDK r27 and r29. - validateandroid.yml: keep the latest-stable-NDK step (hygiene, no longer a modules requirement); comment updated. - MODERNIZATION.md: 2.5 record corrected, 2.6 memo precondition 1 marked RESOLVED; README Android note updated. Co-Authored-By: Claude Fable 5 --- .github/workflows/validateandroid.yml | 9 +-- MODERNIZATION.md | 70 +++++++++++-------- README.md | 12 ++-- cmake/StreamrModules.cmake | 32 ++++++--- packages/streamr-dht/StreamrModules.cmake | 32 ++++++--- .../streamr-eventemitter/StreamrModules.cmake | 32 ++++++--- packages/streamr-json/StreamrModules.cmake | 32 ++++++--- .../StreamrModules.cmake | 32 ++++++--- packages/streamr-logger/StreamrModules.cmake | 32 ++++++--- .../streamr-proto-rpc/StreamrModules.cmake | 32 ++++++--- .../StreamrModules.cmake | 32 ++++++--- packages/streamr-utils/StreamrModules.cmake | 32 ++++++--- 12 files changed, 242 insertions(+), 137 deletions(-) diff --git a/.github/workflows/validateandroid.yml b/.github/workflows/validateandroid.yml index 374a6058..bb48999b 100644 --- a/.github/workflows/validateandroid.yml +++ b/.github/workflows/validateandroid.yml @@ -24,10 +24,11 @@ jobs: fetch-depth: 0 submodules: true token: ${{ secrets.GITHUB_TOKEN }} - # The runner's default ANDROID_NDK is an older LTS (r27 = clang 18, - # whose C++ modules support is too immature). Use the latest stable - # NDK preinstalled on the image (r29 = clang 21) — StreamrModules.cmake - # enables the modules on Android when the NDK clang is >= 21. + # The runner's default ANDROID_NDK is an older LTS (r27). Use the + # latest stable NDK preinstalled on the image (r29 = clang 21). + # Modules build on r27+ too (StreamrModules.cmake floors at NDK + # clang 18) — building with the latest stable is hygiene, not a + # modules requirement. - name: use latest stable NDK run: | echo "ANDROID_NDK=$ANDROID_NDK_LATEST_HOME" >> "$GITHUB_ENV" diff --git a/MODERNIZATION.md b/MODERNIZATION.md index 52ca012d..2d2711b3 100644 --- a/MODERNIZATION.md +++ b/MODERNIZATION.md @@ -657,21 +657,20 @@ now ships a module. `supports` expression → `./install.sh --android` failed outright at dependency install. The folly overlay re-allows Android (JUSTIFICATION.md updated); **folly 2026 builds fine on the NDK**. - 2. **The NDK's clang (18/19) cannot build our modules** — observed: - `import streamr.json` fails to provide the `toJson` overload set - (known pre-clang-19/20 `export using` overload bugs). Resolution: - `STREAMR_MODULES_SUPPORTED=OFF` on Android in StreamrModules.cmake — - Android builds libraries textually from headers (the façade keeps - them as source of truth), module units are skipped - (`streamr_add_module_library` falls back to STATIC + stub source so - PUBLIC usage requirements stay identical), and test/example targets - (which import, and are never executed on Android) are not built. - The Android workflow also stops running lint (host-independent; - covered by validate.yml; import-using test files have no compile - commands on an Android tree). - UPDATE (2.6): the failing NDK was the CI runner's default r27 - (clang 18.0.2); the gate is now version-based and Android CI uses - NDK r29 (clang 21) — see the 2.6 memo. + 2. **`import streamr.json` failed on Android** — every imported name + reported as undeclared (the `toJson` overload set etc.). Diagnosed + AT THE TIME as NDK-clang `export using` overload bugs; resolved by + forcing `STREAMR_MODULES_SUPPORTED=OFF` on Android (libraries built + textually from headers, module units skipped via the STATIC + + stub-source fallback so PUBLIC usage requirements stay identical, + import-using test/example targets not built). The Android workflow + also stopped running lint (host-independent; covered by + validate.yml; import-using test files have no compile commands on + an Android tree). + CORRECTED in 2.6: the compiler was never the problem — the real + cause was a `-pthread` BMI configuration mismatch (see the 2.6 + memo). Android now builds the modules on NDK r27+ (clang >= 18); + the textual fallback remains only for older NDKs. ### FINAL FAÇADE-STAGE METRICS (vs Phase 2.0 baselines, macOS, idle) | Metric | Baseline | Final (7 packages migrated) | Target | Verdict | @@ -707,20 +706,30 @@ generated proto, and the C API header. Consolidation is what delivers the so-far-unmet incremental-rebuild targets (a partition edit stops invalidating whole-package BMIs). -**Consolidation is currently BLOCKED by a platform constraint discovered -at the 2.5 Android gate**: the NDK's clang cannot build the modules, so -Android compiles the libraries textually FROM THE HEADERS that -consolidation would delete. Deleting internal headers breaks the Android -build outright — including the shipping artifact (proxyclient `.so` / -Kotlin wrapper). +**The 2.5 Android blocker is RESOLVED (2.6).** The Phase 2.5 failure +(`import streamr.json` → every imported name "undeclared") was NOT an +NDK-clang capability gap. Root cause, found by reproducing the CI +failure locally and bisecting against NDK r27/r28/r29: the import-using +test TUs compile with `-pthread` (inherited from GTest's +`Threads::Threads`) while the module libraries' BMIs were compiled +without it; on Android `-pthread` toggles a language option that clang +validates when loading a module file +(`-Wmodule-file-config-mismatch` — the FIRST error in the log; the +"undeclared identifier" flood is its cascade), so the BMI silently +fails to load. The raw NDK compilers (clang 18, 19 and 21) all compile +and import the façade modules correctly when invoked by hand. Fix: the +module-library helpers in StreamrModules.cmake make `-pthread` a PUBLIC +compile option on Android, so the BMI and every importer always agree. +Android CI keeps using the runner's latest stable NDK (r29, clang 21); +`STREAMR_MODULES_SUPPORTED` keeps a version floor at the oldest +verified NDK clang (18 = r27), with the textual fallback below it. ### Preconditions for starting consolidation -1. **A modules-capable NDK clang on Android.** The 2.5 failure was on the - CI runner's default NDK r27 (clang 18.0.2). NDK r29 (latest stable, - Oct 2025) ships clang 21 — the Android CI now uses r29 and - StreamrModules.cmake gates Android modules on NDK clang >= 21 - (validated on the Phase 2.6 PR's androidbuild leg). Older NDKs fall - back to the textual build automatically. +1. ~~A modules-capable NDK on Android~~ **RESOLVED (2.6)** — the 2.5 + failure was the `-pthread` BMI mismatch above, not the compiler. + Android builds the façade modules on NDK r27+ (verified locally on + r27 and r29 for streamr-json + streamr-eventemitter, and on the + Phase 2.6 PR's androidbuild leg for the full monorepo). 2. clangd modules support matures enough to lint purview code (today it mis-unifies preamble/BMI types; with code in partitions there is no header fallback for the linter — coverage would regress from "full" @@ -743,9 +752,10 @@ per-package with a grep-enforced "nothing includes them" gate. ### Interim posture (adopted now) The façade stage is COMPLETE and delivers: uniform `import streamr.` consumption, −24% clean builds, 250+ tests through import, and module -infrastructure exercised on macOS/Linux/iOS. Headers remain the linted -source of truth; Android remains textual. This is a stable resting point -— nothing rots by waiting for the NDK. +infrastructure exercised on macOS/Linux/iOS/Android. Headers remain the +linted source of truth. With the Android blocker resolved, consolidation +is gated only by preconditions 2 (clangd lint coverage of purview code) +and 3 (dht partition DAG). This is a stable resting point. ## Success metrics (measured macOS + Linux, end of Phase 2.5) - ≥25% clean-build wall-clock reduction (dev build with tests). diff --git a/README.md b/README.md index 6f843325..de333496 100644 --- a/README.md +++ b/README.md @@ -88,10 +88,14 @@ Rules of thumb when writing import-using code: * A target whose sources `import` a module must link that module's target directly (`streamr::streamr-`) and call `streamr_enable_imports()` — see `cmake/StreamrModules.cmake`. -* **Android needs NDK r29+** (clang 21) for the modules. With an older - NDK the libraries build textually from the headers and test targets - are skipped — handled automatically by `STREAMR_MODULES_SUPPORTED` - in `StreamrModules.cmake`. +* **Android builds the modules with NDK r27+** (clang 18; the CI uses + the latest stable NDK, currently r29). With an older NDK the + libraries build textually from the headers and test targets are + skipped — handled automatically by `STREAMR_MODULES_SUPPORTED` in + `StreamrModules.cmake`. Note for module authors: the module-library + helpers make `-pthread` part of the PUBLIC interface on Android — + clang refuses to load a BMI whose POSIX-thread setting differs from + the importing TU's. The plan for consolidating code out of headers into the module units (and the platform preconditions for it) is in diff --git a/cmake/StreamrModules.cmake b/cmake/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/cmake/StreamrModules.cmake +++ b/cmake/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-dht/StreamrModules.cmake b/packages/streamr-dht/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-dht/StreamrModules.cmake +++ b/packages/streamr-dht/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-eventemitter/StreamrModules.cmake b/packages/streamr-eventemitter/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-eventemitter/StreamrModules.cmake +++ b/packages/streamr-eventemitter/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-json/StreamrModules.cmake b/packages/streamr-json/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-json/StreamrModules.cmake +++ b/packages/streamr-json/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-libstreamrproxyclient/StreamrModules.cmake b/packages/streamr-libstreamrproxyclient/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-libstreamrproxyclient/StreamrModules.cmake +++ b/packages/streamr-libstreamrproxyclient/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-logger/StreamrModules.cmake b/packages/streamr-logger/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-logger/StreamrModules.cmake +++ b/packages/streamr-logger/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-proto-rpc/StreamrModules.cmake b/packages/streamr-proto-rpc/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-proto-rpc/StreamrModules.cmake +++ b/packages/streamr-proto-rpc/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-trackerless-network/StreamrModules.cmake b/packages/streamr-trackerless-network/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-trackerless-network/StreamrModules.cmake +++ b/packages/streamr-trackerless-network/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() diff --git a/packages/streamr-utils/StreamrModules.cmake b/packages/streamr-utils/StreamrModules.cmake index c312fc98..3c6eb29e 100644 --- a/packages/streamr-utils/StreamrModules.cmake +++ b/packages/streamr-utils/StreamrModules.cmake @@ -10,23 +10,22 @@ # OFF globally (clean compile commands for clangd), and the helpers below # re-enable scanning per target. -# Android builds use the NDK's clang, and modules support depends on its -# version: NDK r27 ships clang 18, whose C++ modules support is too -# immature for the façade modules (known `export using` overload-set bugs — -# observed: `import streamr.json;` failed to provide the toJson overloads). -# NDK r29 ships clang 21, which is expected to handle them. Gate on the -# compiler version: with an older NDK, Android consumes the ordinary -# headers instead (they remain the source of truth during the façade -# stage), the module units are skipped, and import-using test/example -# targets are not built (STREAMR_MODULES_SUPPORTED guards them). +# Android builds use the NDK's clang. The façade modules build correctly +# with NDK r27+ (clang 18) — the failure previously attributed to compiler +# immaturity was a -pthread BMI configuration mismatch, fixed in the +# helpers below. Keep a version floor at the oldest verified NDK clang +# (18 = r27): with an older NDK, Android consumes the ordinary headers +# instead (they remain the source of truth during the façade stage), the +# module units are skipped, and import-using test/example targets are not +# built (STREAMR_MODULES_SUPPORTED guards them). if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android") - if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 21) + if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 18) set(STREAMR_MODULES_SUPPORTED ON) else() set(STREAMR_MODULES_SUPPORTED OFF) message(STATUS "C++ modules disabled: NDK clang ${CMAKE_CXX_COMPILER_VERSION} " - "< 21 (use NDK r29+ for modules on Android)") + "< 18 (use NDK r27+ for modules on Android)") endif() else() set(STREAMR_MODULES_SUPPORTED ON) @@ -112,6 +111,15 @@ function(streamr_add_module_library TARGET) # BMI-compiling target on the consumer side and requires the standard # level to be part of the exported usage requirements. target_compile_features(${TARGET} PUBLIC cxx_std_26) + # Importers must agree with the BMI on POSIX-thread support: clang + # validates language options when loading a module file. On Android, + # -pthread is not implied — test executables inherit it from GTest's + # Threads::Threads while library targets do not, so a BMI built + # without it cannot be loaded by an import-using test TU (observed: + # -Wmodule-file-config-mismatch, then every imported name reported as + # undeclared). Making the flag part of the PUBLIC interface keeps the + # BMI and every importer consistent. No-op on other platforms. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_target_module_sources( FILES ...) @@ -135,6 +143,8 @@ function(streamr_target_module_sources TARGET) FILES ${ARG_FILES}) set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON) target_compile_features(${TARGET} PUBLIC cxx_std_26) + # See streamr_add_module_library: BMI/importer -pthread consistency. + target_compile_options(${TARGET} PUBLIC $<$:-pthread>) endfunction() # streamr_enable_imports() From 3213238af0fbee5e965c5b8f43a04c3402e80527 Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 02:40:08 +0300 Subject: [PATCH 6/7] Guard protoc plugin with NOT CMAKE_CROSSCOMPILING With Android modules enabled, the proto-rpc test/example block now runs on Android builds and exposed protobuf-streamr-plugin: it links protobuf::libprotoc, which vcpkg does not provide for cross targets. The plugin is host tooling (proto.sh codegen) and was only ever skipped on Android as a side effect of the modules gate being OFF there. Give it its own cross-compile guard, independent of modules support. Co-Authored-By: Claude Fable 5 --- packages/streamr-proto-rpc/CMakeLists.txt | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/packages/streamr-proto-rpc/CMakeLists.txt b/packages/streamr-proto-rpc/CMakeLists.txt index a28942ce..9abe80be 100644 --- a/packages/streamr-proto-rpc/CMakeLists.txt +++ b/packages/streamr-proto-rpc/CMakeLists.txt @@ -90,19 +90,26 @@ file(WRITE "${CMAKE_BINARY_DIR}/streamr-proto-rpc-config.cmake" "endif()\n" "include(${CMAKE_BINARY_DIR}/streamr-proto-rpc-config-in.cmake)\n") -# Test/example targets import the modules — skip them where modules are -# unsupported (Android; they are never executed there anyway). -if(NOT IOS AND STREAMR_MODULES_SUPPORTED) +# The protoc codegen plugin is host tooling (invoked by proto.sh to +# regenerate the checked-in *.pb.* files) — never built for cross +# targets: vcpkg does not provide protobuf::libprotoc for Android/iOS +# and the plugin runs on the developer machine anyway. (Previously this +# sat inside the modules block below and was skipped on Android only as +# a side effect of the modules gate being OFF there.) +if(NOT CMAKE_CROSSCOMPILING) add_executable(protobuf-streamr-plugin src/PluginCodeGeneratorMain.cpp include/streamr-proto-rpc/PluginCodeGenerator.hpp) target_include_directories( protobuf-streamr-plugin PRIVATE $ $) - target_link_libraries(protobuf-streamr-plugin + target_link_libraries(protobuf-streamr-plugin PRIVATE protobuf::libprotobuf protobuf::libprotoc) +endif() - +# Test/example targets import the modules — skip them where modules are +# unsupported (they are never executed on Android anyway). +if(NOT IOS AND STREAMR_MODULES_SUPPORTED) enable_testing() find_package(GTest CONFIG REQUIRED) From 95e94c6176e3440ff86425ef2a0e053237637c22 Mon Sep 17 00:00:00 2001 From: Petri Savolainen Date: Sat, 4 Jul 2026 03:47:06 +0300 Subject: [PATCH 7/7] Retrigger CI: flaky WebsocketClientServerTest on macos-26 Co-Authored-By: Claude Fable 5