Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/validateandroid.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
workflow_dispatch:

jobs:
install-lint:
install:
if: contains(github.event.head_commit.message, 'androidbuild') || contains(github.event.pull_request.title, 'androidbuild') || github.event_name == 'workflow_dispatch'
runs-on: macos-26
steps:
Expand All @@ -26,7 +26,7 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
- name: install
uses: ./.github/workflows/reusable/cached-install
- name: lint
run:
./lint.sh
shell: bash
# No lint step: linting is host-independent and runs on the
# validate.yml legs. On an Android tree the import-using test files
# have no compile commands (tests are not built on Android — see
# StreamrModules.cmake), so clangd produces spurious diagnostics.
39 changes: 38 additions & 1 deletion MODERNIZATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -575,7 +575,7 @@ The first `:protos` partition — the pattern rehearsal for 2.4/2.5.
generated client/server pb stubs), both examples build; downstream
dht/tn standalone builds unchanged; root tree 307/307; full lint.

## Phase 2.4 — streamr-dht + THE BENCH CHECKPOINT (PR pending)
## Phase 2.4 — streamr-dht + THE BENCH CHECKPOINT ✅ (PR #32, merged)
The biggest package migrated, and the checkpoint did exactly what it was
designed for: **it caught an architecture flaw by measurement and forced a
(good) design change.**
Expand Down Expand Up @@ -632,6 +632,43 @@ before any consolidation gains.
trackerless-network + proxyclient downstream builds unchanged; root
tree 307/307; full lint green.

## Phase 2.5 — trackerless-network + proxyclient, FINAL METRICS (PR pending)
The migration surface is complete: every C++ package that exports an API
now ships a module.
- **streamr-trackerless-network**: `:protos` over the NetworkRpc trio
(22 messages + 4 enums in the GLOBAL namespace — NetworkRpc.proto
declares no package, so the partition uses plain `export using ::X;` —
plus 6+6 generated stubs, which the protoc plugin emits in
`streamr::protorpc`) + coarse `:all` over the 13 public headers. All 12
test TUs flipped to `import streamr.trackerlessnetwork` with ZERO
fallout (first package to flip cleanly on the first build).
- **streamr-libstreamrproxyclient**: deliberately NO module. Its C header
`streamrproxyclient.h` is the permanent public ABI; the implementation
TU keeps textual includes until consolidation (its internal
`LibProxyClientApi.hpp` dies there anyway, and mixing import with an
internal header that reaches every stack would re-trigger the clangd
false-ODR issue). Verified unchanged: 15/15 tests.
- New standing gate applied (2.4 lesson): a LOCAL RELEASE build+test of
the migrated package (caught nothing this time — the folly/protobuf
overlay patches hold).

### FINAL FAÇADE-STAGE METRICS (vs Phase 2.0 baselines, macOS, idle)
| Metric | Baseline | Final (7 packages migrated) | Target | Verdict |
|---|---|---|---|---|
| Clean root build | 102 s | **78 s (−24%)** | −25% | ✅ effectively met (78–88 s across repeats) |
| `SLogger.hpp` touch | 62–70 s | 58 s (−6…−12%) | −40% | ❌ not yet — consolidation work |
| `StreamID.hpp` touch | 19 s | 32 s (+68%) | −40% | ❌ regression — BMI-chain latency |

**Honest reading**: the clean-build target is effectively met already at
the façade stage (test TUs load BMIs instead of re-parsing header
stacks). The incremental targets are NOT met and cannot be met by the
façade alone: while headers remain the textual source of truth inside the
`:all` GMFs, ANY header touch invalidates the package BMI and cascades
down the module DAG. Consolidation (2.6) is where the incremental wins
must come from — code moves into partitions, headers disappear, and a
one-partition edit stops invalidating whole packages. That was always the
sequencing; the numbers now quantify exactly why consolidation matters.

## Lint/IDE survival
- During the façade stage, headers remain the fully-linted source of truth
(`lint.sh` globs only `*.hpp/*.cpp`); `.cppm` added to clang-format only.
Expand Down
88 changes: 61 additions & 27 deletions cmake/StreamrModules.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,46 @@
# OFF globally (clean compile commands for clangd), and the helpers below
# re-enable scanning per target.

# CMake's C++ modules support requires the Ninja generator (Makefiles cannot
# express the dynamically discovered module dependency graph).
if(NOT CMAKE_GENERATOR MATCHES "Ninja")
message(FATAL_ERROR
"C++ modules require the Ninja generator; current generator is "
"'${CMAKE_GENERATOR}'. install-prerequisities.sh/setenvs.sh export "
"CMAKE_GENERATOR=Ninja — source one of them (and ./clean.sh once if "
"the build dir was configured with another generator).")
# Android builds use the NDK's clang (18/19), whose C++ modules support is
# too immature for the façade modules (known `export using` overload-set
# bugs — observed: `import streamr.json;` fails to provide the toJson
# overloads). Android consumes the ordinary headers instead (they remain the
# source of truth during the façade stage), the module units are skipped,
# and packages must not build their import-using test targets
# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships
# clang >= 22 — consolidation (2.6) requires it, as headers disappear then.
if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android")
set(STREAMR_MODULES_SUPPORTED OFF)
else()
set(STREAMR_MODULES_SUPPORTED ON)
endif()

# AppleClang has no CMake modules support; the build already uses Homebrew
# LLVM everywhere (homebrewClang.cmake), so this only guards misconfiguration.
if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang")
message(FATAL_ERROR
"CMake does not support C++ modules with AppleClang. The build is "
"expected to use Homebrew LLVM (see homebrewClang.cmake / "
"LLVM_PREFIX).")
endif()
if(STREAMR_MODULES_SUPPORTED)
# CMake's C++ modules support requires the Ninja generator (Makefiles
# cannot express the dynamically discovered module dependency graph).
if(NOT CMAKE_GENERATOR MATCHES "Ninja")
message(FATAL_ERROR
"C++ modules require the Ninja generator; current generator is "
"'${CMAKE_GENERATOR}'. install-prerequisities.sh/setenvs.sh export "
"CMAKE_GENERATOR=Ninja — source one of them (and ./clean.sh once if "
"the build dir was configured with another generator).")
endif()

# CMP0155 NEW: scan C++20+ sources for module dependencies by default.
# Scanning stays globally disabled via CMAKE_CXX_SCAN_FOR_MODULES OFF until
# a target opts in through the helpers below.
cmake_policy(SET CMP0155 NEW)
# AppleClang has no CMake modules support; the build already uses Homebrew
# LLVM everywhere (homebrewClang.cmake), so this only guards
# misconfiguration.
if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang")
message(FATAL_ERROR
"CMake does not support C++ modules with AppleClang. The build is "
"expected to use Homebrew LLVM (see homebrewClang.cmake / "
"LLVM_PREFIX).")
endif()

# CMP0155 NEW: scan C++20+ sources for module dependencies by default.
# Scanning stays globally disabled via CMAKE_CXX_SCAN_FOR_MODULES OFF
# until a target opts in through the helpers below.
cmake_policy(SET CMP0155 NEW)
endif()

# Opt-in `import std;` (experimental in CMake, OFF by default — see the
# MODERNIZATION.md decision: no rollout now). CMake gates the feature behind
Expand All @@ -59,18 +76,29 @@ endif()
# directory). STATIC rather than INTERFACE: module interface units are
# compiled TUs, so even a previously header-only package gains a compiled
# archive when it grows a module.
#
# Where modules are unsupported (Android/NDK), the target is still created
# as a STATIC library — with a generated stub source instead of module
# units — so callers can keep using PUBLIC include dirs/links identically.
function(streamr_add_module_library TARGET)
cmake_parse_arguments(ARG "" "" "FILES" ${ARGN})
if(NOT ARG_FILES)
message(FATAL_ERROR "streamr_add_module_library(${TARGET}): FILES is required")
endif()
add_library(${TARGET} STATIC)
target_sources(${TARGET}
PUBLIC
FILE_SET CXX_MODULES
BASE_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/modules
FILES ${ARG_FILES})
set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON)
if(STREAMR_MODULES_SUPPORTED)
target_sources(${TARGET}
PUBLIC
FILE_SET CXX_MODULES
BASE_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/modules
FILES ${ARG_FILES})
set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON)
else()
set(STUB ${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-no-modules-stub.cpp)
file(WRITE ${STUB}
"// C++ modules are not built on this platform (see StreamrModules.cmake).\n")
target_sources(${TARGET} PRIVATE ${STUB})
endif()
# PUBLIC compile feature (not just CMAKE_CXX_STANDARD): when another
# build tree imports this target from its export, CMake synthesizes a
# BMI-compiling target on the consumer side and requires the standard
Expand All @@ -83,12 +111,15 @@ endfunction()
# Adds C++ module interface units to an EXISTING library target (used by
# packages that already compile ordinary sources, e.g. generated protobuf
# .cc files). Same effect as streamr_add_module_library() minus the
# add_library().
# add_library(). No-op where modules are unsupported.
function(streamr_target_module_sources TARGET)
cmake_parse_arguments(ARG "" "" "FILES" ${ARGN})
if(NOT ARG_FILES)
message(FATAL_ERROR "streamr_target_module_sources(${TARGET}): FILES is required")
endif()
if(NOT STREAMR_MODULES_SUPPORTED)
return()
endif()
target_sources(${TARGET}
PUBLIC
FILE_SET CXX_MODULES
Expand All @@ -105,5 +136,8 @@ endfunction()
# the global CMAKE_CXX_SCAN_FOR_MODULES OFF would leave the import edges
# undiscovered and the build would race the BMIs.
function(streamr_enable_imports TARGET)
if(NOT STREAMR_MODULES_SUPPORTED)
return()
endif()
set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON)
endfunction()
9 changes: 9 additions & 0 deletions overlayports/folly/JUSTIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,12 @@ the protobuf `VarintParseSlowArm` overlay patch (anonymous-namespace /
static definitions in headers are incompatible with GMF+textual mixing).

Delete when upstream folly stops using anonymous namespaces in headers.

## supports expression relaxed for Android (added Phase 2.5)

The upstream 2026 port marks Android unsupported (`!android`), which
broke `./install.sh --android` outright at dependency-install time. This
monorepo built folly for arm64-android with the pre-2026 baseline, and the
Android build is a supported product target (Kotlin wrapper) — the overlay
removes `!android` so vcpkg attempts the build. If folly-on-NDK regresses
in a future baseline, this is the first place to look.
2 changes: 1 addition & 1 deletion overlayports/folly/vcpkg.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"description": "An open-source C++ library developed and used at Facebook. The library is UNSTABLE on Windows",
"homepage": "https://github.com/facebook/folly",
"license": "Apache-2.0",
"supports": "(windows & x64 & !uwp & !mingw) | (!windows & !android & (x64 | arm64))",
"supports": "(windows & x64 & !uwp & !mingw) | (!windows & (x64 | arm64))",
"dependencies": [
"boost-chrono",
"boost-context",
Expand Down
4 changes: 3 additions & 1 deletion packages/streamr-dht/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,9 @@ file(WRITE "${CMAKE_BINARY_DIR}/streamr-dht-config.cmake"
"endif()\n"
"include(${CMAKE_BINARY_DIR}/streamr-dht-config-in.cmake)\n")

if(NOT IOS)
# Test/example targets import the modules — skip them where modules are
# unsupported (Android; they are never executed there anyway).
if(NOT IOS AND STREAMR_MODULES_SUPPORTED)
enable_testing()
find_package(GTest CONFIG REQUIRED)

Expand Down
88 changes: 61 additions & 27 deletions packages/streamr-dht/StreamrModules.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,46 @@
# OFF globally (clean compile commands for clangd), and the helpers below
# re-enable scanning per target.

# CMake's C++ modules support requires the Ninja generator (Makefiles cannot
# express the dynamically discovered module dependency graph).
if(NOT CMAKE_GENERATOR MATCHES "Ninja")
message(FATAL_ERROR
"C++ modules require the Ninja generator; current generator is "
"'${CMAKE_GENERATOR}'. install-prerequisities.sh/setenvs.sh export "
"CMAKE_GENERATOR=Ninja — source one of them (and ./clean.sh once if "
"the build dir was configured with another generator).")
# Android builds use the NDK's clang (18/19), whose C++ modules support is
# too immature for the façade modules (known `export using` overload-set
# bugs — observed: `import streamr.json;` fails to provide the toJson
# overloads). Android consumes the ordinary headers instead (they remain the
# source of truth during the façade stage), the module units are skipped,
# and packages must not build their import-using test targets
# (STREAMR_MODULES_SUPPORTED below guards them). Revisit when the NDK ships
# clang >= 22 — consolidation (2.6) requires it, as headers disappear then.
if(VCPKG_TARGET_TRIPLET MATCHES "android" OR CMAKE_SYSTEM_NAME STREQUAL "Android")
set(STREAMR_MODULES_SUPPORTED OFF)
else()
set(STREAMR_MODULES_SUPPORTED ON)
endif()

# AppleClang has no CMake modules support; the build already uses Homebrew
# LLVM everywhere (homebrewClang.cmake), so this only guards misconfiguration.
if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang")
message(FATAL_ERROR
"CMake does not support C++ modules with AppleClang. The build is "
"expected to use Homebrew LLVM (see homebrewClang.cmake / "
"LLVM_PREFIX).")
endif()
if(STREAMR_MODULES_SUPPORTED)
# CMake's C++ modules support requires the Ninja generator (Makefiles
# cannot express the dynamically discovered module dependency graph).
if(NOT CMAKE_GENERATOR MATCHES "Ninja")
message(FATAL_ERROR
"C++ modules require the Ninja generator; current generator is "
"'${CMAKE_GENERATOR}'. install-prerequisities.sh/setenvs.sh export "
"CMAKE_GENERATOR=Ninja — source one of them (and ./clean.sh once if "
"the build dir was configured with another generator).")
endif()

# CMP0155 NEW: scan C++20+ sources for module dependencies by default.
# Scanning stays globally disabled via CMAKE_CXX_SCAN_FOR_MODULES OFF until
# a target opts in through the helpers below.
cmake_policy(SET CMP0155 NEW)
# AppleClang has no CMake modules support; the build already uses Homebrew
# LLVM everywhere (homebrewClang.cmake), so this only guards
# misconfiguration.
if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang")
message(FATAL_ERROR
"CMake does not support C++ modules with AppleClang. The build is "
"expected to use Homebrew LLVM (see homebrewClang.cmake / "
"LLVM_PREFIX).")
endif()

# CMP0155 NEW: scan C++20+ sources for module dependencies by default.
# Scanning stays globally disabled via CMAKE_CXX_SCAN_FOR_MODULES OFF
# until a target opts in through the helpers below.
cmake_policy(SET CMP0155 NEW)
endif()

# Opt-in `import std;` (experimental in CMake, OFF by default — see the
# MODERNIZATION.md decision: no rollout now). CMake gates the feature behind
Expand All @@ -59,18 +76,29 @@ endif()
# directory). STATIC rather than INTERFACE: module interface units are
# compiled TUs, so even a previously header-only package gains a compiled
# archive when it grows a module.
#
# Where modules are unsupported (Android/NDK), the target is still created
# as a STATIC library — with a generated stub source instead of module
# units — so callers can keep using PUBLIC include dirs/links identically.
function(streamr_add_module_library TARGET)
cmake_parse_arguments(ARG "" "" "FILES" ${ARGN})
if(NOT ARG_FILES)
message(FATAL_ERROR "streamr_add_module_library(${TARGET}): FILES is required")
endif()
add_library(${TARGET} STATIC)
target_sources(${TARGET}
PUBLIC
FILE_SET CXX_MODULES
BASE_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/modules
FILES ${ARG_FILES})
set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON)
if(STREAMR_MODULES_SUPPORTED)
target_sources(${TARGET}
PUBLIC
FILE_SET CXX_MODULES
BASE_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/modules
FILES ${ARG_FILES})
set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON)
else()
set(STUB ${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-no-modules-stub.cpp)
file(WRITE ${STUB}
"// C++ modules are not built on this platform (see StreamrModules.cmake).\n")
target_sources(${TARGET} PRIVATE ${STUB})
endif()
# PUBLIC compile feature (not just CMAKE_CXX_STANDARD): when another
# build tree imports this target from its export, CMake synthesizes a
# BMI-compiling target on the consumer side and requires the standard
Expand All @@ -83,12 +111,15 @@ endfunction()
# Adds C++ module interface units to an EXISTING library target (used by
# packages that already compile ordinary sources, e.g. generated protobuf
# .cc files). Same effect as streamr_add_module_library() minus the
# add_library().
# add_library(). No-op where modules are unsupported.
function(streamr_target_module_sources TARGET)
cmake_parse_arguments(ARG "" "" "FILES" ${ARGN})
if(NOT ARG_FILES)
message(FATAL_ERROR "streamr_target_module_sources(${TARGET}): FILES is required")
endif()
if(NOT STREAMR_MODULES_SUPPORTED)
return()
endif()
target_sources(${TARGET}
PUBLIC
FILE_SET CXX_MODULES
Expand All @@ -105,5 +136,8 @@ endfunction()
# the global CMAKE_CXX_SCAN_FOR_MODULES OFF would leave the import edges
# undiscovered and the build would race the BMIs.
function(streamr_enable_imports TARGET)
if(NOT STREAMR_MODULES_SUPPORTED)
return()
endif()
set_target_properties(${TARGET} PROPERTIES CXX_SCAN_FOR_MODULES ON)
endfunction()
4 changes: 3 additions & 1 deletion packages/streamr-eventemitter/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@ export(TARGETS streamr-eventemitter
file(WRITE "${CMAKE_BINARY_DIR}/streamr-eventemitter-config.cmake"
"list(APPEND CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH})\n"
"include(${CMAKE_BINARY_DIR}/streamr-eventemitter-config-in.cmake)\n")
if(NOT IOS)
# Test/example targets import the modules — skip them where modules are
# unsupported (Android; they are never executed there anyway).
if(NOT IOS AND STREAMR_MODULES_SUPPORTED)
enable_testing()
find_package(GTest CONFIG REQUIRED)

Expand Down
Loading
Loading