Repository navigation
Build ROCm against TheRock 10.0.0, the ROCm XLA uses - #237
Merged
Merged
Conversation
XLA now builds its MORI collective kernels for ROCm, which call __builtin_amdgcn_global_load_b128, a builtin of AMD's downstream clang the 7.10.0a20251103 nightly does not have. Move to the TheRock multiarch 10.0.0 release XLA itself pins (third_party/gpus/rocm/rocm_redist.bzl), whose clang has it for every target we build. - The platform tag follows: gpu_version 10.0 in place of 7.1, in the build matrix and in the runtime platform augmentation. - The soname suffix is stripped by pattern, not by the nightly's exact version, and the clang resource directory is read from the tarball. - Enzyme-JAX's patch to XLA's ROCm crosstool hardcodes clang/22/include (the resource directory it declares to the inclusion check); 10.0 ships clang 23, so alias clang/22 to it and point hipcc's -isystem paths through the alias. - 10.0 ships amdgcn -> lib/llvm/amdgcn itself; creating it again made ln follow the link and nest a self-loop (lib/llvm/amdgcn/amdgcn), which breaks the rocm repository's amdgcn/** glob. Drop that ln. - 10.0 ships neither libhipsolver_fortran nor libamd_comgr_loader; drop their installs and the libhipsolver_fortran product. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016zErYp7upmqr4NHfhod9UD
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enzyme-JAX's JAX bump (EnzymeAD/Enzyme-JAX#3368) fails the ROCm Reactant_jll build. The new XLA builds its MORI collective kernels (
xla/backends/gpu/collectives/mori_kernels.cu.cc, underif_rocm_is_configured), and MORI calls__builtin_amdgcn_global_load_b128:Why a ROCm bump:
7.10.0a20251103, whose clang doesn't have it.third_party/gpus/rocm/rocm_redist.bzl, same sha256 as here).AMD clang 23.0.0git) compiles the builtin toglobal_load_dwordx4/global_load_b128for every target inTF_ROCM_AMDGPU_TARGETS: gfx908, gfx90a, gfx942, gfx950, gfx1030, gfx1100, gfx1101, gfx1200 and gfx1201.Changes:
therock-dist-linux-multiarch-10.0.0.tar.gz, replacing the gfx120X nightly.gpu_versionbecomes10.0instead of7.1, in the build matrix and inplatform_augmentation.jl, which picks the ROCm build when an AMD driver is present. Reactant.jl doesn't refer to the tag.*.so.7.1.25442-19ae9ff849) become a pattern that strips TheRock's build suffix. 10.0 names them*.so.7.15.26333-0000000. I checked the loop on the 10.0 files: the.so.7links resolve.clang/23; 7.1 shippedclang/22.cuda_wrappersoverrides now read the version from the tarball.workspace.bzlpatches XLA's ROCm crosstool with a hardcodedlib/llvm/lib/clang/22/include:rocm_configure.bzltakes its realpath, and the crosstool declares it to the inclusion check. The first CI run failed there (/workspace/srcdir/lib/llvm/lib/clang/22 (No such file or directory)). The recipe now symlinksclang/22→clang/23and spells hipcc's-isystempaths throughclang/22, so both declared spellings match what the compiler reports. Checked on the 10.0 tree:clang/22/includeresolves toclang/23/include, includingstddef.handcuda_wrappers.amdgcnlink: 10.0 already shipsamdgcn -> lib/llvm/amdgcn. The recipe's ownln -sfollowed it and createdlib/llvm/amdgcn/amdgcn, a link back to its own directory, so the second CI run failed globbingrocm_dist/amdgcn/**(Symbolic link loop). Thatlnis removed.libhipsolver_fortranand nolibamd_comgr_loader(libamd_comgr.so.3is still there). Their installs and thelibhipsolver_fortranproduct are dropped; Reactant.jl uses neither. Every other installed library and product exists in 10.0.Not verified here: the full BinaryBuilder build; that's what this PR's CI runs. Changing the ROCm toolchain also invalidates the cached host tools on that job, so it will hit the
__throw_bad_array_new_lengthfailure until #236 is merged.🤖 Generated with Claude Code
https://claude.ai/code/session_016zErYp7upmqr4NHfhod9UD