fix: native Linux CUDA/ROCm backend download (Resolves #767) - #851
Open
1337hero wants to merge 4 commits into
Open
fix: native Linux CUDA/ROCm backend download (Resolves #767)#8511337hero wants to merge 4 commits into
1337hero wants to merge 4 commits into
Conversation
Installing the CUDA or ROCm backend on Linux downloaded a Windows .exe and
silently fell back to CPU: the release assets weren't platform-specific and
only Windows binaries were published.
- Platform-qualify GPU server assets: voicebox-server-{cuda,rocm}-<plat>.tar.gz
and {cuda,rocm}-libs-<plat>-<version>.tar.gz, selected at download time via
server_asset_platform().
- Add build-cuda-linux / build-rocm-linux release jobs. ROCm pulls torch from
the PyTorch ROCm index (whose wheels bundle the HIP/rocBLAS/hipBLASLt/aotriton
runtime) instead of the Windows-only rocm_sdk wheels; GPU torch is installed
before requirements.txt so pip never resolves the default CUDA stack.
- build_binary.py --rocm: gate the Windows rocm_sdk collection/hook on Windows;
on Linux PyInstaller's torch hook collects the bundled .so runtime directly.
- package_{rocm,cuda}.py: --platform flag; Linux-aware classifier (leading-lib
and versioned .so, torch/lib/<lib>/ + torch/share/miopen kernel-data trees).
- Offer the ROCm download on Linux AMD hosts (/dev/kfd), gated to platforms we
actually build so unsupported arches aren't offered a 404.
- Pin free-disk-space by SHA; unit-test the asset-platform contract.
Verified end-to-end on Linux/gfx1201 (3x Radeon AI PRO R9700): the built ELF
reports gpu_available + ROCm + backend_variant=rocm. CUDA Linux path is
build/test-verified only (no NVIDIA hardware).
- require --platform in package_cuda/package_rocm; drop the triplicated platform_tag() auto-detect (CI always passes it explicitly) - share matches_lib_prefix() and sha256_file() from package_cuda in package_rocm instead of copy-pasting - collapse build-cuda-linux/build-rocm-linux into one matrixed build-gpu-linux job so the variants can't drift
Contributor
📝 WalkthroughWalkthroughThis PR adds platform-aware GPU asset naming across the release pipeline, introduces Linux ROCm capability detection, updates packaging and download flows to use platform-qualified archive names, and adds a Linux GPU release build job. ChangesPlatform-qualified GPU assets and Linux ROCm support
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant health.py
participant platform_detect.py
health.py->>platform_detect.py: is_amd_rocm_capable()
platform_detect.py->>platform_detect.py: server_asset_platform()
alt Windows
platform_detect.py->>platform_detect.py: is_amd_gpu_windows()
else Linux
platform_detect.py->>platform_detect.py: check /dev/kfd exists
end
platform_detect.py-->>health.py: supports_rocm boolean
sequenceDiagram
participant cuda.py / rocm.py
participant platform_detect.py
participant GitHub Release
cuda.py / rocm.py->>platform_detect.py: server_asset_platform()
platform_detect.py-->>cuda.py / rocm.py: platform token
cuda.py / rocm.py->>GitHub Release: fetch server/libs archive with platform token in filename
GitHub Release-->>cuda.py / rocm.py: archive + sha256
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/release.yml:
- Around line 367-381: The new write-permission job in release workflow needs
hardening around checkout and pip caching. Update the `actions/checkout@v4` step
in `build-gpu-linux` to disable persisted git credentials via
`persist-credentials: false`, and review the `actions/setup-python@v5` step so
`cache: "pip"` is either removed or narrowly scoped for this job because it can
be used before a release push. Use the `build-gpu-linux` job and the
`actions/checkout` / `actions/setup-python` steps as the identifiers to locate
the changes.
In `@backend/services/cuda.py`:
- Around line 290-292: The CUDA download path in cuda.py is using
server_asset_platform() without checking whether the host is actually supported,
which can produce a non-existent asset name on unsupported platforms. Add a
CUDA-specific platform guard in the download flow before constructing
server_archive and libs_archive, using the same kind of supported-platform check
pattern as SUPPORTED_GPU_ASSET_PLATFORMS, so unsupported hosts are rejected
before any download is attempted.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 63abd598-391d-4132-abc7-5a52c70d08e1
⛔ Files ignored due to path filters (1)
bun.lockis excluded by!**/*.lock
📒 Files selected for processing (15)
.github/workflows/release.ymlCHANGELOG.mdapp/src/components/ServerTab/GpuPage.tsxapp/src/lib/api/types.tsbackend/build_binary.pybackend/models.pybackend/routes/health.pybackend/services/cuda.pybackend/services/rocm.pybackend/tests/test_amd_gpu_detect.pybackend/tests/test_package_rocm.pybackend/tests/test_rocm_download.pybackend/utils/platform_detect.pyscripts/package_cuda.pyscripts/package_rocm.py
…coping - Add SUPPORTED_GPU_ASSET_PLATFORMS guard in CUDA download flow to reject unsupported platforms before constructing asset URLs - Disable persisted git credentials in build-gpu-linux checkout step - Remove unused pip cache from build-gpu-linux setup-python step Both changes reduce the attack surface of write-permission release jobs and prevent silent 404s on unsupported platforms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Build and publish Linux CUDA and ROCm GPU server binaries. Installing either backend on Linux now downloads a native ELF, not a Windows .exe.
TLDR
PR #817 shipped Linux ROCm setup —
just setup-pythonpicks the right PyTorch index for AMD or NVIDIA, and the Docker entrypoint grants GPU device access. But the downloadable GPU backends (Settings → GPU → Install) still only published Windows assets. On Linux, the download "succeeded" and extracted a Windows binary. Voicebox fell back to CPU silently. Whomp whomp.Why the other approach falls short
PR #770 blocks the Linux CUDA download with a 409 Conflict and tells users to run a Python backend instead. It's an OKish error message, not a fix. It leaves the ROCm path untouched, does not publish Linux server binaries, and asks Linux GPU users to bypass the app's own backend management.
This PR closes the gap: the same download flow works on Windows and Linux. No alternate workflow, no "use something else."
What changed
Release pipeline (.github/workflows/release.yml)
build-cuda-linuxandbuild-rocm-linuxjobs. Each installs the GPU-specific PyTorch index first, builds with PyInstaller, and packages into platform-qualified archives.-windows-x86_64to asset names.Asset naming (all download URLs)
voicebox-server-cuda.tar.gz→voicebox-server-cuda-linux-x86_64.tar.gzcuda-libs-cu128-v1.tar.gz→cuda-libs-linux-x86_64-cu128-v1.tar.gzserver_asset_platform().Linux ROCm torch (vs. Windows)
rocm_sdkwheels. The PyInstaller hook collects them explicitly.download.pytorch.org/whl/rocmbundle HIP/rocBLAS/hipBLASLt/aotriton.soinsidetorch/lib/. PyInstaller collects them automatically. Thebuild_binary.py --rocmpath now gates the explicitrocm_sdkcollection on Windows.Packaging scripts (package_cuda.py, package_rocm.py)
--platformflag for cross-platform CI runs. Default auto-detects from the host.libprefix, versioned.so(e.g.libaotriton_v2.so.0.11.2),torch/lib/<library>/kernel-data trees,torch/share/miopen/kernel DBs.rocrollerandaotritonto ROCm file prefixes.ROCm download gate (UI)
is_amd_rocm_capable()replacesis_amd_gpu_windows(). On Linux it checks/dev/kfd. Both paths are gated onSUPPORTED_GPU_ASSET_PLATFORMSso unsupported arches (e.g.linux-arm64) are not offered a 404.Tests
server_asset_platform()unit tests pin the exact platform tokens.is_amd_rocm_capable()tests cover Linux with/without/dev/kfd, ARM rejection, macOS rejection.Tested
gpu_available: true,backend_variant: "rocm". End-to-end TTS generation verified.Closes #767
Summary by CodeRabbit
.sha256files.