Working Linux install (Fedora 43, RTX 40-series, Python 3.12)

[Liatach]

Getting Trellis2 GGUF working on Linux (Fedora 43, RTX 40-series)

Posting a complete, working install for Linux + an RTX 40-series (Ada / sm_89) card, plus the
full list of problems hit along the way. The short version: the prebuilt Linux wheels don't
include Ada (sm_89) kernels, so several native extensions must be rebuilt from source. Once
that's done, the whole image-to-3D pipeline runs.

---

Part 1 — Clean install instructions

Environment

• Fedora 43, RTX 4070 Ti (12 GB), Nvidia driver 580 (reports CUDA 13.0)
• Python 3.12, PyTorch 2.8.0+cu128, CUDA Toolkit 12.8

1. CUDA 12.8 toolkit (independent of the 13.0 driver)

sudo dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora41/x86_64/cuda-fedora41.repo
sudo dnf install cuda-toolkit-12-8
nvcc --version   # should say release 12.8

2. GCC 14 (CUDA 12.8 nvcc does not support Fedora 43's default GCC 15)

sudo dnf install gcc14 gcc14-c++

3. Python 3.12 venv + PyTorch

python3.12 -m venv ~/ai/comfy-env
source ~/ai/comfy-env/bin/activate

# scope CUDA + Ada arch to the venv
cat >> ~/ai/comfy-env/bin/activate << 'EOF'

export CUDA_HOME=/usr/local/cuda-12.8
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
export TORCH_CUDA_ARCH_LIST="8.9"
EOF
source ~/ai/comfy-env/bin/activate

pip install torch==2.8.0+cu128 torchvision==0.23.0+cu128 torchaudio==2.8.0+cu128 --index-url https://download.pytorch.org/whl/cu128

4. Pin torch so later installs can't upgrade it

echo "torch==2.8.0+cu128
torchvision==0.23.0+cu128
torchaudio==2.8.0+cu128" > ~/ai/constraints.txt

mkdir -p ~/ai/comfy-env/pip
cat > ~/ai/comfy-env/pip/pip.ini << 'EOF'
[global]
constraint = /home/<user>/ai/constraints.txt
extra-index-url = https://download.pytorch.org/whl/cu128
EOF

5. ComfyUI + nodes

cd ~/ai
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI && pip install -r requirements.txt
cd custom_nodes
git clone https://github.com/visualbruno/ComfyUI-Trellis2.git
git clone https://github.com/Aero-Ex/ComfyUI-Trellis2-GGUF.git
git clone https://github.com/city96/ComfyUI-GGUF.git
pip install gguf

6. Patch the CUDA math header (glibc 2.41 fix — see Part 2)

Add noexcept (true) to sinpi/sinpif/cospi/cospif/rsqrt/rsqrtf declarations in
/usr/local/cuda-12.8/targets/x86_64-linux/include/crt/math_functions.h. (sed one-liners per
function; back up the file first.)

7. Install prebuilt wheels (the ones that work) with --no-deps

cd ~/ai/ComfyUI/custom_nodes/ComfyUI-Trellis2/wheels/Linux/Torch291
for w in cumesh custom_rasterizer nvdiffrec_render; do
  pip install --force-reinstall --no-deps "$w"*.whl
done
pip install trimesh plyfile easydict zstandard

8. Rebuild the sm_89-sensitive extensions from source

export TORCH_CUDA_ARCH_LIST="8.9"
export CUDAHOSTCXX=/usr/bin/g++-14 CC=/usr/bin/gcc-14 CXX=/usr/bin/g++-14

# flex_gemm
git clone https://github.com/JeffreyXiang/FlexGEMM.git /tmp/FlexGEMM
pip install /tmp/FlexGEMM --no-build-isolation --force-reinstall --no-deps

# o_voxel (from microsoft/TRELLIS.2, recursive for bundled Eigen)
git clone --recursive https://github.com/microsoft/TRELLIS.2.git /tmp/TRELLIS.2
pip install /tmp/TRELLIS.2/o-voxel --no-build-isolation --force-reinstall --no-deps

# nvdiffrast (JIT-compiles at runtime; arch var must be set then too)
git clone -b v0.4.0 https://github.com/NVlabs/nvdiffrast.git /tmp/nvdiffrast
pip install /tmp/nvdiffrast --no-build-isolation --force-reinstall --no-deps

9. Apply the o_voxel tiled-mesh patch (GGUF fork needs it)

cp ~/ai/ComfyUI/custom_nodes/ComfyUI-Trellis2-GGUF/patch/flexible_dual_grid.py \
   ~/ai/comfy-env/lib64/python3.12/site-packages/o_voxel/convert/flexible_dual_grid.py

10. Models

hf download facebook/dinov3-vitl16-pretrain-lvd1689m --local-dir ~/ai/ComfyUI/models/facebook/dinov3-vitl16-pretrain-lvd1689m
hf download microsoft/TRELLIS.2-4B --local-dir ~/ai/ComfyUI/models/trellis2/TRELLIS.2-4B

(DINOv3 is gated — accept access on HF first. GGUF Q4_K_M weights auto-download on first run.)

11. Launch

cd ~/ai/ComfyUI
PYTORCH_ALLOC_CONF=expandable_segments:True python main.py --listen 127.0.0.1

Recommended settings for 12 GB

• Loader: GGUF Q4_K_M, flash_attn, low_vram=true
• Generator: pipeline_type=512 (not 1024_cascade), use_tiled_decoder=true
• Post-process: dual_contouring_resolution=512, texture_size=2048

---

Part 2 — Problems encountered and workarounds

1. undefined symbol: ...SymBool...guard_or_false... on import.
The prebuilt wheels need a newer PyTorch ABI than 2.7.0 exposes. Fix: use torch 2.8.0 (not 2.7.0).

2. CUDA version mismatch when pip builds wheels from source (detected 12.8 mismatches compiled 13.0).
The cu128 torch wheels report internal CUDA 13.0. When pip tries to build cumesh/flex_gemm from
source it fails. Fix: install all wheels with --no-deps so pip never rebuilds them.

3. cp312 wheels but wrong Python. The Linux wheels are cp312 — the venv must be Python 3.12,
not 3.13+.

4. torch keeps silently upgrading to 2.12.0 (CPU), re-breaking everything with
operator torchvision::nms does not exist. Installing misc deps pulls a newer torch.
Fix: pin torch in a constraints file referenced from pip.ini (Part 1 step 4).

5. flash_attn ABI mismatch. Need the wheel matching torch's _GLIBCXX_USE_CXX11_ABI.
Check with python -c "import torch; print(torch._C._GLIBCXX_USE_CXX11_ABI)" — was True here,
so use the cxx11abiTRUE flash-attn wheel (2.8.3, cu12, torch2.8, cp312), installed with --no-deps.

6. CUDA error: no kernel image is available at runtime (the big one).
torch's arch list is sm_70/75/80/86/90/100/120 — no sm_89. Ada cards have no compatible kernel
in the prebuilt extensions. Fix: rebuild flex_gemm, o_voxel, and nvdiffrast from source with
TORCH_CUDA_ARCH_LIST="8.9".

7. Source builds fail with 98 errors in hash.cu. Fedora 43's GCC 15 is too new for CUDA 12.8.
Fix: install GCC 14 and point nvcc at it with CUDAHOSTCXX=/usr/bin/g++-14.

8. After GCC 14, ~6 errors remain: exception specification is incompatible for cospi/sinpi/rsqrt.
This is the glibc 2.41 vs CUDA conflict (new noexcept specifiers). Fix: patch CUDA's
crt/math_functions.h to add noexcept (true) to those six declarations.

9. ImportError: cannot import name 'tiled_flexible_dual_grid_to_mesh'.
Microsoft's upstream o-voxel doesn't have the tiled function the GGUF fork calls (the import is at
module top, so toggling use_tiled_decoder off doesn't help). Fix: copy the GGUF fork's
patch/flexible_dual_grid.py over the installed o_voxel file. The tiled function is pure-Python
chunking over existing kernels — no recompile needed.

10. OOM on 12 GB at the shape decoder / remesh. 1024_cascade and 1024 dual-contouring exceed
12 GB. Fix: pipeline_type=512, dual_contouring_resolution=512, texture_size=2048, low_vram=true,
and launch with PYTORCH_ALLOC_CONF=expandable_segments:True.

11. nvdiffrast RasterizeGLContext still calls the CUDA rasterizer internally, so swapping
CUDA→GL context in nodes.py does NOT avoid the sm_89 kernel. Fix: rebuild nvdiffrast from source
(it JIT-compiles at runtime, so TORCH_CUDA_ARCH_LIST=8.9 must also be set in the runtime env,
not just at build).

12. UNRESOLVED — texture baking produces rainbow/scrambled output.
Geometry generates correctly end-to-end, but the baked texture is rainbow noise. Not yet diagnosed.
Suspects: linear-vs-sRGB color-space mismatch in the bake; bake_on_vertices/use_custom_normals
settings; reorient_vertices desyncing UVs vs attributes; or ComfyUI's Preview3D misreporting color
space (check the .glb in Blender or an external glTF viewer before assuming the bake is wrong).
If anyone has solved Ada/Linux texture baking on this stack, please comment.

[Liatach]

12. SOLVED — rainbow/scrambled texture = Q4_K_M quantization of the texture model.
The colors come out as smooth, over-saturated, image-mismatched "tie-dye" (rainbow vomit). Root cause:
the texture SLAT flow model (slat_flow_imgshape2tex) does not survive Q4_K_M quantization. Color is a
high-frequency, per-voxel signal and is far more quant-sensitive than geometry — the sparse-structure and
shape models tolerate Q4_K_M fine, but the Q4 texture DiT produces a coherent-but-wrong color volume.

What it is NOT (all ruled out by diagnosis, so don't chase these):

• Not a UV-baking bug. The bake is correct — adjacent UV texels map to spatially-adjacent mesh points
  (measured coherence ~0.0015 over a ~1-unit mesh).
• Not linear-vs-sRGB, not bake_on_vertices, not use_custom_normals, not reorient_vertices, not
  Preview3D. The rainbow is present in the raw voxel/vertex colors — it shows up the same way on the
  .glb in Blender. The colors are wrong at the source, before any baking or viewing.
• Not file corruption and not your code — every run produces it, including with vertex-color output and
  with the texture bake disabled entirely.

Diagnostic tell: the vertex colors are locally smooth (neighbouring vertices agree) but globally
saturated and uncorrelated with the photo — and the mean color is still roughly correct (e.g. skin tone).
A nearest-neighbour "color coherence" metric will call this "smooth" and mislead you; judge it from an
actual render, not a coherence number.

Fix: run the texture model at higher precision (Q8_0 is near-lossless). Geometry stays at Q4_K_M.

Easiest (whole pipeline) — in the GGUF loader node, set model_format to GGUF Q8_0. Works, but also
bumps the shape/sparse models to Q8 (a little more VRAM/disk).

Lean / texture-only (recommended for 12 GB) — download just the Q8_0 texture weights and let the texture
stage load them while everything else stays Q4_K_M. The GGUF loader reads the quant type from the file
itself (not the filename), so the simplest no-code method is to swap the file the texture stage already
looks for:

cd ~/ai/ComfyUI/models/Trellis2/texture
# get the higher-precision texture weights
hf download Aero-Ex/Trellis2-GGUF \
  texture/slat_flow_imgshape2tex_dit_1_3B_512_bf16_Q8_0.gguf \
  --local-dir ~/ai/ComfyUI/models/Trellis2
# keep the original, then put the Q8_0 weights where the Q4_K_M loader expects them
mv slat_flow_imgshape2tex_dit_1_3B_512_bf16_Q4_K_M.gguf \
   slat_flow_imgshape2tex_dit_1_3B_512_bf16_Q4_K_M.gguf.q4bak
cp slat_flow_imgshape2tex_dit_1_3B_512_bf16_Q8_0.gguf \
   slat_flow_imgshape2tex_dit_1_3B_512_bf16_Q4_K_M.gguf

Restart ComfyUI and re-run with the loader still on Q4_K_M; the texture stage now loads Q8 weights and the
colors come back. (Q5_K_M / Q6_K are also published if you want to trade a little quality for VRAM. Note
these are the 512 texture weights — if you switch to a 1024/Pixal3D workflow, grab the matching 1024
quant instead.)

[solun888]

Followed the steps and changed "TORCH_CUDA_ARCH_LIS" for 3080 ,working on cachyOS ! Thanks !

[Lozano71]

I can add couple of things:
First of all, prebuilt wheels for PyTorch 2.11.0 and cp313 are seem to be not usable. in requirements.txt mentioned Open3D which has no support of cp313, only cp312 and older. And also dinov3 has usable non-gated mirror here: https://huggingface.co/camenduru/dinov3-vitl16-pretrain-lvd1689m

[n00n0i]

Adding another verified working Linux setup (Ubuntu, RTX 4070 Ti SUPER 16GB, Python 3.13) — plus a root-cause finding for the no kernel image is available errors that Ada (sm_89) users hit on Linux.

Root cause of "no kernel image" on Ada / RTX 40-series

Inspected the bundled wheels with cuobjdump --list-elf / --list-ptx:

Wheel set	Embedded arch	Runs on sm_89 (RTX 40)?
wheels/Linux/Torch291 (cp312)	nvdiffrast, o_voxel, nvdiffrec_render, flex_gemm: sm_120 only (cubin and PTX)	❌ — sm_120 PTX can't JIT down; this produces no kernel image
wheels/Linux/Torch2110 (cp313)	sm_86 cubins (+ others)	✅ — sm_86 cubins are binary-compatible within compute capability 8.x
wheels/Linux/Torch270 (cp312)	sm_86 + sm_86 PTX	✅ (but the set is missing nvdiffrec_render)

So Ada users on Linux should use the Torch2110/cp313 set. Rebuilding from source is not needed.

Verified working combo (RTX 4070 Ti SUPER 16GB)

• Python 3.13 venv, ComfyUI master
• torch 2.11.0+cu130 — important: the Torch2110 wheels link libcudart.so.13, so the cu128 build fails at import with libcudart.so.13: cannot open shared object file
• wheels/Linux/Torch2110/*.whl from this repo
• city96/ComfyUI-GGUF installed alongside (required for the GGUF node's native dequant path — without it Q8_0 tensors load un-dequantized and load_state_dict fails with shape mismatches like 1632 vs 1536)
• PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True set at server launch
• Full TRELLIS.2 Q8_0 GGUF pipeline: image → GLB (485k faces, 2048² PBR) in ~250 s; no OOM with low_vram=true
• Note: open3d has no cp313 wheel yet, so its cleanup node is unavailable (pymeshlab/meshlib cover it)

FYI: three bugs found in the GGUF companion node (Aero-Ex/ComfyUI-Trellis2-GGUF — its issue tracker is disabled, noting here since users combine both)

1. patch/remeshing.py drops two init lines present in the original wheel source (hashmap_vox = _init_hashmap(...) and the equivalent hashmap_vert line in remesh_narrow_band_dc) → NameError: name 'hashmap_vox' is not defined in the standalone Remesh node.
2. With low_vram=true the NAF/JAFAR upsampler is tiled (naf_tile_factor=4) for the shape stage but forced to naf_tile_factor=1 for the texture stage → guaranteed OOM at 1024 on 16 GB cards. Allowing tiling on the texture stage as well fixes Pixal3D generation on 16 GB.
3. use_tiled_encoder / use_tiled_decoder_for_texture on the MeshTexturing node call SparseUnetVaeDecoder._tiled_forward, which doesn't exist → AttributeError. Leave both false.

Hope this saves other Ada users some time.

[n00n0i]

Follow-up on my earlier comment (RTX 4070 Ti SUPER 16GB / Python 3.13 / Torch2110 wheels):

Compatibility confirmation after updating ComfyUI 0.24.0 → 0.27.0: the full setup keeps working unchanged — Torch2110/cp313 wheel set, torch 2.11.0+cu130, city96/ComfyUI-GGUF alongside, TRELLIS.2 + Pixal3D GGUF pipelines all pass. No wheel rebuilds needed across the ComfyUI update. The same instance now also runs LTX-2.3 / Wan 2.2 / HunyuanVideo 1.5 / Ideogram 4 / Qwen-Image / Krea 2 GGUF-or-fp8 workflows side by side on the 16 GB card, so the Trellis wheel stack coexists fine with everything ComfyUI 0.27 ships.

One extra data point for Ada users: system RAM matters more than VRAM once several model families are cached — 31 GB RAM is workable but tight; keep_models_loaded=false on the Trellis/Pixal loaders is the difference between smooth swaps and OOM-kills when mixing with other 20 GB+ model families.