You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Discrete diffusion pipelines fail with a CPU generator when running on GPU/MPS #15285
The reproducibility guide recommends a CPU torch.Generator. The image pipelines support this because randn_tensor samples on the CPU and moves the result to the device. The discrete diffusion code instead passes the generator straight to torch.randint / torch.multinomial / torch.rand on the pipeline's device, and PyTorch rejects a CPU generator there.
DiffusionGemmaPipeline fails on every call, even with greedy decoding, because the initial canvas is drawn with torch.randint(..., device=device, generator=generator).
LLaDA2Pipeline fails whenever temperature > 0.
The same pattern is in BlockRefinementScheduler, DiscreteDDIMScheduler and EntropyBoundScheduler (_sample_from_logits, the random-token renoising in step, and add_noise).
I tested on MPS. I don't have a CUDA machine, but PyTorch does the same device check on CUDA (see #6501), so CUDA should fail the same way.
Proposed fix: draw on generator.device and move the result to the pipeline's device, the same approach as #13228 for Helios. I'd like to open a PR for this with tests. #14741 keeps the same calls.
Reproduction
This uses the tiny checkpoint from the DiffusionGemma tests. The full model (google/diffusiongemma-26B-A4B-it) doesn't fit on my machine.
It runs with torch.Generator("mps"), or with both on CPU.
Context
No response
Logs
File "src/diffusers/pipelines/diffusion_gemma/pipeline_diffusion_gemma.py", line 347, in __call__
canvas = torch.randint(
RuntimeError: Expected a 'mps' device typefor generator but found 'cpu'
Confirming the CUDA half of this, which the report could only infer from MPS.
On CUDA (GTX 1650, torch 2.15.0+cu132, diffusers 0.43.0), the pipeline's exact call shape fails the same way as the reported MPS case:
torch.randint(0, 100, (2, 4), device="cuda", generator=torch.Generator("cpu").manual_seed(0))
# RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'
And the contrast that makes the pattern precise: diffusers' own randn_tensor accepts the same combination without error —
randn_tensor((2, 3), generator=torch.Generator("cpu").manual_seed(0), device="cuda")
# OK — returns a cuda:0 tensor
— because it draws on the generator's device and moves the result, which is the behaviour the reproducibility guide's CPU-generator advice relies on. So the split is exactly where the report puts it: code paths that sample through randn_tensor honour a CPU generator on a GPU pipeline; the discrete-diffusion sites that hand the generator straight to torch.randint / torch.rand / torch.multinomial on the pipeline device reject it, on CUDA just as on MPS.
Scope note: this verifies the device check at the call shape DiffusionGemmaPipeline uses for its initial canvas (the report's traceback site). I did not run the full pipeline or the LLaDA2 sampling path end-to-end; those share the same call pattern per the report's source read, but their confirmation is still the MPS one.
Activity
1OF1Research commentedon Nov 14, 2026
Confirming the CUDA half of this, which the report could only infer from MPS.
On CUDA (GTX 1650, torch 2.15.0+cu132, diffusers 0.43.0), the pipeline's exact call shape fails the same way as the reported MPS case:
And the contrast that makes the pattern precise: diffusers' own
randn_tensoraccepts the same combination without error —— because it draws on the generator's device and moves the result, which is the behaviour the reproducibility guide's CPU-generator advice relies on. So the split is exactly where the report puts it: code paths that sample through
randn_tensorhonour a CPU generator on a GPU pipeline; the discrete-diffusion sites that hand the generator straight totorch.randint/torch.rand/torch.multinomialon the pipeline device reject it, on CUDA just as on MPS.Scope note: this verifies the device check at the call shape
DiffusionGemmaPipelineuses for its initial canvas (the report's traceback site). I did not run the full pipeline or the LLaDA2 sampling path end-to-end; those share the same call pattern per the report's source read, but their confirmation is still the MPS one.Fix discrete diffusion pipelines with a CPU generator on GPU/MPS (hug…
Fix discrete diffusion pipelines with a CPU generator on GPU/MPS (hug…