Skip to content

Discrete diffusion pipelines fail with a CPU generator when running on GPU/MPS #15285

Open

Description

@yyu-260

Describe the bug

The reproducibility guide recommends a CPU torch.Generator. The image pipelines support this because randn_tensor samples on the CPU and moves the result to the device. The discrete diffusion code instead passes the generator straight to torch.randint / torch.multinomial / torch.rand on the pipeline's device, and PyTorch rejects a CPU generator there.

  • DiffusionGemmaPipeline fails on every call, even with greedy decoding, because the initial canvas is drawn with torch.randint(..., device=device, generator=generator).
  • LLaDA2Pipeline fails whenever temperature > 0.
  • The same pattern is in BlockRefinementScheduler, DiscreteDDIMScheduler and EntropyBoundScheduler (_sample_from_logits, the random-token renoising in step, and add_noise).

I tested on MPS. I don't have a CUDA machine, but PyTorch does the same device check on CUDA (see #6501), so CUDA should fail the same way.

Proposed fix: draw on generator.device and move the result to the pipeline's device, the same approach as #13228 for Helios. I'd like to open a PR for this with tests. #14741 keeps the same calls.

Reproduction

This uses the tiny checkpoint from the DiffusionGemma tests. The full model (google/diffusiongemma-26B-A4B-it) doesn't fit on my machine.

import torch
from transformers import AutoProcessor, DiffusionGemmaForBlockDiffusion

from diffusers import BlockRefinementScheduler, DiffusionGemmaPipeline


model_id = "trl-internal-testing/tiny-DiffusionGemmaForBlockDiffusion"
pipe = DiffusionGemmaPipeline(
    model=DiffusionGemmaForBlockDiffusion.from_pretrained(model_id, dtype=torch.float32),
    scheduler=BlockRefinementScheduler(),
    processor=AutoProcessor.from_pretrained(model_id),
).to("mps")

generator = torch.Generator("cpu").manual_seed(0)
out = pipe(prompt="Name a color.", gen_length=32, num_inference_steps=4, generator=generator)
print(out.texts)

It runs with torch.Generator("mps"), or with both on CPU.

Context

No response

Logs

File "src/diffusers/pipelines/diffusion_gemma/pipeline_diffusion_gemma.py", line 347, in __call__
    canvas = torch.randint(
RuntimeError: Expected a 'mps' device type for generator but found 'cpu'

System Info

  • 🤗 Diffusers version: 0.42.0.dev0 (main @ f598e6e)
  • Platform: macOS-26.6.2-arm64-arm-64bit
  • Python version: 3.11.14
  • PyTorch version (GPU?): 2.15.0 (False)
  • Transformers version: 5.22.0
  • Accelerator: Apple M1 Pro
  • Using GPU in script?: Yes (MPS)
  • Using distributed or parallel set-up in script?: No

Who can help?

@yiyixuxu @dg845

Activity

  1. 1OF1Research commented on Nov 14, 2026

    @1OF1Research

    Confirming the CUDA half of this, which the report could only infer from MPS.

    On CUDA (GTX 1650, torch 2.15.0+cu132, diffusers 0.43.0), the pipeline's exact call shape fails the same way as the reported MPS case:

    torch.randint(0, 100, (2, 4), device="cuda", generator=torch.Generator("cpu").manual_seed(0))
    # RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'

    And the contrast that makes the pattern precise: diffusers' own randn_tensor accepts the same combination without error —

    randn_tensor((2, 3), generator=torch.Generator("cpu").manual_seed(0), device="cuda")
    # OK — returns a cuda:0 tensor

    — because it draws on the generator's device and moves the result, which is the behaviour the reproducibility guide's CPU-generator advice relies on. So the split is exactly where the report puts it: code paths that sample through randn_tensor honour a CPU generator on a GPU pipeline; the discrete-diffusion sites that hand the generator straight to torch.randint / torch.rand / torch.multinomial on the pipeline device reject it, on CUDA just as on MPS.

    Scope note: this verifies the device check at the call shape DiffusionGemmaPipeline uses for its initial canvas (the report's traceback site). I did not run the full pipeline or the LLaDA2 sampling path end-to-end; those share the same call pattern per the report's source read, but their confirmation is still the MPS one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions