UNABLE TO GENERATE IMAGE.

[parthwagh9999]

Thank you for this model.

Everything is installed and configured.
Models are downloaded.
No output has generated.
Also, it is using ram, not VRAM of gpu.

Please tell me Am I missing something?

Under transformers folder

has cuda

Thank You

[primecai]

Thanks for trying out the model. Could you look into which line the code got stuck? I'm guessing it has something to do with Windows, as I have only tested the codebase on Linux.

[parthwagh9999]

Thanks for trying out the model. Could you look into which line the code got stuck? I'm guessing it has something to do with Windows, as I have only tested the codebase on Linux.

before running this, bunch of models has downloaded, but due to power cut, that draft is not saved in this reply.

it has downloaded flux dev

This is the debugging code

"
import argparse
import torch
from PIL import Image
from diffusers.utils import load_image

from pipeline import FluxConditionalPipeline
from transformer import FluxTransformer2DConditionalModel

pipe = None

def init_pipeline(model_path, lora_path):
"""Initialize the global pipeline (pipe)."""
global pipe
print("[INFO] Initializing the pipeline...")

try:
    transformer = FluxTransformer2DConditionalModel.from_pretrained(
        model_path,
        torch_dtype=torch.bfloat16,
        low_cpu_mem_usage=False,
        ignore_mismatched_sizes=True
    )
    print("[INFO] Transformer model loaded successfully.")
    
    pipe = FluxConditionalPipeline.from_pretrained(
        "black-forest-labs/FLUX.1-dev",
        transformer=transformer,
        torch_dtype=torch.bfloat16
    )
    print("[INFO] Pipeline loaded successfully.")

    pipe.load_lora_weights(lora_path)
    print("[INFO] LoRA weights loaded.")

    pipe.to("cuda")
    print("[INFO] Pipeline moved to GPU.")

except Exception as e:
    print(f"[ERROR] Failed to initialize pipeline: {e}")

def process_image_and_text(image, text, gemini_prompt, guidance, i_guidance, t_guidance):
"""Process the given image and text using the global pipeline."""
print("[INFO] Processing image and text...")

# Center-crop image
try:
    w, h = image.size
    print(f"[INFO] Original image size: {w}x{h}")
    
    min_size = min(w, h)
    image = image.crop(((w - min_size) // 2, 
                        (h - min_size) // 2, 
                        (w + min_size) // 2, 
                        (h + min_size) // 2))
    image = image.resize((512, 512))
    print("[INFO] Image resized to 512x512.")
    
    control_image = load_image(image)
    
except Exception as e:
    print(f"[ERROR] Failed to process the image: {e}")
    return None

try:
    print("[INFO] Running inference...")
    result = pipe(
        prompt=text.strip().replace("\n", "").replace("\r", ""),
        negative_prompt="",
        num_inference_steps=28,
        height=512,
        width=1024,
        guidance_scale=guidance,
        image=control_image,
        guidance_scale_real_i=i_guidance,
        guidance_scale_real_t=t_guidance,
        gemini_prompt=gemini_prompt,
    ).images[0]

    print("[INFO] Inference completed successfully.")
    return result

except Exception as e:
    print(f"[ERROR] Inference failed: {e}")
    return None

def parse_args():
parser = argparse.ArgumentParser(description="Run Diffusion Self-Distillation.")

parser.add_argument(
    "--model_path", 
    type=str, 
    default="/home/shengqu/repos/SimpleTuner/output/1x2_v1/checkpoint-172000/transformer", 
    help="Path to the model checkpoint."
)
parser.add_argument(
    "--lora_path", 
    type=str, 
    default="/home/shengqu/repos/SimpleTuner/output/1x2_v1/checkpoint-172000/pytorch_lora_weights.safetensors", 
    help="Path to the lora checkpoint."
)
parser.add_argument(
    "--image_path", 
    type=str, 
    required=True,
    help="Path to the input image."
)
parser.add_argument(
    "--text", 
    type=str, 
    required=True, 
    help="The text prompt."
)
parser.add_argument(
    "--disable_gemini_prompt", 
    action="store_true",
    help="Flag to disable gemini prompt. If not set, gemini_prompt is True."
)
parser.add_argument(
    "--guidance", 
    type=float, 
    default=3.5, 
    help="Guidance scale for the pipeline."
)
parser.add_argument(
    "--i_guidance", 
    type=float, 
    default=1.0, 
    help="Image guidance scale."
)
parser.add_argument(
    "--t_guidance", 
    type=float, 
    default=1.0, 
    help="Text guidance scale."
)
parser.add_argument(
    "--output_path", 
    type=str, 
    default="output.png", 
    help="Path to save the output image."
)

return parser.parse_args()

def main():
args = parse_args()

print("[INFO] Starting script...")

# Initialize pipeline
init_pipeline(args.model_path, args.lora_path)

# Open the image
try:
    image = Image.open(args.image_path).convert("RGB")
    print(f"[INFO] Loaded input image: {args.image_path}")

except Exception as e:
    print(f"[ERROR] Failed to open image: {e}")
    return

# Process image and text
result_image = process_image_and_text(
    image, 
    args.text, 
    not args.disable_gemini_prompt, 
    args.guidance, 
    args.i_guidance, 
    args.t_guidance
)

if result_image is not None:
    try:
        result_image.save(args.output_path)
        print(f"[INFO] Output saved to {args.output_path}")
    except Exception as e:
        print(f"[ERROR] Failed to save output image: {e}")
else:
    print("[ERROR] No output generated.")

if name == "main":
main()

"

still no image generated.

Thank you so much for quick reply.

[parthwagh9999]

And It is consuming my all 64 GB of ram, but not 24GB of VRAM.

H:\consistent character or image\diffusion-self-distillation>python generate.py --model_path "H:\consistent character or image\diffusion-self-distillation\transformers" --lora_path "H:\path\to\pytorch_lora_weights.safetensors" --image_path "C:\Users\parth\OneDrive\Pictures\Screenshots\Screenshot 2025-02-25 190417.png" --text "this character sitting on a chair" --output_path "output.png" --guidance 3.5 --i_guidance 1.0 --t_guidance 1.0
[INFO] Starting script...
[INFO] Initializing the pipeline...
Some weights of the model checkpoint at H:\consistent character or image\diffusion-self-distillation\transformers were not used when initializing FluxTransformer2DConditionalModel: ['transformer_blocks.14.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.21.attn.to_v.lora_B.default.weight', 'transformer_blocks.1.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.26.attn.to_v.lora_A.default.weight', 'transformer_blocks.10.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.5.attn.to_k.lora_A.default.weight', 'transformer_blocks.1.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.2.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.6.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.28.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.24.attn.to_k.lora_B.default.weight', 'transformer_blocks.17.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.36.attn.to_k.lora_B.default.weight', 'transformer_blocks.5.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.10.attn.to_k.lora_B.default.weight', 'transformer_blocks.8.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.37.attn.to_q.lora_A.default.weight', 'transformer_blocks.11.attn.to_v.lora_A.default.weight', 'transformer_blocks.0.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.33.attn.to_v.lora_A.default.weight', 'transformer_blocks.14.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.9.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.1.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.37.attn.to_v.lora_A.default.weight', 'transformer_blocks.15.attn.to_v.lora_A.default.weight', 'transformer_blocks.6.attn.to_out.0.lora_A.default.weight', 'single_transformer_blocks.14.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.22.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.4.attn.to_q.lora_B.default.weight', 'transformer_blocks.6.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.12.attn.to_q.lora_A.default.weight', 'transformer_blocks.13.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.6.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.36.attn.to_k.lora_A.default.weight', 'transformer_blocks.3.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.5.attn.to_q.lora_A.default.weight', 'transformer_blocks.8.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.13.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.23.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.12.attn.to_k.lora_B.default.weight', 'transformer_blocks.3.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.18.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.3.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.6.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.18.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.14.attn.to_k.lora_A.default.weight', 'transformer_blocks.3.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.18.attn.to_v.lora_B.default.weight', 'transformer_blocks.6.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.13.attn.to_q.lora_B.default.weight', 'transformer_blocks.6.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.7.attn.to_q.lora_A.default.weight', 'transformer_blocks.0.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.5.attn.to_k.lora_B.default.weight', 'transformer_blocks.16.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.28.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.32.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.4.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.25.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.3.attn.to_k.lora_A.default.weight', 'transformer_blocks.2.attn.to_k.lora_A.default.weight', 'transformer_blocks.1.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.8.attn.to_q.lora_B.default.weight', 'transformer_blocks.17.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.3.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.10.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.36.attn.to_v.lora_A.default.weight', 'transformer_blocks.12.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.2.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.30.attn.to_v.lora_B.default.weight', 'transformer_blocks.11.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.13.attn.to_q.lora_A.default.weight', 'transformer_blocks.13.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.17.attn.to_out.0.lora_A.default.weight', 'single_transformer_blocks.16.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.17.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.32.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.13.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.16.attn.to_k.lora_B.default.weight', 'transformer_blocks.11.attn.to_v.lora_B.default.weight', 'transformer_blocks.3.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.13.attn.to_v.lora_B.default.weight', 'transformer_blocks.13.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.12.attn.to_k.lora_B.default.weight', 'transformer_blocks.6.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.7.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.35.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.8.attn.to_k.lora_A.default.weight', 'transformer_blocks.2.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.32.attn.to_q.lora_A.default.weight', 'transformer_blocks.7.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.31.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.32.attn.to_k.lora_B.default.weight', 'transformer_blocks.1.attn.to_q.lora_A.default.weight', 'transformer_blocks.16.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.0.attn.to_k.lora_B.default.weight', 'transformer_blocks.2.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.6.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.20.attn.to_q.lora_B.default.weight', 'transformer_blocks.9.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.13.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.19.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.33.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.2.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.35.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.27.attn.to_k.lora_A.default.weight', 'transformer_blocks.9.attn.add_v_proj.lora_A.default.weight', 'single_transformer_blocks.8.attn.to_v.lora_B.default.weight', 'transformer_blocks.15.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.16.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.8.attn.to_k.lora_A.default.weight', 'transformer_blocks.1.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.28.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.37.attn.to_k.lora_A.default.weight', 'transformer_blocks.4.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.1.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.3.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.17.attn.to_v.lora_B.default.weight', 'transformer_blocks.6.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.1.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.15.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.34.attn.to_k.lora_A.default.weight', 'transformer_blocks.6.attn.to_q.lora_B.default.weight', 'transformer_blocks.1.attn.to_q.lora_B.default.weight', 'transformer_blocks.13.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.26.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.33.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.5.attn.to_q.lora_B.default.weight', 'transformer_blocks.0.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.8.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.12.attn.to_out.0.lora_A.default.weight', 'single_transformer_blocks.25.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.18.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.16.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.24.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.19.attn.to_k.lora_B.default.weight', 'transformer_blocks.2.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.9.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.29.attn.to_q.lora_A.default.weight', 'transformer_blocks.15.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.31.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.1.attn.to_q.lora_B.default.weight', 'transformer_blocks.12.attn.to_q.lora_B.default.weight', 'transformer_blocks.5.attn.to_v.lora_B.default.weight', 'transformer_blocks.14.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.13.attn.to_v.lora_B.default.weight', 'transformer_blocks.3.attn.to_v.lora_A.default.weight', 'transformer_blocks.8.attn.to_v.lora_B.default.weight', 'transformer_blocks.4.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.28.attn.to_k.lora_A.default.weight', 'transformer_blocks.13.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.34.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.0.attn.to_v.lora_A.default.weight', 'transformer_blocks.6.attn.to_k.lora_B.default.weight', 'transformer_blocks.0.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.25.attn.to_k.lora_A.default.weight', 'transformer_blocks.11.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.10.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.18.attn.add_v_proj.lora_A.default.weight', 'single_transformer_blocks.23.attn.to_q.lora_B.default.weight', 'transformer_blocks.8.attn.to_q.lora_A.default.weight', 'transformer_blocks.14.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.34.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.30.attn.to_k.lora_A.default.weight', 'transformer_blocks.16.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.8.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.35.attn.to_q.lora_A.default.weight', 'transformer_blocks.0.attn.to_k.lora_B.default.weight', 'transformer_blocks.8.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.20.attn.to_k.lora_A.default.weight', 'transformer_blocks.5.attn.to_k.lora_B.default.weight', 'transformer_blocks.10.attn.to_q.lora_A.default.weight', 'transformer_blocks.18.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.8.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.12.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.18.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.19.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.29.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.11.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.10.attn.to_q.lora_B.default.weight', 'transformer_blocks.8.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.14.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.11.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.12.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.0.attn.to_k.lora_A.default.weight', 'transformer_blocks.4.attn.to_q.lora_B.default.weight', 'transformer_blocks.3.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.11.attn.to_q.lora_A.default.weight', 'transformer_blocks.18.attn.to_q.lora_B.default.weight', 'transformer_blocks.7.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.4.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.18.attn.to_v.lora_A.default.weight', 'transformer_blocks.9.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.3.attn.to_q.lora_A.default.weight', 'transformer_blocks.16.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.1.attn.to_q.lora_A.default.weight', 'transformer_blocks.10.attn.to_k.lora_A.default.weight', 'transformer_blocks.4.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.9.attn.to_q.lora_A.default.weight', 'transformer_blocks.3.attn.to_q.lora_A.default.weight', 'transformer_blocks.9.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.30.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.7.attn.to_k.lora_A.default.weight', 'transformer_blocks.7.attn.to_k.lora_A.default.weight', 'transformer_blocks.13.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.20.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.27.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.2.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.15.attn.to_q.lora_A.default.weight', 'transformer_blocks.0.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.3.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.11.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.33.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.25.attn.to_k.lora_B.default.weight', 'transformer_blocks.8.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.1.attn.to_k.lora_B.default.weight', 'transformer_blocks.8.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.10.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.14.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.16.attn.to_v.lora_B.default.weight', 'transformer_blocks.12.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.1.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.34.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.22.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.24.attn.to_k.lora_A.default.weight', 'transformer_blocks.7.attn.add_v_proj.lora_A.default.weight', 'single_transformer_blocks.4.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.13.attn.to_k.lora_B.default.weight', 'transformer_blocks.9.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.21.attn.to_q.lora_B.default.weight', 'transformer_blocks.8.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.14.attn.to_q.lora_B.default.weight', 'transformer_blocks.17.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.28.attn.to_v.lora_B.default.weight', 'transformer_blocks.9.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.16.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.4.attn.to_v.lora_A.default.weight', 'transformer_blocks.6.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.8.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.8.attn.to_q.lora_B.default.weight', 'transformer_blocks.15.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.17.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.4.attn.to_k.lora_B.default.weight', 'transformer_blocks.2.attn.to_out.0.lora_A.default.weight', 'single_transformer_blocks.22.attn.to_q.lora_B.default.weight', 'transformer_blocks.10.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.6.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.6.attn.to_q.lora_B.default.weight', 'transformer_blocks.4.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.2.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.21.attn.to_q.lora_A.default.weight', 'transformer_blocks.7.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.17.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.34.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.26.attn.to_v.lora_B.default.weight', 'transformer_blocks.9.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.7.attn.to_v.lora_B.default.weight', 'transformer_blocks.3.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.7.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.0.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.0.attn.to_v.lora_B.default.weight', 'transformer_blocks.2.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.12.attn.to_v.lora_B.default.weight', 'transformer_blocks.16.attn.to_k.lora_A.default.weight', 'transformer_blocks.18.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.14.attn.to_v.lora_B.default.weight', 'transformer_blocks.9.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.18.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.27.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.19.attn.to_q.lora_A.default.weight', 'transformer_blocks.17.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.34.attn.to_v.lora_A.default.weight', 'transformer_blocks.2.attn.to_q.lora_B.default.weight', 'transformer_blocks.10.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.2.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.10.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.22.attn.to_k.lora_B.default.weight', 'transformer_blocks.7.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.3.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.10.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.15.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.7.attn.to_v.lora_B.default.weight', 'transformer_blocks.15.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.28.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.12.attn.to_q.lora_B.default.weight', 'transformer_blocks.6.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.27.attn.to_q.lora_B.default.weight', 'transformer_blocks.2.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.17.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.31.attn.to_q.lora_A.default.weight', 'transformer_blocks.9.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.12.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.21.attn.to_k.lora_B.default.weight', 'transformer_blocks.5.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.10.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.31.attn.to_k.lora_A.default.weight', 'transformer_blocks.16.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.14.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.33.attn.to_q.lora_B.default.weight', 'transformer_blocks.17.attn.to_k.lora_A.default.weight', 'transformer_blocks.0.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.14.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.32.attn.to_k.lora_A.default.weight', 'transformer_blocks.4.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.14.attn.to_v.lora_A.default.weight', 'transformer_blocks.10.attn.add_q_proj.lora_A.default.weight', 'single_transformer_blocks.19.attn.to_q.lora_B.default.weight', 'transformer_blocks.16.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.7.attn.to_k.lora_B.default.weight', 'transformer_blocks.17.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.6.attn.to_v.lora_B.default.weight', 'transformer_blocks.4.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.21.attn.to_k.lora_A.default.weight', 'transformer_blocks.17.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.14.attn.to_q.lora_B.default.weight', 'transformer_blocks.16.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.5.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.20.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.15.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.4.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.19.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.17.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.8.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.11.attn.to_k.lora_B.default.weight', 'transformer_blocks.9.attn.to_k.lora_B.default.weight', 'transformer_blocks.6.attn.to_v.lora_B.default.weight', 'transformer_blocks.7.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.5.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.36.attn.to_v.lora_B.default.weight', 'transformer_blocks.15.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.13.attn.to_q.lora_A.default.weight', 'transformer_blocks.1.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.11.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.3.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.25.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.2.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.24.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.0.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.27.attn.to_k.lora_B.default.weight', 'transformer_blocks.18.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.14.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.12.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.29.attn.to_v.lora_B.default.weight', 'transformer_blocks.16.attn.to_v.lora_A.default.weight', 'transformer_blocks.3.attn.to_out.0.lora_A.default.weight', 'single_transformer_blocks.27.attn.to_q.lora_A.default.weight', 'transformer_blocks.1.attn.to_v.lora_A.default.weight', 'transformer_blocks.6.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.32.attn.to_q.lora_B.default.weight', 'transformer_blocks.8.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.8.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.7.attn.to_q.lora_A.default.weight', 'transformer_blocks.11.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.4.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.15.attn.to_v.lora_B.default.weight', 'transformer_blocks.15.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.20.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.31.attn.to_q.lora_B.default.weight', 'transformer_blocks.17.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.5.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.14.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.17.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.9.attn.to_v.lora_B.default.weight', 'transformer_blocks.4.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.18.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.12.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.1.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.18.attn.to_k.lora_B.default.weight', 'transformer_blocks.0.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.12.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.15.attn.to_q.lora_B.default.weight', 'transformer_blocks.11.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.11.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.35.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.10.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.23.attn.to_q.lora_A.default.weight', 'transformer_blocks.13.attn.to_k.lora_B.default.weight', 'transformer_blocks.0.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.35.attn.to_v.lora_A.default.weight', 'transformer_blocks.14.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.6.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.9.attn.to_k.lora_A.default.weight', 'transformer_blocks.0.attn.to_v.lora_A.default.weight', 'transformer_blocks.15.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.13.attn.to_k.lora_A.default.weight', 'transformer_blocks.13.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.2.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.11.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.12.attn.to_k.lora_A.default.weight', 'transformer_blocks.10.attn.to_v.lora_A.default.weight', 'transformer_blocks.5.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.23.attn.to_k.lora_B.default.weight', 'transformer_blocks.17.attn.to_v.lora_B.default.weight', 'transformer_blocks.15.attn.to_q.lora_B.default.weight', 'transformer_blocks.14.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.17.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.11.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.22.attn.to_q.lora_A.default.weight', 'transformer_blocks.4.attn.to_v.lora_B.default.weight', 'transformer_blocks.17.attn.to_q.lora_B.default.weight', 'transformer_blocks.13.attn.to_k.lora_A.default.weight', 'transformer_blocks.17.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.3.attn.to_k.lora_B.default.weight', 'transformer_blocks.7.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.10.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.5.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.11.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.13.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.13.attn.to_out.0.lora_B.default.weight', 'transformer_blocks.15.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.5.attn.to_k.lora_A.default.weight', 'transformer_blocks.11.attn.to_q.lora_B.default.weight', 'transformer_blocks.2.attn.to_q.lora_A.default.weight', 'transformer_blocks.17.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.15.attn.to_k.lora_B.default.weight', 'transformer_blocks.16.attn.to_q.lora_A.default.weight', 'transformer_blocks.12.attn.add_k_proj.lora_B.default.weight', 'transformer_blocks.1.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.24.attn.to_v.lora_B.default.weight', 'transformer_blocks.18.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.3.attn.to_v.lora_A.default.weight', 'transformer_blocks.12.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.1.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.15.attn.to_q.lora_A.default.weight', 'transformer_blocks.10.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.16.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.9.attn.to_v.lora_A.default.weight', 'transformer_blocks.10.attn.add_q_proj.lora_B.default.weight', 'transformer_blocks.0.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.11.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.30.attn.to_q.lora_A.default.weight', 'transformer_blocks.12.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.37.attn.to_q.lora_B.default.weight', 'transformer_blocks.5.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.16.attn.to_v.lora_B.default.weight', 'transformer_blocks.2.attn.to_v.lora_A.default.weight', 'transformer_blocks.11.attn.to_k.lora_A.default.weight', 'transformer_blocks.5.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.26.attn.to_k.lora_B.default.weight', 'transformer_blocks.0.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.33.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.37.attn.to_k.lora_B.default.weight', 'transformer_blocks.11.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.4.attn.add_q_proj.lora_B.default.weight', 'single_transformer_blocks.37.attn.to_v.lora_B.default.weight', 'transformer_blocks.15.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.0.attn.to_out.0.lora_B.default.weight', 'single_transformer_blocks.0.attn.to_q.lora_B.default.weight', 'transformer_blocks.4.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.5.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.30.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.31.attn.to_v.lora_B.default.weight', 'transformer_blocks.5.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.18.attn.to_k.lora_A.default.weight', 'transformer_blocks.9.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.12.attn.add_v_proj.lora_A.default.weight', 'single_transformer_blocks.18.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.22.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.29.attn.to_k.lora_A.default.weight', 'transformer_blocks.17.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.30.attn.to_q.lora_B.default.weight', 'transformer_blocks.9.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.5.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.7.attn.to_k.lora_B.default.weight', 'transformer_blocks.10.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.2.attn.to_v.lora_A.default.weight', 'transformer_blocks.14.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.12.attn.to_v.lora_A.default.weight', 'transformer_blocks.18.attn.to_q.lora_A.default.weight', 'transformer_blocks.4.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.16.attn.to_add_out.lora_A.default.weight', 'single_transformer_blocks.12.attn.to_v.lora_A.default.weight', 'transformer_blocks.2.attn.to_k.lora_B.default.weight', 'transformer_blocks.14.attn.to_k.lora_A.default.weight', 'transformer_blocks.4.attn.to_v.lora_A.default.weight', 'transformer_blocks.5.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.6.attn.to_add_out.lora_B.default.weight', 'transformer_blocks.15.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.3.attn.to_k.lora_A.default.weight', 'transformer_blocks.10.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.15.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.15.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.11.attn.to_v.lora_B.default.weight', 'transformer_blocks.13.attn.to_out.0.lora_A.default.weight', 'transformer_blocks.7.attn.to_v.lora_A.default.weight', 'transformer_blocks.9.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.8.attn.add_v_proj.lora_A.default.weight', 'single_transformer_blocks.9.attn.to_k.lora_A.default.weight', 'transformer_blocks.11.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.7.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.0.attn.add_k_proj.lora_B.default.weight', 'single_transformer_blocks.29.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.1.attn.to_v.lora_B.default.weight', 'single_transformer_blocks.36.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.20.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.7.attn.to_v.lora_A.default.weight', 'transformer_blocks.10.attn.to_v.lora_B.default.weight', 'transformer_blocks.9.attn.to_q.lora_A.default.weight', 'single_transformer_blocks.36.attn.to_q.lora_A.default.weight', 'transformer_blocks.5.attn.to_q.lora_B.default.weight', 'transformer_blocks.1.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.24.attn.to_q.lora_B.default.weight', 'transformer_blocks.0.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.6.attn.to_v.lora_A.default.weight', 'transformer_blocks.3.attn.add_k_proj.lora_A.default.weight', 'transformer_blocks.4.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.2.attn.to_q.lora_B.default.weight', 'transformer_blocks.18.attn.add_v_proj.lora_B.default.weight', 'single_transformer_blocks.23.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.29.attn.to_v.lora_A.default.weight', 'transformer_blocks.16.attn.add_v_proj.lora_B.default.weight', 'transformer_blocks.14.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.25.attn.to_q.lora_B.default.weight', 'single_transformer_blocks.5.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.23.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.1.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.26.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.26.attn.to_q.lora_B.default.weight', 'transformer_blocks.2.attn.add_v_proj.lora_A.default.weight', 'transformer_blocks.14.attn.add_q_proj.lora_A.default.weight', 'transformer_blocks.1.attn.to_v.lora_B.default.weight', 'transformer_blocks.16.attn.add_k_proj.lora_A.default.weight', 'single_transformer_blocks.9.attn.to_k.lora_B.default.weight', 'single_transformer_blocks.35.attn.to_k.lora_A.default.weight', 'single_transformer_blocks.18.attn.to_k.lora_B.default.weight', 'transformer_blocks.7.attn.to_q.lora_B.default.weight', 'transformer_blocks.18.attn.to_add_out.lora_A.default.weight', 'transformer_blocks.7.attn.to_add_out.lora_B.default.weight', 'single_transformer_blocks.21.attn.to_v.lora_A.default.weight', 'single_transformer_blocks.3.attn.to_k.lora_B.default.weight', 'transformer_blocks.16.attn.to_out.0.lora_A.default.weight', 'single_transformer_blocks.18.attn.to_v.lora_B.default.weight']

• This IS expected if you are initializing FluxTransformer2DConditionalModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing FluxTransformer2DConditionalModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  [INFO] Transformer model loaded successfully.
  Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 12.08it/s]
  Loading pipeline components...: 86%|████████████████████████████████████████████▌ | 6/7 [00:00<00:00, 8.28it/s]You set add_prefix_space. The tokenizer needs to be converted from the slow tokenizers
  Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:01<00:00, 6.48it/s]
  [INFO] Pipeline loaded successfully.
  [ERROR] Failed to initialize pipeline: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'H:\path\to\pytorch_lora_weights.safetensors'.
  [INFO] Loaded input image: C:\Users\parth\OneDrive\Pictures\Screenshots\Screenshot 2025-02-25 190417.png
  [INFO] Processing image and text...
  [INFO] Original image size: 460x511
  [INFO] Image resized to 512x512.
  [INFO] Running inference...
  [ERROR] Inference failed: Your default credentials were not found. To set up Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information.
  [ERROR] No output generated.

H:\consistent character or image\diffusion-self-distillation>

[primecai]

I think there are two problems:
Firstly, you may need to set up Gemini. If you do not want to, please put --disable_gemini_prompt in your command (I strongly recommend setting up the Gemini API since it is quite helpful in describing the subject and enhances the prompt, which will make the outputs more robust).
Secondly, there is a Windows path issue. The transformer is loaded successfully as you see from the printed weights (those loads of prints are expected), but there is an error:
[ERROR] Failed to initialize pipeline: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'H:\path\to\pytorch_lora_weights.safetensors'.
You omitted the path, but I guess the original one has something that does not work for diffusers? Note the error message Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96.

[parthwagh9999]

Thank you so much, I will check and update.

[parthwagh9999]

I have tried so much, not able to run it. used Claude, to modify code, but not working.
Please let me know when you have built it for windows.

Thank you so much for your help.

[rockerBOO]

The code uses low_cpu_mem_usage=False, which causes it to do some extra things to load the models. Removing that and adding accelerate package makes it load very quickly.

[parthwagh9999]

The code uses low_cpu_mem_usage=False, which causes it to do some extra things to load the models. Removing that and adding accelerate package makes it load very quickly.

Thank you for suggestion,

I tried it, removing also, and replacing with True, it is still not generating due to very very slow iteration times even in the rtx 4090.

[rockerBOO]

see my PR for some other improvements, its using shared memory im guessing. im on a 3060 and can get 5s/it. maybe you're not using your GPU? See https://pytorch.org/get-started/locally/ if you haven't setup pytorch for your GPU

[parthwagh9999]

see my PR for some other improvements, its using shared memory im guessing. im on a 3060 and can get 5s/it. maybe you're not using your GPU? See https://pytorch.org/get-started/locally/ if you haven't setup pytorch for your GPU

Could you please confirm, is pytorch is running or not?

[rockerBOO]

yeah it's using the shared GPU memory so it will be slower. could try the pipe.enable_model_cpu_offload() to offload the T5 and other parts.

[parthwagh9999]

yeah it's using the shared GPU memory so it will be slower. could try the pipe.enable_model_cpu_offload() to offload the T5 and other parts.

thank you, i tried it but not working. giving me error.

Could you please share your generate.py and other files you have edited. and command you are using.

[rockerBOO]

see the prs for my code

[parthwagh9999]

see the prs for my code

This command gave me this error message

So used this comand

All above error are for generate.py file with these three changes only,

[andrewgrz]

I am getting the same Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! message as the above user. This is on unedited main as of this message which is 7eda4408e784e82bd2f1bf3abfe38049331f57a1