Skip to content

[enhancement]: Ministral 3B GGUF text encoder (ERNIE-Image) #9731

Description

@Pfannkuchensack

Is there an existing issue for this?

  • I have searched the existing issues

What should this feature add?

Load GGUF builds of ERNIE-Image's Ministral 3B text encoder. Today configs/mistral_encoder.py refuses them
("Ministral 3B GGUF encoders are not supported"): the Mistral GGUF loader builds a MistralModel with RoPE read
from GGUF metadata, while Ministral 3B needs Ministral3Model (YaRN scaling and a position-dependent attention
scale). Packed nvfp4 Ministral builds are refused as well.

Alternatives

Use the 7.7 GB safetensors encoder (the ERNIE-Image starters install it).

Additional Content

  • Demand: moritalous/Ministral-3-3B-Base-ERNIE-Image-TE-sdcpp-GGUF, ponpoke/ERNIE-Image-Abliterated-GGUF
    (text encoder files).
  • With ERNIE-Image GGUF transformers (feat(ernie-image): load GGUF transformers #9725) at ~5 GB, the encoder is now the largest part of a GGUF install.

The full matrix of what loads today is in #9726 (Model Format Support page).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions