Skip to content

Pulling both model weights and MTP from s3:// leads to a crash #537

Description

@gshtrasberg

Description

Observed on vLLM 0.25 + ModelExpress 0.4.1.

For models whose MTP (speculative draft) weights live in the same s3:// checkpoint
as the main weights, loading the MTP weights fails. ModelStreamerStrategy is gated
on MX_MODEL_URI (is_available) but resolves the actual streaming source as
model_config.model_weights or model_config.model — and for the MTP draft neither
points at the checkpoint.

The target model loads fine (vLLM's runai pull sets model_weights=s3://…). But for
method="mtp", vLLM sets SpeculativeConfig.model = target_model_config.model (the
local snapshot dir) and builds draft_model_config = ModelConfig(model=<snapshot>, model_weights=None). The MTP weights are in the checkpoint MX_MODEL_URI
points at, but the strategy never uses it.

Reproduction

Run vLLM with s3://path/to/GLM-5.2 as the model and MX_MODEL_URI set, plus
--speculative-config={"method":"mtp","num_speculative_tokens":5,"moe_backend":"triton"}.

The runai streamer pulls the main weights successfully, then fails on the MTP draft:
Cannot find any safetensors model weights with /root/.cache/vllm/assets/model_streamer/

Proposed fix

When the primary source (model_weights or model) yields no streamable weights,
fall back to streaming from MX_MODEL_URI (the gate env the strategy already reads),
into the existing draft module (without re-initializing it).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions