chore: remove VLLM_USE_FLASHINFER_MOE_FP4 flag#614
Conversation
…sue vllm-project#366) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates several model configuration YAML files to bump the minimum vLLM version to 0.24.0 and removes manual FlashInfer MoE environment variables, as vLLM now auto-selects these kernels on Blackwell and Hopper architectures. However, the review feedback highlights numerous inconsistencies across the modified files: while the minimum version was updated, corresponding deployment guides, Docker images, dependencies, and ROCm wheel references still point to older vLLM versions. These contradictions should be resolved to ensure accurate documentation and prevent potential installation or runtime issues.
| model: | ||
| model_id: "zai-org/GLM-5" | ||
| min_vllm_version: "0.16.0" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the dependencies section (line 29) and the guide (lines 105 and 131) still explicitly pin and recommend vllm==0.19.0. This is a direct contradiction and will cause installation conflicts or runtime issues. Please update the pinned dependency and guide instructions to match the new minimum version of 0.24.0.
| model: | ||
| model_id: "MiniMaxAI/MiniMax-M2.5" | ||
| min_vllm_version: "0.20.2" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the docker_image on line 22 is still set to "vllm/vllm-openai:v0.20.2". This creates a contradiction where the specified Docker image is older than the minimum required vLLM version. Please update the docker_image to a version that is at least v0.24.0.
| model: | ||
| model_id: "zai-org/GLM-5.1" | ||
| min_vllm_version: "0.19.1" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (lines 96 and 124) still recommends and installs vllm==0.19.0. This contradiction should be resolved by updating the guide instructions to use at least v0.24.0.
| model: | ||
| model_id: "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16" | ||
| min_vllm_version: "0.11.2" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide still states vLLM >= 0.11.2 (0.12.0 recommended) (line 86) and pulls the v0.12.0 Docker image (line 92). This contradiction should be resolved by updating the guide and Docker pull commands to reference at least v0.24.0.
| model: | ||
| model_id: "moonshotai/Kimi-K2.5" | ||
| min_vllm_version: "0.19.1" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (line 150) still states vLLM version: >= 0.15.0. Please update the guide to reflect the new minimum version of 0.24.0.
| model: | ||
| model_id: "arcee-ai/Trinity-Large-Thinking" | ||
| min_vllm_version: "0.11.1" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (line 78) still states vLLM >= 0.11.1. Please update the guide to reflect the new minimum version of 0.24.0.
| model: | ||
| model_id: "Qwen/Qwen3-VL-235B-A22B-Instruct" | ||
| min_vllm_version: "0.11.0" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (line 109) still states # Install vLLM >= 0.11.0. Please update the guide to reflect the new minimum version of 0.24.0.
| model: | ||
| model_id: "Qwen/Qwen3-Next-80B-A3B-Instruct" | ||
| min_vllm_version: "0.10.0" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (line 148) still references rocm/0.14.1/rocm700. Please update the guide to reference a ROCm wheel version that is at least 0.24.0.
| model: | ||
| model_id: "Qwen/Qwen3-235B-A22B-Instruct-2507" | ||
| min_vllm_version: "0.10.0" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (line 94) still references rocm/0.14.1/rocm700. Please update the guide to reference a ROCm wheel version that is at least 0.24.0.
| model: | ||
| model_id: "deepseek-ai/DeepSeek-V3.2" | ||
| min_vllm_version: "0.18.0" | ||
| min_vllm_version: "0.24.0" |
There was a problem hiding this comment.
The min_vllm_version has been updated to "0.24.0", but the guide (line 100) still states Version 0.18.0 or later. Please update the guide to reflect the new minimum version of 0.24.0.
|
should docker version also be bumped? |
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
…review) Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
e936a3a to
ffd7d62
Compare
|
I think we don't need to update version? the model minimal version is the first version vLLM support. I think just replace env with |
3 participants
After v0.23.1rc0, the flag VLLM_USE_FLASHINFER_MOE_FP4 seems to be auto selected. See
vllm/model_executor/layers/fused_moe/oracle/nvfp4.py,select_nvfp4_moe_backend().Users can now use
--moe-backendto specify.Part of the clean up effort: #366