Update the Qwen3.6-35B-A3B-NVFP4 DGX Spark guide with tuned arguments#643
Update the Qwen3.6-35B-A3B-NVFP4 DGX Spark guide with tuned arguments#643prodney-dev wants to merge 1 commit into
Conversation
prodney-dev
commented
Jul 16, 2026
prodney-dev
commented
- Specify the VLLM container image with which the command was verified
- Enable chunked prefill
- Async scheduling for improving latency
- Adjust memory utilization and max num seqs
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the configuration for the Qwen3.6-35B-A3B model, adding a new vLLM version prerequisite and adjusting several serving parameters such as GPU memory utilization, maximum sequence count, chunked prefill, and async scheduling. The review feedback correctly points out that enabling both chunked prefill and prefix caching simultaneously will cause vLLM to fail at startup due to mutual exclusivity, and recommends removing the prefix caching option.
| --enable-chunked-prefill \ | ||
| --async-scheduling \ | ||
| --enable-prefix-caching \ |
There was a problem hiding this comment.
In vLLM, chunked prefill (--enable-chunked-prefill) and prefix caching (--enable-prefix-caching) are mutually exclusive. Enabling both simultaneously will cause vLLM to fail at startup with a configuration error. Since this PR specifically enables chunked prefill, --enable-prefix-caching should be removed from this command.
--enable-chunked-prefill \
--async-scheduling \There was a problem hiding this comment.
This command was validated on DGX spark with vllm/vllm-openai:v0.24.0-ubuntu2404 container.
ad33d42 to
890f581
Compare
- Specify the VLLM container image with which the command was verified - Enable chunked prefill - Async scheduling for improving latency - Adjust memory utilization and max num seqs Signed-off-by: Prerit Rodney <240578995+prodney-dev@users.noreply.github.com>
890f581 to
2c019c9
Compare
|
LGTM |
|
Looks good to me |
3 participants
