Skip to content

fix: five small guards (NextN post-load, MLA draft warmup hook, zero-expert Marlin, deep_gemm) - #71

Open
architehc wants to merge 1 commit into
kvcache-ai:mainfrom
architehc:sm120-misc-guards
Open

fix: five small guards (NextN post-load, MLA draft warmup hook, zero-expert Marlin, deep_gemm)#71
architehc wants to merge 1 commit into
kvcache-ai:mainfrom
architehc:sm120-misc-guards

Conversation

@architehc

Copy link
Copy Markdown

Five independent small fixes found while serving GLM-5.2 with kt-kernel CPU experts:

  1. NextN draft model: `run_post_load` passthrough (post-load hooks never ran for the draft weights).
  2. Deepseek weight loader: fused `weight_shape` metadata merge (loading MTP checkpoints with fused shards).
  3. `FlashInferMLAMultiStepDraftBackend`: `on_after_cuda_graph_warmup_pass` hook so draft backends can finalize after warmup.
  4. Marlin MoE repack: skip when a layer has zero GPU experts (crash when all experts are CPU-offloaded).
  5. `configure_deep_gemm_num_sms`: no-op when deep_gemm was never imported (NameError on the dual-stream decode path with JIT deepgemm disabled).

🤖 Generated with Claude Code

- NextN run_post_load passthrough (post-load hooks never ran for draft)
- fused weight_shape metadata merge in the deepseek weight loader
- FlashInferMLAMultiStepDraftBackend on_after_cuda_graph_warmup_pass hook
- zero-GPU-expert Marlin repack skip (crash when all experts offloaded)
- configure_deep_gemm_num_sms no-op when deep_gemm was never imported
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant