Skip to content

fix(glm): inline GLM-MoE-DSA routing to survive transformers without route_tokens_to_experts (#144) - #166

Merged
drunkcoding merged 1 commit into
devfrom
fix/glm-route-tokens-compat
Aug 16, 2026
Merged

fix(glm): inline GLM-MoE-DSA routing to survive transformers without route_tokens_to_experts (#144)#166
drunkcoding merged 1 commit into
devfrom
fix/glm-route-tokens-compat

Conversation

@drunkcoding

@drunkcoding drunkcoding commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Problem (#144)

SyncGlmMoeDsaMoEBlock reused HuggingFace's internal GlmMoeDsaMoE.route_tokens_to_experts (bound to the offload block's self). Investigation showed the GLM routing architecture changed across the pinned transformers>=5.3,<6 range:

  • transformers 5.12: GlmMoeDsaTopkRouter.forward returns raw router_logits; GlmMoeDsaMoE.route_tokens_to_experts(logits) does the grouped-sigmoid-top-k and returns (indices, weights).
  • transformers 5.15 (what CI resolves): route_tokens_to_experts is removed; the routing moved into GlmMoeDsaTopkRouter.forward, which now returns a 3-tuple (router_logits, topk_weights, topk_indices).

So GLM was genuinely broken on 5.15 (not just a test issue): #147's guard turned the crash into a RuntimeError and the test module-skipped, hiding it. The self.gate(...) output being a tuple on 5.15 is exactly why the block's routing failed ('tuple' object has no attribute 'sigmoid').

Fix — version-robust routing

_route is now adaptive:

  • 5.15+: self.gate(...) returns (router_logits, topk_weights, topk_indices) → unpack and use directly.
  • 5.12: self.gate(...) returns raw logits → apply an inlined _route_tokens_to_experts (a faithful copy of 5.12's grouped-sigmoid-top-k).

The fragile GlmMoeDsaMoE reference (and import) is removed, so routing no longer depends on an unstable HF internal. test_glm_routing.py: the module-level skip becomes a per-test skipif on test_routing_parity only (that test compares against the HF method, absent on 5.15), so the other GLM tests now run and exercise the block on all builds.

Verification

  • transformers 5.12 (local, real): pytest test_glm_routing.py → 5 passed. Direct numerical parity: inline routing == HF route_tokens_to_experts (torch.equal indices, allclose atol 1e-6 weights). Method-absent simulation: construct + forward OK.
  • transformers 5.15 (downloaded the wheel, read the exact source): verified the block's adaptive _route unpacks the real (router_logits, topk_weights, topk_indices) contract and forward() produces correct-shape output.
  • ruff format/check, py_compile, unit-test collection: clean.

Note: a pre-existing environment-only failure (test_glm_moe_dsa.py::test_glm_cpp_dequant_matches_python, missing compiled _store.dequant_fp8_blockwise) is unrelated (fails identically on clean dev).

Closes #144.

…-5.15 (#144)

SyncGlmMoeDsaMoEBlock borrowed transformers' internal GlmMoeDsaMoE.route_tokens_to_experts, which 5.15 removed after moving routing into GlmMoeDsaTopkRouter (now returns (router_logits, topk_weights, topk_indices)). _route is now adaptive: unpack the router tuple on 5.15+, else apply an inlined grouped-sigmoid-top-k (mirrors 5.12 route_tokens_to_experts). Removes the fragile dep+import. Verified: 5.12 numerical parity + method-absent path; 5.15 tuple-contract unpack + forward. Test: module-skip -> per-test skipif on parity only. Closes #144.
@drunkcoding
drunkcoding force-pushed the fix/glm-route-tokens-compat branch from cef76c9 to 2dd96f8 Compare August 16, 2026 14:58
@drunkcoding
drunkcoding merged commit af17617 into dev Aug 16, 2026
8 checks passed
@drunkcoding
drunkcoding deleted the fix/glm-route-tokens-compat branch August 16, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant