Skip to content

GLM-5.2 offload wrapper hard-depends on GlmMoeDsaMoE.route_tokens_to_experts (missing on some transformers 5.x) #144

Description

@drunkcoding

Summary

SyncGlmMoeDsaMoEBlock hard-depends on a HuggingFace internal method that is not present across the pinned transformers>=5.3.0,<6 range:

# moe_infinity/models/glm_moe_dsa.py:62  (in __init__)
self._hf_route_tokens = GlmMoeDsaMoE.route_tokens_to_experts
# ...
# :71  (in _route)
return self._hf_route_tokens(self, router_logits)

The wrapper borrows HF's unbound route_tokens_to_experts to guarantee routing parity, with no hasattr/version guard. On any transformers where transformers.models.glm_moe_dsa exists but GlmMoeDsaMoE has no route_tokens_to_experts, this raises:

AttributeError: type object 'GlmMoeDsaMoE' has no attribute 'route_tokens_to_experts'

at instantiation (not just under test) — so GLM-5.2 offload is broken on those versions, not merely a test failure.

Impact / where it shows

  • CI unit-tests (3.10) and unit-tests (3.12): tests/python/unit/test_glm_routing.py::test_routing_parity FAILS.
  • The default unit-tests job passes only because its transformers lacks glm_moe_dsa entirely, so the test is importorskip-skipped.
  • This is pre-existing on feat/dflash-tracks-abcd (independent of the #143 merge / model_offload.py conflict resolution).

Proposed fix (author to confirm intent)

Pick one:

  1. Pin transformers to a version range that guarantees GlmMoeDsaMoE.route_tokens_to_experts (tighten the >=5.3,<6 pin).
  2. Guarded fallback: getattr(GlmMoeDsaMoE, "route_tokens_to_experts", None), and if absent, reimplement the routing inline (version-independent) instead of borrowing the HF internal.
  3. Gate the whole GLM registration on the specific transformers version that provides the API.

Acceptance criteria

  • SyncGlmMoeDsaMoEBlock instantiates on all supported transformers versions in the pin.
  • test_glm_routing.py::test_routing_parity passes (or is correctly skipped) across the CI matrix.

Context

Surfaced while converging the DFlash stack to dev (#143). Related GLM work: #130.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions