Metal fmt=4 grouped-int4 decode: attention + routed experts (#585) - #587

Merged

JustVugg merged 8 commits into

JustVugg:devfrom

RDouglasSharp:metal-attn-fmt4

Aug 8, 2026

Commits on Aug 8, 2026

metal: adapt #587's fmt=4 kv_b + moe_block coverage onto our #457 base

monotophic
authored and
RDouglasSharp
committed
19d4004
moe: gs-consistency guard for fmt=4 GPU batches (review F2)

monotophic
authored and
RDouglasSharp
committed
95fdfda
engine: kv_b fmt-gate notice, re-derived for post-#587 two-format gate (r3)

monotophic
authored and
RDouglasSharp
committed
3be4753
probe: env-gated top-2 logit-gap dump at greedy selection (COLI_LOGIT_GAP)

monotophic
authored and
RDouglasSharp
committed
32dc3f9
metal: COLI_METAL_MOE_EXACT — opt-in CPU-exact fmt=4 experts, fast batched default (#587 gate-2)
RDouglasSharp
committed
4427dab
metal: COLI_METAL_MOE_EXACT covers all fmt=4 GPU paths (attn/layer decode + dense)
RDouglasSharp
committed
36a555a
metal: scope COLI_METAL_MOE_EXACT by path, not format

monotophic
authored and
RDouglasSharp
committed
e3bccc4
test: reconcile test_backend_metal with post-rebase dev (qgs=0 at fp8/e8 call sites, rename local fp8_nblk to avoid quant.h clash)
RDouglasSharp
committed
184b3a5