feat(dflash): converge DFlash spec-decode stack → dev (supersedes #140) - #143

Merged

drunkcoding merged 64 commits into

devfrom

feat/dflash-tracks-abcd

Aug 10, 2026

Commits on Jul 30, 2026

feat(dflash): add DFlash speculative decoding module for GPT-OSS
drunkcoding
committed
0010c9d
docs(dflash): add runnable GPT-OSS speculative decoding example
drunkcoding
committed
9b02fab
test(dflash): add greedy-agreement eval harness
drunkcoding
committed
28993ae

Commits on Aug 3, 2026

feat(glm): register glmmoedsa architecture (GlmMoeDsaForCausalLM, type=5, transformers>=5.12 guarded)
drunkcoding
committed
276fcf2
refactor(fp8): relocate dequant_fp8_blockwise to utils/fp8.py (back-compat re-export preserved)
drunkcoding
committed
d797138
feat(glm): hf_config parse_moe_param/parse_expert_id GLM branch (MTP-78 skip guard)
drunkcoding
committed
e6a982d
feat(glm): SyncGlmMoeDsaMoEBlock wrapper (HF-gate routing delegation)
drunkcoding
committed
b02c3bd
feat(glm): FP8 block dequant-on-load for GLM experts (honors modules_to_not_convert)
drunkcoding
committed
5fdaab7
feat(glm): DSA indexer classification + IndexShare ownership map
drunkcoding
committed
5ff3c27
feat(glm): wire GlmMoeDsa into offload engine (patch/unpatch, isinstance, first_k_dense, eager attn, MTP-safe)
drunkcoding
committed
5b2c2b6
docs(glm): add GLM-5.2 to supported models + FP8 offload usage
drunkcoding
committed
b91f1b1
test(glm): verify MLA/DSA attention tensors stay resident (not misrouted as experts)
drunkcoding
committed
c675f35
test(glm): base contract chain + RAM/GPU-gated e2e smoke (skips until fp8-in-store)
drunkcoding
committed
7e319b9
test(glm): verify IndexShare needs no offload change (indexers only on 'full' layers)
drunkcoding
committed
cc6ad2e
feat(glm): fp8-in-store Python layer (glm_fp8_in_store flag, scale sidecar, SetScales interface)
drunkcoding
committed
16ea9ca
test(glm): budget-split coherence + gated 32k long-context prefill smoke
drunkcoding
committed
034406b
test(glm): consolidated DSA/IndexShare/MLA integration test (real-config consistency)
drunkcoding
committed
eb03ba8
feat(glm): DFlash adapter hook (availability + pairing validation, reuses dflash.py)
drunkcoding
committed
9b39a10
feat(glm): native FP8 e4m3 block-scale dequant kernel + set_scales binding (bit-exact vs python; dispatcher H2D integration deferred)
drunkcoding
committed
3167500
fix(glm): model-load fixes exposed by real load (DeepseekV2Moe rename patch, GLM use_native_engine=False, e_score_correction_bias device guard)
drunkcoding
committed
48d5385
test(glm): tiny synthetic GLM reproducer (gated MOE_GLM_TINY) — reproduces native expert-dispatch SIGSEGV
drunkcoding
committed
3db718a
fix(glm): register routed experts (add experts ModuleList + first_k_dense offset) — fixes native dispatch SIGSEGV; tiny GLM now generates end-to-end
drunkcoding
committed
a4ddae4
feat(glm): MTP speculative decoding (built-in layer 78 drafter, lossless-by-construction; validated on tiny model)
drunkcoding
committed
a80b3de
feat(glm): MTP spec-decode instrumentation (accept-length tau, per-step stats, expert-fetch hook) — parity preserved
drunkcoding
committed
ffd93d9
test(glm): forked spec-decode test runner (MTP lossless + stats + DFlash adapter, process-isolated)
drunkcoding
committed
116d253
feat(glm): validate GLM serving via OpenAI api_server_v2 (works as-is on non-paged HF path; +tokenizer save +gated test)
drunkcoding
committed
adc4d7f
test(glm): serving smoke covers completions + chat + streaming (one server, process-isolated)
drunkcoding
committed
951f3f2
feat(glm): native fp8-in-store dequant-on-copy (SetScales + MoEMLP dequant, gated) — Option A==B parity on tiny FP8 GLM; completes T15/T16
drunkcoding
committed
c16cb46
feat(perf-model): minimal roofline package + GLM-5.2 ModelParams/decode roofline (scaffold, cross-plan option b)
drunkcoding
committed
54a6b05
feat(perf-model): GLM benchmark harness (tiny-model decode + MTP on/off + predicted roofline -> CSV)
drunkcoding
committed
cdc510d
feat(perf-model): GLM perf validation/summary + roofline & throughput plots (PDF+PNG) + markdown report
drunkcoding
committed
a1ba227

Commits on Aug 4, 2026

chore(glm): finalize fp8-in-store always-on tail
drunkcoding
committed
d8e5924
fix(offload): bind resident params to correct offload id (GLM q_a_layernorm mis-map)
drunkcoding
committed
437f589
refine(offload): scope resident-weight tensor_ids sort to DENSE nodes only
drunkcoding
committed
03b03b9
fix(offload): bind begin/end to stable tensor id (fixes intermittent resident-weight fetch mis-map)
drunkcoding
committed
4b8b377

Commits on Aug 5, 2026

fix(glm): keep MTP-layer routed experts FP8 on fresh offload
drunkcoding
committed
d80febc
refactor(engine): extract _generate_standard + add spec_strategy seam (spec-off identical)
drunkcoding
committed
a0b0973
feat(engine): rich on-device forward helper + 5-layer hidden-state capture for spec-decode
drunkcoding
committed
48f68f2
feat(dflash): accept-rule + block-build pure fns with hand-checked unit tests
drunkcoding
committed
b0e1ed2
feat(dflash): harden drafter loader (trust_remote_code + dim/vocab/mask asserts)
drunkcoding
committed
1ecde09
test(dflash): tiny synthetic gpt-oss target + drafter fixtures (CPU determinism)
drunkcoding
committed
7c96117
feat(dflash): native draft->verify->rollback state machine (bonus emitted-not-cached)
drunkcoding
committed
ccac7a1
fix(dflash): tiny-target sliding_window 8->128 (>= block_size+1) for reconstructible sliding-window KV rollback
drunkcoding
committed
b1309d2
feat(dflash): edge cases (accept 0/9, EOS mid-block, max_new_tokens, batch==1 guard)
drunkcoding
committed
f6d4d1b
feat(dflash): wire native speculator as GenerationEngine spec_strategy
drunkcoding
committed
0ea5015
test(dflash): tiny-model E2E losslessness parity (native == plain greedy)
drunkcoding
committed
29c544f
test(engine): spec-off byte-identity regression + gpt-oss suite green
drunkcoding
committed
9c176a5
test(dflash): GPU-gated 120B validation harness (agreement-rate/accept-len/tok-s)
drunkcoding
committed
7f62bda
docs(dflash): native-path example + usage notes (trust_remote_code, TP=2 SM120)
drunkcoding
committed
d4a4c51

Commits on Aug 6, 2026

fix(dflash): gather 5-layer feature + posterior to drafter device (multi-GPU TP>1)
drunkcoding
committed
2d10994
fix(gpt-oss): resident-load MXFP4 experts + sinks + router
drunkcoding
committed
ccb7cac

Commits on Aug 7, 2026

feat(dflash): route-ahead expert prefetch (A1-A5)
drunkcoding
committed
c4c74e7
feat(dflash): sampled (non-greedy) speculative decoding accept ops
drunkcoding
committed
c97a8ad
feat(serving): DFlash speculative decoding in continuous-batching engine
drunkcoding
committed
671edda
feat(dflash): speculator core — batched decoding, sampled/route-ahead hooks, drafter-driven contract
drunkcoding
committed
99384cd
test(dflash): GPU-gated 20b losslessness + serving-vs-sync harnesses
drunkcoding
committed
efb4385
style(dflash): apply ruff-format to new spec-decode/serving files
drunkcoding
committed
ac2689a
chore(format): pre-commit run --all-files (repo-wide ruff-format + clang-format)
drunkcoding
committed
fe7410b
chore(format): pre-commit convergence (ruff-format idempotency)
drunkcoding
committed
4579a3f

Commits on Aug 9, 2026

fix(prefetch): guard null derefs in ReleaseTensor
drunkcoding
committed
a0f75b2
feat(qwen3.5): enable offloaded text-only serving
drunkcoding
committed
86eb0b2
feat(dflash): offloaded Qwen3.5 speculative decoding + route-ahead
drunkcoding
committed
f9453ff
chore(format): ruff check --fix + ruff-format convergence (0.6.9)
drunkcoding
committed
65bda13

Commits on Aug 10, 2026

Merge branch 'dev' into feat/dflash-tracks-abcd
drunkcoding
committed
b1a553c