WIP: PD-DFlash serving scheduler — Phase 1 (Task 1: freeze experiment contract) - #156
Draft
drunkcoding wants to merge 4 commits into
Draft
WIP: PD-DFlash serving scheduler — Phase 1 (Task 1: freeze experiment contract)#156drunkcoding wants to merge 4 commits into
drunkcoding wants to merge 4 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP — Phase 1 (Task 1 only): freeze the PD-DFlash serving experiment contract
Status: DRAFT / not for merge. This is the first review checkpoint of the
PD-DFlash serving-scheduler plan. Only Task 1 ("Freeze the experiment schema
and cost-model decision") is implemented; everything after it is deliberately
deferred (see below).
Context
docs/superpowers/plans/2026-08-14-pd-dflash-serving-scheduler.md(currently an untracked working-doc captured in a stash; not committed to a
branch).
docs/design/pd-dflash-moe-serving.md— PR docs(design): PD-dflash-MoE serving design doc #132(
docs/pd-dflash-moe-design).origin/dev@0e4904f(gpt-oss expert offload landed ondev,which unblocked this follow-up). The plan text references
feat/dflash-spec-decodeas its base, but that route-ahead prototype (PR feat(dflash): converge spec-decode stack → dev (gpt-oss DFlash + route-ahead prototype + GLM-5.2 + perf-model) #140)is already merged into
dev— verified the trees are identical forspec_decode/,serving/, andtests/python/dflash/. Sodevis the correctbase and no other feature branch is touched.
What this PR adds (Task 1 — pure Python, CPU-only)
benchmarks/dflash/__init__.py— package marker.benchmarks/dflash/report.py— the immutable result/cost-model contract,imported later by both the GPU runner and the aggregator:
REQUIRED_METRICS— the frozen §8 metric tuple (10 metrics incl.route_ahead_prefetch_coverageandwasted_prefetch_bytesas bytes).HideInequality(frozen) +evaluate_hide_inequality(...)— the §7route-ahead hiding inequality
(1 - r)·s·M / BW ≤ t_draft + t_router + overlap, evaluated from measured terms only (rejects r∉[0,1],non-positive measured bandwidth, negative time/byte terms).
validate_result_matrix(...)— requires baselines B0–B3 and every metricper row; permits B3's explicit
UNAVAILABLE_CAPACITYstatus.tests/python/dflash/test_pd_dflash_report.py— 10 CPU-only contract tests(TDD: written red-first, then implemented).
Zero edits to existing code; no hot offload / gpt-oss / GLM / spec-decode paths
touched.
Verification (evidence)
pytest -q tests/python/dflash/test_pd_dflash_report.py→ 10 passed.ruff check(isort) → All checks passed;ruff format --check→ clean.Deferred to later checkpoints (NOT in this PR)
(
pd_dflash_serving.py), byte-accurate route-ahead observations(
_route_ahead_stats.py+model_offload.py/expert_executor.py/expert_prefetcher.pyplumbing), and the actual §8 + BM1 matrix runs. Thesemodify hot offload paths and require an RTX PRO 6000 (sm_120) + FP4-offloaded
checkpoints to run, so they are the hardware/human review-checkpoint step.
{tokens, expert-bytes}deficit scheduler.band, overlap/e2e). Per the plan, no C++ ships without its paired benchmark
passing.
Notes
hide_window_seconds == 0.15assertion is a float landmine (
0.04+0.01+0.10 == 0.15isFalse), so thatsingle equality is asserted via
pytest.approx(in-code comment explains it).tests/README.md; it does not exist in the repo — usedpyproject.toml+tests/conftest.pyfor harness conventions.