Skip to content

WIP: PD-DFlash serving scheduler — Phase 1 (Task 1: freeze experiment contract) - #156

Draft
drunkcoding wants to merge 4 commits into
devfrom
feat/pd-dflash-serving-scheduler
Draft

WIP: PD-DFlash serving scheduler — Phase 1 (Task 1: freeze experiment contract)#156
drunkcoding wants to merge 4 commits into
devfrom
feat/pd-dflash-serving-scheduler

Conversation

@drunkcoding

Copy link
Copy Markdown
Contributor

WIP — Phase 1 (Task 1 only): freeze the PD-DFlash serving experiment contract

Status: DRAFT / not for merge. This is the first review checkpoint of the
PD-DFlash serving-scheduler plan. Only Task 1 ("Freeze the experiment schema
and cost-model decision") is implemented; everything after it is deliberately
deferred (see below).

Context

What this PR adds (Task 1 — pure Python, CPU-only)

  • benchmarks/dflash/__init__.py — package marker.
  • benchmarks/dflash/report.py — the immutable result/cost-model contract,
    imported later by both the GPU runner and the aggregator:
    • REQUIRED_METRICS — the frozen §8 metric tuple (10 metrics incl.
      route_ahead_prefetch_coverage and wasted_prefetch_bytes as bytes).
    • HideInequality (frozen) + evaluate_hide_inequality(...) — the §7
      route-ahead hiding inequality (1 - r)·s·M / BW ≤ t_draft + t_router + overlap, evaluated from measured terms only (rejects r∉[0,1],
      non-positive measured bandwidth, negative time/byte terms).
    • validate_result_matrix(...) — requires baselines B0–B3 and every metric
      per row; permits B3's explicit UNAVAILABLE_CAPACITY status.
  • tests/python/dflash/test_pd_dflash_report.py — 10 CPU-only contract tests
    (TDD: written red-first, then implemented).

Zero edits to existing code; no hot offload / gpt-oss / GLM / spec-decode paths
touched.

Verification (evidence)

  • pytest -q tests/python/dflash/test_pd_dflash_report.py10 passed.
  • ruff check (isort) → All checks passed; ruff format --checkclean.
  • LSP diagnostics on all three changed files → no diagnostics.

Deferred to later checkpoints (NOT in this PR)

  • Task 2–3 (rest of Phase A): the opt-in B0–B3 GPU runner
    (pd_dflash_serving.py), byte-accurate route-ahead observations
    (_route_ahead_stats.py + model_offload.py/expert_executor.py/
    expert_prefetcher.py plumbing), and the actual §8 + BM1 matrix runs. These
    modify hot offload paths and require an RTX PRO 6000 (sm_120) + FP4-offloaded
    checkpoints
    to run, so they are the hardware/human review-checkpoint step.
  • Phase B (Tasks 4–6): DRAFT/VERIFY lifecycle states + the 2-D
    {tokens, expert-bytes} deficit scheduler.
  • Phase C (Tasks 7–10): benchmark-gated C++ hops (batched issuance, priority
    band, overlap/e2e). Per the plan, no C++ ships without its paired benchmark
    passing.

Notes

  • One documented deviation: the plan's literal hide_window_seconds == 0.15
    assertion is a float landmine (0.04+0.01+0.10 == 0.15 is False), so that
    single equality is asserted via pytest.approx (in-code comment explains it).
  • The task referenced tests/README.md; it does not exist in the repo — used
    pyproject.toml + tests/conftest.py for harness conventions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant