Frontier pre-release v0.2: Prefill-Decode Disaggregation (PDD)

Release notes

Frontier pre-release-v0.2 adds Prefill-Decode Disaggregation (PDD) alongside the existing co-location path. This release provides a reproducible open-source surface for studying disaggregated LLM serving, with calibrated simulation models and one-click examples. 🚀

Highlights

• PDD support: Sequential pd-disaggregation with separate PREFILL and unified DECODE clusters, connected by KV cache transfer modeling.
• 🧪 Runnable examples: Offline and online PDD recipes for Dense, MoE, Thinking Mode, Speculative Decoding / MTP, and Prefix Caching workloads.
• 📊 Metrics-first validation: PDD examples emit request-level CSV and system-level JSON metrics, including KV transfer and latency fields.
• Serving optimizations: Examples stay aligned with Chunked Prefill, CUDA Graph mode controls, Speculative Decoding / MTP, Prefix Caching, and MoE topology checks.
• Clean public surface: Legacy split-decode terminology removed; unsupported AFD research surfaces are excluded from the v0.2 examples.
• ✅ Regression coverage: Public-surface and PDD contract tests guard supported entrypoints, documentation scope, KV transfer entities, transfer predictors, and scheduler behavior.

Quick start

python -m pip install -e '.[test]'
PYTHONPATH=$PWD PYTHONDONTWRITEBYTECODE=1 pytest tests/unit/test_examples_pdd_scripts.py -q -p no:cacheprovider
bash examples/architecture/pdd/offline/dense_model_basic.sh