test(integration): per-model examples + GPU real-checkpoint smoke tests - #145
Merged
Conversation
Add runnable examples (deepseek-v2, mixtral, qwen3, qwen3.5, glm-5.2, dbrx, jamba, olmoe, nllb) and gpu-marked, MOE_<MODEL>_SMOKE-gated real-checkpoint smoke regression tests (8 decoder MoEs parametrized + NLLB). Skip cleanly without GPU/env; excluded from CPU CI via 'gpu' marker. deepseek_v2_chat_example.py also fills a dangling README reference.
drunkcoding
force-pushed
the
feat/model-examples-regression
branch
from
August 10, 2026 16:56
54ae16a to
ab9833e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds per-model runnable examples and real-checkpoint GPU smoke regression tests for the supported models, so the offload +
generate()path has coverage per model beforedevis promoted tomain.Examples (
examples/)deepseek_v2_chat_example.py(also fixes a dangling README reference),mixtral_example.py,qwen3_example.py,qwen3_5_example.py,glm_example.py,dbrx_example.py,jamba_example.py,olmoe_example.py,nllb_example.py.Each follows the
examples/readme_example.pytemplate (MoE(ckpt, {offload_path, device_memory_ratio})→ tokenizer →generate), with per-model quirks: GLM-5.2 usesdevice_memory_ratio=0.5; NLLB uses the encoder-decoder translation path (src_lang,forced_bos_token_id);trust_remote_codeset per model. GPT-OSS already hasdflash_gpt_oss_example.py.Regression tests (
tests/python/integration/)test_model_smoke.py— parametrized over the 8 decoder MoEs (deepseek_v2, mixtral, qwen3, qwen3.5, gpt_oss, dbrx, jamba, olmoe): load real checkpoint →generate(max_new_tokens=16)→ assert output grew + non-empty decode.test_nllb_smoke.py— dedicated NLLB-MoE translation smoke.test_glm_smoke.py.Gating (won't touch CI): every test is
@pytest.mark.gpuand guarded by its ownMOE_<MODEL>_SMOKE=1env var and atorch.cuda.is_available()skip. Heavy imports (torch/transformers/moe_infinity) are inside the tests. CI runs-m "not gpu"and doesn't collecttests/python/integration, so these skip cleanly and never block CI. Run locally with e.g.MOE_MIXTRAL_SMOKE=1 pytest tests/python/integration/test_model_smoke.py -k mixtral.Verification
py_compileall 11 files: passruff check: All checks passedpytest --collect-only: 9 tests collect, skip cleanly with no GPU/envexamples/+tests/python/integration/