[None][test] Remove 33 closed-bug waive entries for main by trtllm-agent · Pull Request #16576 · NVIDIA/TensorRT-LLM

trtllm-agent · 2026-07-19T12:13:30Z

Auto-generated Un-waive PR

Created by: TensorRT LLM CI (requested by qa@nvidia.com)
Target branch: main
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_UPDATE_WAIVES/114/
Closed bug(s) un-waived: 6075533, 6206179, 6276981, 6316983, 6316984, 6322073, 6344107, 6367805, 6368053, 6372711, 6384375, 6388129, 6402054, 6402500, 6412102, 6413724, 6416249, 6418021, 6418410, 6418822, 6422334, 6423845, 6423926, 6426865, 6432831

Waive entries removed

accuracy/test_disaggregated_serving.py::TestDeepSeekV3Lite::test_guided_decoding[llguidance-mtp_nextn=0] SKIP (https://nvbugs/6426865)
accuracy/test_disaggregated_serving.py::TestDeepSeekV3Lite::test_guided_decoding[llguidance-mtp_nextn=2] SKIP (https://nvbugs/6075533)
accuracy/test_disaggregated_serving.py::TestQwen3NextInstruct::test_auto_dtype[use_py_transceiver=True] SKIP (https://nvbugs/6402054)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus_chunked_prefill[latency] SKIP (https://nvbugs/6276981)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True] SKIP (https://nvbugs/6412102)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=True] SKIP (https://nvbugs/6388129)
accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_bf16_mtp SKIP (https://nvbugs/6206179)
accuracy/test_llm_api_pytorch.py::TestStep3_7::test_fp8_block_scales[tp_size=4-ep_size=4-mtp_nextn=3] SKIP (https://nvbugs/6367805)
accuracy/test_llm_api_pytorch.py::TestStep3_7::test_nvfp4[tp_size=4-ep_size=4-mtp_nextn=3] SKIP (https://nvbugs/6367805)
accuracy/test_llm_api_pytorch_multimodal.py::TestStep3_7::test_nvfp4[mtp_nextn=3] SKIP (https://nvbugs/6367805)
full:A100X/unittest/llmapi/test_llm_pytorch.py -m "part0" SKIP (https://nvbugs/6416249)
full:B200/disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp4-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6344107)
full:B200/disaggregated/test_disaggregated.py::test_disaggregated_stress_test[input8k-output1k-conc512-gpt_oss_120b_eagle_trtllm_stress] SKIP (https://nvbugs/6413724)
full:B300/accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus[baseline_pp4_mtp1] SKIP (https://nvbugs/6423845)
full:B300/accuracy/test_llm_api_pytorch_multimodal.py::TestQwen2_5_VL_7B::test_auto_dtype SKIP (https://nvbugs/6316983)
full:B300/disaggregated/test_disaggregated.py::test_disaggregated_ctxpp2_genpp2[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6322073)
full:DGX_B300/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_4gpus[ep4-mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=False] SKIP (https://nvbugs/6432831)
full:GB200/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_eagle3[eagle3_one_model=False-overlap_scheduler=False] SKIP (https://nvbugs/6402500)
full:GB200/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[llguidance-eagle3_one_model=False] SKIP (https://nvbugs/6402500)
full:GB200/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[xgrammar-eagle3_one_model=False] SKIP (https://nvbugs/6402500)
full:GB300/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_auto_dtype[True-False-False-False] SKIP (https://nvbugs/6316984)
full:H100/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[llguidance-eagle3_one_model=False] SKIP (https://nvbugs/6422334)
full:H100/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[xgrammar-eagle3_one_model=False] SKIP (https://nvbugs/6422334)
full:H100/disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp4-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6344107)
full:H100/test_e2e.py::test_multi_nodes_eval[Qwen3/Qwen3-235B-A22B-tp16-mmlu] SKIP (https://nvbugs/6423926)
full:H100/test_e2e.py::test_openai_chat_guided_decoding[openai/gpt-oss-120b] SKIP (https://nvbugs/6384375)
full:H100/test_e2e.py::test_ptp_quickstart_advanced_deepseek_multi_nodes[DeepSeek-V3] SKIP (https://nvbugs/6418410)
full:H20/disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp4-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6344107)
test_e2e.py::test_draft_token_tree_quickstart_advanced_eagle3[Llama-3.1-8b-Instruct-llama-3.1-model/Llama-3.1-8B-Instruct-EAGLE3-LLaMA3.1-Instruct-8B] SKIP (https://nvbugs/6368053)
test_e2e.py::test_draft_token_tree_quickstart_advanced_eagle3_depth_1_tree[Llama-3.1-8b-Instruct-llama-3.1-model/Llama-3.1-8B-Instruct-EAGLE3-LLaMA3.1-Instruct-8B] SKIP (https://nvbugs/6368053)
unittest/_torch/misc/test_share_tensor.py::TestShareTensor::test_share_tensor_different_dtypes SKIP (https://nvbugs/6418021)
unittest/_torch/ray_orchestrator/multi_gpu/test_llm_update_weights_multi_gpu.py -m "part0" SKIP (https://nvbugs/6372711)
unittest/_torch/visual_gen/test_wan21_i2v_pipeline.py::TestWanI2VBatchGeneration::test_batch_prompt_shape SKIP (https://nvbugs/6418822)

This PR was auto-generated by TensorRT LLM CI. Please review before merging.

Summary by CodeRabbit

Tests
- Updated the test waiver list to reflect current test coverage across serving, multimodal, end-to-end, attention, model, and visual-generation scenarios.
- Added waivers for newly identified hardware, quantization, data type, and execution configurations.
- Removed obsolete waivers for scenarios that no longer require exclusions.

Closed bug(s): 6075533, 6206179, 6276981, 6316983, 6316984, 6322073, 6344107, 6367805, 6368053, 6372711, 6384375, 6388129, 6402054, 6402500, 6412102, 6413724, 6416249, 6418021, 6418410, 6418822, 6422334, 6423845, 6423926, 6426865, 6432831 Requested by: qa@nvidia.com Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_UPDATE_WAIVES/114/ Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

xinhe-nv · 2026-07-20T02:15:25Z

/bot run

coderabbitai · 2026-07-20T02:16:59Z

📝 Walkthrough

Walkthrough

Updated tests/integration/test_lists/waives.txt by adding and removing targeted SKIP entries across model configurations, hardware-specific integration suites, end-to-end tests, unit tests, and visual generation tests.

Changes

Integration test waiver refresh

Layer / File(s)	Summary
Model and multimodal waiver updates `tests/integration/test_lists/waives.txt`	Updated DeepSeek, Qwen, Step3, and MistralLarge entries for disaggregated serving, latency, and multimodal configurations.
Hardware-specific integration waivers `tests/integration/test_lists/waives.txt`	Revised B200, B300, DGX_H200, GB300, H100, and H20 full-system and disaggregated-serving waivers.
Unit and end-to-end waiver updates `tests/integration/test_lists/waives.txt`	Replaced selected multi-node evaluation, quickstart, unit, model, MoE, LoRA, and visual-generation waivers.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#16515: Updates overlapping DeepSeek V32 disaggregated-serving auto-dtype waivers.
NVIDIA/TensorRT-LLM#16573: Updates overlapping DGX_H200 DeepSeekV32 FP8 blockscale waivers.
NVIDIA/TensorRT-LLM#16528: Modifies overlapping DeepSeek V3 Lite BF16 waiver entries.

Suggested reviewers: larryxfly

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: removing closed-bug waive entries.
Description check	✅ Passed	The description covers the automated waive removal, target branch, Jenkins build, and the closed bugs un-waived.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-07-20T02:23:37Z

PR_Github #60217 [ run ] triggered by Bot. Commit: 9c41eb9 Link to invocation

tensorrt-cicd · 2026-07-20T02:59:40Z

PR_Github #60217 [ run ] completed with state FAILURE. Commit: 9c41eb9
/LLM/main/L0_MergeRequest_PR pipeline #48588 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

xinhe-nv · 2026-07-20T03:16:40Z

/bot run

tensorrt-cicd · 2026-07-20T03:23:27Z

PR_Github #60243 [ run ] triggered by Bot. Commit: 9f164b8 Link to invocation

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

tensorrt-cicd · 2026-07-20T05:45:39Z

PR_Github #60243 [ run ] completed with state SUCCESS. Commit: 9f164b8
/LLM/main/L0_MergeRequest_PR pipeline #48609 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv · 2026-07-20T05:50:01Z

/bot skip --comment "unwaive closed test"

tensorrt-cicd · 2026-07-20T05:55:52Z

PR_Github #60280 [ skip ] triggered by Bot. Commit: 8632a9a Link to invocation

tensorrt-cicd · 2026-07-20T06:06:36Z

PR_Github #60280 [ skip ] completed with state SUCCESS. Commit: 8632a9a
Skipping testing for commit 8632a9a

Link to invocation

trtllm-agent requested review from LarryXFly, StanleySun639, crazydemo, jieli-matrix and xinhe-nv July 19, 2026 12:13

github-actions Bot assigned trtllm-agent Jul 19, 2026

xinhe-nv marked this pull request as ready for review July 20, 2026 02:15

Merge branch 'main' into trtllm-ci-report/unwaive-20260719-114808

9c41eb9

xinhe-nv requested review from a team as code owners July 20, 2026 02:15

xinhe-nv requested review from BowenFu, QiJune, tburt-nv and yuanjingx87 July 20, 2026 02:15

xinhe-nv enabled auto-merge (squash) July 20, 2026 02:15

coderabbitai Bot mentioned this pull request Jul 20, 2026

[TRTQA-3274][test] Waive 3 failed cases for main in QA CI #16585

Merged

coderabbitai Bot mentioned this pull request Jul 20, 2026

[None][infra] Waive 5 failed cases for main in post-merge 2844 #16587

Merged

Merge branch 'main' into trtllm-ci-report/unwaive-20260719-114808

9f164b8

Merge branch 'main' into trtllm-ci-report/unwaive-20260719-114808

b283998

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

coderabbitai Bot mentioned this pull request Jul 20, 2026

[https://nvbugs/6305365][chore] Unwaive piecewise cudagraph related tests #16591

Merged

1 task

Merge branch 'main' into trtllm-ci-report/unwaive-20260719-114808

4e91280

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv approved these changes Jul 20, 2026

View reviewed changes

Merge branch 'main' into trtllm-ci-report/unwaive-20260719-114808

8632a9a

xinhe-nv merged commit 447a628 into NVIDIA:main Jul 20, 2026
7 checks passed

xinhe-nv deleted the trtllm-ci-report/unwaive-20260719-114808 branch July 20, 2026 06:11

This was referenced Jul 20, 2026

[TRTQA-3282][test] Waive 3 failed cases for main in QA CI #16600

Merged

[TRTLLM-14473][chore] Remove legacy TensorRT backend tests examples and CI plumbing #16610

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[None][test] Remove 33 closed-bug waive entries for main#16576

[None][test] Remove 33 closed-bug waive entries for main#16576
xinhe-nv merged 6 commits into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/unwaive-20260719-114808

trtllm-agent commented Jul 19, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

xinhe-nv commented Jul 20, 2026

Uh oh!

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

xinhe-nv commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

xinhe-nv commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

trtllm-agent commented Jul 19, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Auto-generated Un-waive PR

Waive entries removed

Summary by CodeRabbit

Uh oh!

xinhe-nv commented Jul 20, 2026

Uh oh!

coderabbitai Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

xinhe-nv commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

xinhe-nv commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

trtllm-agent commented Jul 19, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading