Skip to content

[TRTQA-3273][test] Waive 4 failed cases for main in QA CI - #16584

Merged
xinhe-nv merged 1 commit into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/waive-20260720-024219
Jul 20, 2026
Merged

[TRTQA-3273][test] Waive 4 failed cases for main in QA CI#16584
xinhe-nv merged 1 commit into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/waive-20260720-024219

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Bug(s): 6473161

Waive entries added

full:L40S/accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8[fp8kv=False-attn_backend=FLASHINFER-torch_compile=True] SKIP (https://nvbugs/6473161)
full:L40S/accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8_4gpus[tp4-fp8kv=True-attn_backend=FLASHINFER-torch_compile=True] SKIP (https://nvbugs/6473161)
full:L40S/accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8[fp8kv=True-attn_backend=FLASHINFER-torch_compile=True] SKIP (https://nvbugs/6473161)
full:L40S/accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_fp8_4gpus[tp4-fp8kv=False-attn_backend=FLASHINFER-torch_compile=True] SKIP (https://nvbugs/6473161)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Added skip coverage for FP8 and multi-GPU FP8 Llama 3.1 accuracy tests.
    • Documented the associated known issue reference for these test cases.

Bug(s): 6473161
Requested by: qa@nvidia.com

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@xinhe-nv
xinhe-nv enabled auto-merge (squash) July 20, 2026 02:42
@xinhe-nv

Copy link
Copy Markdown
Collaborator

/bot skip --comment "skip test on L40S only"

@xinhe-nv xinhe-nv changed the title [None][test] Waive 4 failed cases for main in QA CI [TRTQA-3273][test] Waive 4 failed cases for main in QA CI Jul 20, 2026
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds four L40S waiver entries for Llama 3.1 8B instruct FP8 accuracy tests, covering single- and four-GPU variants with both fp8kv settings.

Changes

L40S FP8 test waivers

Layer / File(s) Summary
Add FP8 skip entries
tests/integration/test_lists/waives.txt
Skips test_fp8 and test_fp8_4gpus parameterizations using FLASHINFER, TorchCompile, and both fp8kv values, referencing nvbugs/6473161.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR includes the bug and waive list, but it omits the required Description, Test Coverage, and PR Checklist sections from the template. Add the missing template sections with a short issue/solution summary, the relevant tests or note why none apply, and a completed checklist.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately describes the main change: waiving four QA CI failures on main.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/integration/test_lists/waives.txt (1)

288-291: 📐 Maintainability & Code Quality | 🔵 Trivial

Waiver coverage is sufficient for the stated scope.

The four entries cover both test variants, both fp8kv values, and the single-/four-GPU cases. No additional test-list file is needed; remove these waivers in a follow-up once NVBug 6473161 is fixed.

As per path instructions, coverage for this waiver change is sufficient, with restoration tracked outside this PR.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/waives.txt` around lines 288 - 291, Keep the
four NVBug 6473161 waivers in the existing test list unchanged, covering both
fp8 test variants, both fp8kv values, and single-/four-GPU cases. Do not add
another test-list file; restoration should occur separately after the underlying
issue is fixed.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 288-291: Add the repository-standard NVIDIA copyright header with
the current year at the beginning of the modified waiver list in waives.txt,
before the existing waiver entries; leave the entries unchanged.

---

Nitpick comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 288-291: Keep the four NVBug 6473161 waivers in the existing test
list unchanged, covering both fp8 test variants, both fp8kv values, and
single-/four-GPU cases. Do not add another test-list file; restoration should
occur separately after the underlying issue is fixed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6b185a89-52c7-4c72-bd95-ac4c8ff8d7e8

📥 Commits

Reviewing files that changed from the base of the PR and between fa54a19 and d6789b0.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment thread tests/integration/test_lists/waives.txt
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60223 [ skip ] triggered by Bot. Commit: d6789b0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60223 [ skip ] completed with state SUCCESS. Commit: d6789b0
Skipping testing for commit d6789b0

Link to invocation

@xinhe-nv
xinhe-nv merged commit 3ab7175 into NVIDIA:main Jul 20, 2026
16 checks passed
@xinhe-nv
xinhe-nv deleted the trtllm-ci-report/waive-20260720-024219 branch July 20, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants