[TRTLLM-7250][fix] Add failed cases into waives.txt

[xinhe-nv]

waive failed cases.

Summary by CodeRabbit

• Tests
  • Updated the integration test suite to skip a specific disaggregated serving scenario, preventing a known unstable case from running.
  • This reduces false negatives in automated runs, improving CI reliability and providing faster, clearer feedback to developers.
  • No changes to product behavior or public APIs; end-user experience remains unchanged.

[xinhe-nv]

/bot run --skip-test

[coderabbitai]

📝 Walkthrough

Walkthrough

Added a new SKIP entry to tests/integration/test_lists/waives.txt to skip a specific disaggregated serving accuracy test case for Llama3.1 8B Instruct (context parallelism 4, tensor parallelism 2), referencing nvbugs/5434320.

Changes

Cohort / File(s)	Summary of Changes
Test skip list update tests/integration/test_lists/waives.txt	Added a SKIP line for accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_ctx_pp_gen_tp_asymmetric[MMLU-gen_tp=2-ctx_pp=4] with bug reference https://nvbugs/5434320.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

• [None][chore] Add failed cases into waives.txt #7109 — Also updates tests/integration/test_lists/waives.txt to add SKIP entries for disaggregated-serving accuracy tests.
• [None][chore] Add failed cases into waives.txt #7251 — Adds multiple SKIP entries in tests/integration/test_lists/waives.txt; similar scope to this change.
• [None][chore] Add failed cases into waives.txt #7735 — Modifies the same waives.txt to add SKIP entries for other failing tests.

Suggested reviewers

• crazydemo
• LarryXFly

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The PR description "waive failed cases." is overly terse and does not follow the repository's required template; it omits the detailed Description, Test Coverage, list of affected tests/files, rationale for the waiver, and the PR checklist entries that reviewers need to evaluate the change. Because these mandatory sections are missing, reviewers cannot verify which tests were waived, why they were waived, or how the change was validated.	Please update the PR description to follow the repository template by adding a clear Description explaining why the tests are being waived, enumerating the exact tests or file entries added to tests/integration/test_lists/waives.txt, linking the TRTLLM-7250 ticket or NVBugs reference, and including Test Coverage and checklist items that show how the waiver was validated; if the waiver is temporary, state the expected resolution steps and timeframe.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title "[TRTLLM-7250][fix] Add failed cases into waives.txt" directly describes the primary change — adding failed test cases to the waives.txt skip list — and includes a valid ticket identifier plus the [fix] type, making it concise and relevant for repository history.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

• Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
• Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

tests/integration/test_lists/waives.txt (1)

353-353: Optional: co-locate with existing disaggregated Llama3.1 8B waivers for easier maintenance.

Consider moving this line near Lines 318–321, which already waive other TestLlama3_1_8BInstruct cases, to keep related entries grouped.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a5cfc83 and f41e4f2.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt (1 hunks)

🧰 Additional context used

🧠 Learnings (4)

📓 Common learnings

Learnt from: tongyuantongyu
PR: NVIDIA/TensorRT-LLM#7781
File: tests/integration/test_lists/waives.txt:313-313
Timestamp: 2025-09-17T02:48:52.732Z
Learning: In TensorRT-LLM, `tests/integration/test_lists/waives.txt` is specifically for waiving/skipping tests, while other test list files like those in `test-db/` and `qa/` directories are for different test execution contexts (pre-merge, post-merge, QA tests). The same test appearing in both waives.txt and execution list files is intentional - the test is part of test suites but will be skipped due to the waiver.

📚 Learning: 2025-09-17T02:48:52.732Z

Learnt from: tongyuantongyu
PR: NVIDIA/TensorRT-LLM#7781
File: tests/integration/test_lists/waives.txt:313-313
Timestamp: 2025-09-17T02:48:52.732Z
Learning: In TensorRT-LLM, `tests/integration/test_lists/waives.txt` is specifically for waiving/skipping tests, while other test list files like those in `test-db/` and `qa/` directories are for different test execution contexts (pre-merge, post-merge, QA tests). The same test appearing in both waives.txt and execution list files is intentional - the test is part of test suites but will be skipped due to the waiver.

Applied to files:

• tests/integration/test_lists/waives.txt

📚 Learning: 2025-08-29T14:07:45.863Z

Learnt from: EmmaQiaoCh
PR: NVIDIA/TensorRT-LLM#7370
File: tests/unittest/trt/model_api/test_model_quantization.py:24-27
Timestamp: 2025-08-29T14:07:45.863Z
Learning: In TensorRT-LLM's CI infrastructure, pytest skip markers (pytest.mark.skip) are properly honored even when test files have __main__ blocks that call test functions directly. The testing system correctly skips tests without requiring modifications to the __main__ block execution pattern.

Applied to files:

• tests/integration/test_lists/waives.txt

📚 Learning: 2025-07-28T17:06:08.621Z

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Applied to files:

• tests/integration/test_lists/waives.txt

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Pre-commit Check

🔇 Additional comments (1)

tests/integration/test_lists/waives.txt (1)

353-353: Verify pytest node-id matches waiver entry

Waiver at tests/integration/test_lists/waives.txt:353 exists, but accuracy/test_disaggregated_serving.py was not present in the checkout; confirm pytest emits the exact node-id "MMLU-gen_tp=2-ctx_pp=4" for test_ctx_pp_gen_tp_asymmetric. Run:

rg -n 'def test_ctx_pp_gen_tp_asymmetric' -S
rg -n 'class TestLlama3_1_8BInstruct' -S
pytest --collect-only -q | rg 'test_ctx_pp_gen_tp_asymmetric'

[tensorrt-cicd]

PR_Github #19077 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #19077 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14311 (Partly Tested) completed with status: 'FAILURE'

[xinhe-nv]

/bot run --skip-test

[tensorrt-cicd]

PR_Github #19104 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #19104 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14332 (Partly Tested) completed with status: 'SUCCESS'

[xinhe-nv]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #19142 [ reuse-pipeline ] triggered by Bot

[tensorrt-cicd]

PR_Github #19142 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #19104 (Partly Tested) for commit 130d7b1

[xinhe-nv]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #19150 [ reuse-pipeline ] triggered by Bot

[tensorrt-cicd]

PR_Github #19150 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #19104 (Partly Tested) for commit 0be42c1

[xinhe-nv]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #19157 [ reuse-pipeline ] triggered by Bot

[tensorrt-cicd]

PR_Github #19157 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #19104 (Partly Tested) for commit 17580d8