Skip to content

Use backend to replace macro to control enablement of MNNVL all reduce #4635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Jun 12, 2025
Merged

Use backend to replace macro to control enablement of MNNVL all reduce #4635

merged 10 commits into from
Jun 12, 2025

Conversation

HuiGao-NV
Copy link
Collaborator

No description provided.

@HuiGao-NV HuiGao-NV requested a review from a team as a code owner May 25, 2025 01:09
@HuiGao-NV HuiGao-NV requested review from dcampora and litaotju May 25, 2025 01:09
@juney-nvidia juney-nvidia requested a review from zongfeijing May 25, 2025 01:13
@juney-nvidia
Copy link
Collaborator

Thanks, Jerry. This indeed makes the code cleaner.

June

@HuiGao-NV
Copy link
Collaborator Author

/bot run

@HuiGao-NV HuiGao-NV requested a review from hlu1 May 26, 2025 01:22
@tensorrt-cicd
Copy link
Collaborator

PR_Github #6402 [ run ] triggered by Bot

@HuiGao-NV HuiGao-NV changed the title Use backend to replace macro to control enable MNVL all reduce Use backend to replace macro to control enablement of MNNVL all reduce May 26, 2025
zongfeijing
zongfeijing reviewed May 26, 2025
View reviewed changes
kaiyux
kaiyux reviewed May 26, 2025
View reviewed changes
@tensorrt-cicd
Copy link
Collaborator

PR_Github #6402 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4680 completed with status: 'FAILURE'

@HuiGao-NV
Copy link
Collaborator Author

/bot run --stage-list="DGX_H100-4_GPUs-PyTorch-Others-1"

1 similar comment
@HuiGao-NV
Copy link
Collaborator Author

/bot run --stage-list="DGX_H100-4_GPUs-PyTorch-Others-1"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #6459 [ run ] triggered by Bot

@HuiGao-NV HuiGao-NV self-assigned this May 26, 2025
@tensorrt-cicd
Copy link
Collaborator

PR_Github #6459 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4726 (Partly Tested) completed with status: 'SUCCESS'

hlu1
hlu1 reviewed May 27, 2025
View reviewed changes
hlu1
hlu1 requested changes May 27, 2025
View reviewed changes
@HuiGao-NV HuiGao-NV marked this pull request as draft May 28, 2025 00:57
@HuiGao-NV
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #7074 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #7074 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #5119 completed with status: 'FAILURE'

@HuiGao-NV
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #7105 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #7105 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #5136 completed with status: 'FAILURE'

@HuiGao-NV
Copy link
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8389 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8389 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #6078 (Partly Tested) completed with status: 'FAILURE'

@HuiGao-NV
Copy link
Collaborator Author

HuiGao-NV commented Jun 11, 2025 •
edited
Loading

Failure on fp8_block_scaling_gemm with error which is not related to this change: RuntimeError: Assertion failed: cudaFuncSetAttribute(kernel, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_size) == cudaSuccess

It's a known issue. Need to rerun.

@HuiGao-NV
Copy link
Collaborator Author

/bot run --stage-list="DGX_H100-4_GPUs-PyTorch-DeepSeek-1"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8487 [ run ] triggered by Bot

HuiGao-NV added 8 commits June 11, 2025 13:46
@HuiGao-NV
Use backend to replace macro to control enable MNVL all reduce
aaeb67b 
Signed-off-by: Hui Gao <[email protected]>
Signed-off-by: Hui Gaoâ� <[email protected]>
@HuiGao-NV
Revert some change in tests
6130848 
Signed-off-by: Hui Gao <[email protected]>
Signed-off-by: Hui Gaoâ� <[email protected]>
@HuiGao-NV
Fix MNNVL name
225c1b0 
Signed-off-by: Hui Gao <[email protected]>
Signed-off-by: Hui Gaoâ� <[email protected]>
@HuiGao-NV
Address comments
d5b372c 
Signed-off-by: Hui Gao <[email protected]>
@HuiGao-NV
Add strategy support in extra llm api config
bd1183d 
Signed-off-by: Hui Gao <[email protected]>
@HuiGao-NV
Fix docs test cases
bd1b058 
Signed-off-by: Hui Gao <[email protected]>
Signed-off-by: Hui Gaoâ� <[email protected]>
@HuiGao-NV
Address comments
d93a876 
Signed-off-by: Hui Gao <[email protected]>
@HuiGao-NV
Address comments to remove code setting strategies to Linear whem no
a7fab8b 
mappingg

Signed-off-by: Hui Gao <[email protected]>
@HuiGao-NV
Copy link
Collaborator Author

/bot run --stage-list="DGX_H100-4_GPUs-PyTorch-DeepSeek-1" --comment="rerun after rebase with that known issue is waived "

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8499 Bot args parsing error: usage: /bot [-h]
{run,kill,skip,submit,reviewers,reuse-pipeline,reuse-review} ...
/bot: error: unrecognized arguments: --comment=rerun after rebase with that known issue is waived

@HuiGao-NV
Copy link
Collaborator Author

/bot run --stage-list="DGX_H100-4_GPUs-PyTorch-DeepSeek-1"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8506 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8506 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #6166 (Partly Tested) completed with status: 'FAILURE'

@HuiGao-NV
Fix format
e4426ac 
Signed-off-by: Hui Gao <[email protected]>
@HuiGao-NV
Copy link
Collaborator Author

/bot run --stage-list="DGX_H100-4_GPUs-PyTorch-DeepSeek-1"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8555 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8555 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #6206 (Partly Tested) completed with status: 'SUCCESS'

@HuiGao-NV
Copy link
Collaborator Author

/bot skip --comment="After rerun, all multi-gpu stages passed."

@tensorrt-cicd
Copy link
Collaborator

PR_Github #8587 [ skip ] triggered by Bot

@HuiGao-NV HuiGao-NV enabled auto-merge (squash) June 12, 2025 03:16
@tensorrt-cicd
Copy link
Collaborator

PR_Github #8587 [ skip ] completed with state SUCCESS
Skipping testing for commit 5da1db9

@HuiGao-NV HuiGao-NV merged commit 4319237 into NVIDIA:main Jun 12, 2025
3 checks passed
@HuiGao-NV HuiGao-NV deleted the ar_backend branch June 13, 2025 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Superjomn Superjomn Superjomn left review comments

@zongfeijing zongfeijing zongfeijing left review comments

@kaiyux kaiyux kaiyux left review comments

@juney-nvidia juney-nvidia juney-nvidia approved these changes

@hlu1 hlu1 hlu1 approved these changes

@QiJune QiJune QiJune approved these changes

@dcampora dcampora Awaiting requested review from dcampora dcampora is a code owner automatically assigned from NVIDIA/trt-llm-torch-devs

@litaotju litaotju Awaiting requested review from litaotju litaotju is a code owner automatically assigned from NVIDIA/trt-llm-torch-devs

@DomBrown DomBrown Awaiting requested review from DomBrown DomBrown is a code owner automatically assigned from NVIDIA/trt-llm-committed-api-review-committee

@suyoggupta suyoggupta Awaiting requested review from suyoggupta suyoggupta is a code owner automatically assigned from NVIDIA/trt-llm-torch-autodeploy-devs

Assignees

@HuiGao-NV HuiGao-NV

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@HuiGao-NV @juney-nvidia @tensorrt-cicd @QiJune @Superjomn @litaotju @hlu1 @zongfeijing @kaiyux