Add support for blackwell codegen

[drisspg]

Stack from ghstack (oldest at bottom):

• -> Add support for blackwell codegen #141724

cc @ptrblck @msaroufim @eqy

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141724

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit bc4489f with merge base 42ab612 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Skylion007]

@drisspg Do we need to merge in #131493 to get the full benefits of this? I noticed some ptxas warning in the 12.6 builds which I suspect is from our old version of CUTLASS.

[drisspg]

@Skylion007 This is just being forward looking, when the next gen hardware drops it will require new gen_code and if you try to pass that in today CUDA_ARCH_LIST you get an error since we dont correctly regex match the string

[Skylion007]

True @drisspg Is there an issue to track for blackwell support?

[drisspg]

@ptrblck Are there any tracking issues for Blackwell? I imagine some other things will show up as software/hardware gets released

[drisspg]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

[drisspg]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here