Skip to content

[MatMul][DO NOT MERGE] Tracking matmul #2022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 241 commits into from
Mar 20, 2023
Merged

[MatMul][DO NOT MERGE] Tracking matmul #2022

merged 241 commits into from
Mar 20, 2023

Conversation

zasdfgbnm
Copy link
Collaborator

@zasdfgbnm zasdfgbnm commented Sep 30, 2022 •
edited
Loading

Current perf on RTX 3090:

$CUDA_VISIBLE_DEVICES=1 ./build/bin/nvfuser_bench --benchmark_filter=.*Matmul.*Legacy/2048/3456/4096.*

---------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                       Time             CPU   Iterations
---------------------------------------------------------------------------------------------------------------------------------
Nvfuser_Matmul_4warp3stage/no_quant_nvfuser_4warp_TT_Legacy/2048/3456/4096/manual_time        849 us         1032 us          739
Nvfuser_Matmul_4warp3stage/no_quant_nvfuser_4warp_TN_Legacy/2048/3456/4096/manual_time        871 us         1052 us          738
Nvfuser_Matmul_4warp3stage/no_quant_nvfuser_4warp_NT_Legacy/2048/3456/4096/manual_time        867 us         1049 us          737
Nvfuser_Matmul_4warp4stage/no_quant_nvfuser_4warp_TT_Legacy/2048/3456/4096/manual_time        872 us         1053 us          726
Nvfuser_Matmul_4warp4stage/no_quant_nvfuser_4warp_TN_Legacy/2048/3456/4096/manual_time        863 us         1044 us          726
Nvfuser_Matmul_4warp4stage/no_quant_nvfuser_4warp_NT_Legacy/2048/3456/4096/manual_time        874 us         1055 us          728
Nvfuser_Matmul_8warp3stage/no_quant_nvfuser_8warp_TT_Legacy/2048/3456/4096/manual_time        839 us         1020 us          723
Nvfuser_Matmul_8warp3stage/no_quant_nvfuser_8warp_TN_Legacy/2048/3456/4096/manual_time        881 us         1063 us          723
Nvfuser_Matmul_8warp3stage/no_quant_nvfuser_8warp_NT_Legacy/2048/3456/4096/manual_time        842 us         1023 us          723
Nvfuser_Matmul_8warp4stage/no_quant_nvfuser_8warp_TT_Legacy/2048/3456/4096/manual_time        873 us         1054 us          723
Nvfuser_Matmul_8warp4stage/no_quant_nvfuser_8warp_TN_Legacy/2048/3456/4096/manual_time        870 us         1052 us          723
Nvfuser_Matmul_8warp4stage/no_quant_nvfuser_8warp_NT_Legacy/2048/3456/4096/manual_time        841 us         1022 us          722
EagerModeMatmul/no_quant_eagermode_TT_Legacy/2048/3456/4096/manual_time                       893 us          958 us          796
EagerModeMatmul/no_quant_eagermode_TN_Legacy/2048/3456/4096/manual_time                       916 us          985 us          738
EagerModeMatmul/no_quant_eagermode_NT_Legacy/2048/3456/4096/manual_time                       846 us          916 us          838

shmsong and others added 30 commits July 11, 2022 22:15
@shmsong
use custom propagator in ampere TN
6f5ba21
@shmsong
add tile ordering utilities
2329caf
@shmsong
initial matmul scheduler implementation
121af43
@shmsong
use matmul scheduler prototype on ampere and turing test cases
f958c53
@shmsong
extend to support Volta
397f74c
@shmsong
minor cleanup
00d9a57
@shmsong
comment cleanup
d7035aa
@shmsong
minor fix
9ffc61d
@shmsong
add fragment iteration and use it in matmul scheduler
ed0f525
@shmsong
use scheduler params for tests
c972116
@shmsong
fragment support in double buffer
d12a90f
@shmsong
add register double buffering test cases
c306b9b
@shmsong
clean up custom transform propagator
63f561f
@shmsong
Merge remote-tracking branch 'origin/devel' into matmul_propagator
3d47c1f
@shmsong
rebase fix
29f88c7
@shmsong
comment
d029b9f
@shmsong
move bounded selector to common area
5ac053f
@shmsong
Add logic to handle fake boundary tensors in selection.
b51d247
@shmsong
naming and comment
aba5087
@shmsong
remove unused parameters from mma node
426c381
@shmsong
remove unnecessary parameters from mma ir node
6d4f377
@shmsong
rename scheduling variables
5e1f41f
@shmsong
change accumulator tv interface
1960da9
@shmsong @zasdfgbnm
Update torch/csrc/jit/codegen/cuda/scheduler/utils.h
3a411c2 
Co-authored-by: Gao, Xiang <[email protected]>
@shmsong
PR feedback
8f2e4da
@shmsong
Merge branch 'matmul_propagator' of https://github.com/csarofeen/pytorch
eef3a97 
 into matmul_propagator
@shmsong
pipe through parallel type position
6ad2967
@shmsong
Merge remote-tracking branch 'origin/devel' into matmul_propagator
65c8f0a
@shmsong
Revert "fragment support in double buffer"
cd03b00 
This reverts commit d12a90f.
@shmsong
Merge branch 'matmul_propagator' into fragment_iter
380dd66
@zasdfgbnm
more cleanup
370f655
@zasdfgbnm
gcd cleanup
6a1501d
@zasdfgbnm
NVFuserTest.FusionAmpereMatmulSASSModifiersCheck_CUDA
d6d258d
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
b23de7f 
…tmul
@zasdfgbnm
fixes
63921da
@zasdfgbnm
fix
7aecf5b
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
b456917 
…tmul
@zasdfgbnm
fix
65afb1c
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
1f54677 
…tmul
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
9210db8 
…tmul
@zasdfgbnm
fix
edf1a28
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
5800d2f 
…tmul
@zasdfgbnm
fix
14a9ffa
@zasdfgbnm
fix
6fdb1a3
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
cbaa5a3 
…tmul
@zasdfgbnm
fix
5445ea7
@zasdfgbnm
fix
d7212aa
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
4eb7494 
…tmul
@zasdfgbnm
fix
4eb4ef8
@zasdfgbnm
Add debugging utility RAII guard for printting scopes
cc24924
@zasdfgbnm
unformat
cde3fcc
@zasdfgbnm
Merge branch 'DebugPrintScope' of github.com:csarofeen/pytorch into t…
635890d 
…racking-matmul
@zasdfgbnm
fix
3f9fc12
@zasdfgbnm
fix
a4aacc8
@zasdfgbnm
Tracking matmul cleanup (#2557)
dbd3bb6
@zasdfgbnm
Merge branch 'devel' into tracking-matmul
9b7e946
@zasdfgbnm
Merge branch 'devel' of github.com:csarofeen/pytorch into tracking-ma…
d5f9576 
…tmul
@zasdfgbnm zasdfgbnm changed the base branch from devel to rebase-tracking-matmul March 20, 2023 17:14
@zasdfgbnm zasdfgbnm merged commit 86b103b into rebase-tracking-matmul Mar 20, 2023
@zasdfgbnm zasdfgbnm deleted the tracking-matmul branch March 20, 2023 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zasdfgbnm @shmsong @naoyam @drzejan2 @mmigdal-nv