Skip to content

Improve horizontal fusion of contiguous #1292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 48 commits into from
Aug 17, 2022
Merged

Improve horizontal fusion of contiguous #1292

merged 48 commits into from
Aug 17, 2022

Conversation

pfultz2
Copy link
Collaborator

@pfultz2 pfultz2 commented Jul 8, 2022

No description provided.

pfultz2 added 30 commits May 17, 2022 18:56
@pfultz2
Horizontally fuse contiguous
dcd3d04
@pfultz2
Format
39bbf87
@pfultz2
Add tests
8d87d83
@pfultz2
Format
14ac1ae
@pfultz2
Fix contiguous after splits
3fb1b7f
@pfultz2
Format
dfc7bba
@pfultz2
Merge
5a1af3d
@pfultz2
Fix warning
8ea6518
@pfultz2
Add eliminate contiguous test
d3236e3
@pfultz2
Format
a7a4cd0
@pfultz2
Rename var
8e83275
@pfultz2
Fix copy-paste
4ddaf83
@pfultz2
Rename class
9840f75
@pfultz2
Use std::transform
2a8e8d0
@pfultz2
Add step to unsqeeze
3101f6f
@pfultz2
Format
c2c7f49
@pfultz2
Merge branch 'unsqueeze-step' into fuse-horiz-contiguous2
c96b88b
@pfultz2
Adjust transpose slice
fa19756
@pfultz2
Format
1e4e9c5
@pfultz2
Adjust permutation axis
9288a1f
@pfultz2
Format
1555f29
@pfultz2
Rename var
666155d
@pfultz2
Merge branch 'develop' into fuse-horiz-contiguous
fee8435
@pfultz2
Rename test
32e62f9
@pfultz2
Add tests for different steps and axes size
8c756ff
@pfultz2
Format
d6f13dd
@pfultz2
Fix concat transpose
9c6eab1
@pfultz2
Format
18e0c3f
@pfultz2
Merge branch 'develop' into fuse-horiz-contiguous
b8deb54
@pfultz2
Merge branch 'develop' into unsqueeze-step
6498d5a
@migraphx-bot
Copy link
Collaborator

Test Rate new
26bdeb		 Rate old
bb623d		 Diff Compare
torchvision-resnet50 2,220.03 2,219.72 0.01% ✅
torchvision-resnet50_fp16 4,737.44 4,740.18 -0.06% ✅
torchvision-alexnet 4,969.56 4,967.98 0.03% ✅
torchvision-alexnet_fp16 26,136.53 26,169.23 -0.12% ✅
torchvision-densenet121 1,632.35 1,484.03 9.99% ✅
torchvision-densenet121_fp16 2,529.38 2,462.22 2.73% ✅
torchvision-inceptionv3 1,096.53 1,091.19 0.49% ✅
torchvision-inceptionv3_fp16 1,978.92 1,973.06 0.30% ✅
torchvision-vgg16 894.32 894.36 -0.00% ✅
torchvision-vgg16_fp16 1,723.65 1,724.30 -0.04% ✅
cadene-inceptionv4 528.15 522.92 1.00% ✅
cadene-resnext64x4 576.88 576.09 0.14% ✅
slim-mobilenet 6,391.31 6,385.19 0.10% ✅
slim-nasnetalarge 202.92 202.00 0.46% ✅
slim-resnet50v2 2,426.04 2,425.83 0.01% ✅
bert-mrpc-onnx 652.42 635.60 2.65% ✅
bert-mrpc-tf 296.09 290.37 1.97% 🔆
pytorch-examples-wlang-gru 229.56 229.64 -0.03% ✅
pytorch-examples-wlang-lstm 305.08 304.44 0.21% ✅
torchvision-resnet50_1 511.59 509.62 0.39% ✅
torchvision-inceptionv3_1 302.26 298.59 1.23% ✅
torchvision-vgg16_1 462.44 459.61 0.61% 🔆
cadene-dpn92_1 307.51 291.75 5.40% ✅
cadene-resnext101_1 235.89 235.24 0.28% ✅
slim-vgg16_1 63.99 63.65 0.53% ✅
slim-mobilenet_1 1,981.55 1,970.11 0.58% 🔆
slim-inceptionv4_1 194.27 189.63 2.45% ✅
onnx-taau-downsample 258.63 258.21 0.16% ✅

Check results before merge 🔆

umangyadav
umangyadav requested changes Aug 10, 2022
View reviewed changes
@migraphx-bot
Copy link
Collaborator

Test Rate new
a552c6		 Rate old
bb623d		 Diff Compare
torchvision-resnet50 2,223.62 2,219.72 0.18% ✅
torchvision-resnet50_fp16 4,747.56 4,740.18 0.16% ✅
torchvision-alexnet 4,980.31 4,967.98 0.25% ✅
torchvision-alexnet_fp16 26,223.49 26,169.23 0.21% ✅
torchvision-densenet121 1,631.93 1,484.03 9.97% ✅
torchvision-densenet121_fp16 2,522.96 2,462.22 2.47% ✅
torchvision-inceptionv3 1,096.26 1,091.19 0.46% ✅
torchvision-inceptionv3_fp16 1,979.56 1,973.06 0.33% ✅
torchvision-vgg16 895.93 894.36 0.17% ✅
torchvision-vgg16_fp16 1,725.31 1,724.30 0.06% ✅
cadene-inceptionv4 528.75 522.92 1.12% ✅
cadene-resnext64x4 577.44 576.09 0.23% ✅
slim-mobilenet 6,399.68 6,385.19 0.23% ✅
slim-nasnetalarge 203.17 202.00 0.58% ✅
slim-resnet50v2 2,429.74 2,425.83 0.16% ✅
bert-mrpc-onnx 653.67 635.60 2.84% ✅
bert-mrpc-tf 297.31 290.37 2.39% 🔆
pytorch-examples-wlang-gru 229.55 229.64 -0.04% ✅
pytorch-examples-wlang-lstm 305.88 304.44 0.47% ✅
torchvision-resnet50_1 514.11 509.62 0.88% 🔆
torchvision-inceptionv3_1 304.13 298.59 1.86% 🔆
torchvision-vgg16_1 463.55 459.61 0.86% 🔆
cadene-dpn92_1 302.18 291.75 3.57% ✅
cadene-resnext101_1 236.73 235.24 0.63% ✅
slim-vgg16_1 64.01 63.65 0.57% ✅
slim-mobilenet_1 1,992.17 1,970.11 1.12% 🔆
slim-inceptionv4_1 194.66 189.63 2.65% ✅
onnx-taau-downsample 258.98 258.21 0.30% ✅

Check results before merge 🔆

@migraphx-bot
Copy link
Collaborator

Test Rate new
a552c6		 Rate old
bb623d		 Diff Compare
torchvision-resnet50 2,223.80 2,219.72 0.18% ✅
torchvision-resnet50_fp16 4,742.92 4,740.18 0.06% ✅
torchvision-alexnet 4,975.74 4,967.98 0.16% ✅
torchvision-alexnet_fp16 26,166.99 26,169.23 -0.01% ✅
torchvision-densenet121 1,628.10 1,484.03 9.71% ✅
torchvision-densenet121_fp16 2,529.18 2,462.22 2.72% ✅
torchvision-inceptionv3 1,086.31 1,091.19 -0.45% ✅
torchvision-inceptionv3_fp16 1,991.29 1,973.06 0.92% ✅
torchvision-vgg16 894.98 894.36 0.07% ✅
torchvision-vgg16_fp16 1,725.61 1,724.30 0.08% ✅
cadene-inceptionv4 527.91 522.92 0.95% ✅
cadene-resnext64x4 576.70 576.09 0.11% ✅
slim-mobilenet 6,397.25 6,385.19 0.19% ✅
slim-nasnetalarge 203.21 202.00 0.60% ✅
slim-resnet50v2 2,428.31 2,425.83 0.10% ✅
bert-mrpc-onnx 653.21 635.60 2.77% ✅
bert-mrpc-tf 296.08 290.37 1.97% 🔆
pytorch-examples-wlang-gru 228.99 229.64 -0.28% ✅
pytorch-examples-wlang-lstm 306.33 304.44 0.62% 🔆
torchvision-resnet50_1 514.09 509.62 0.88% 🔆
torchvision-inceptionv3_1 302.85 298.59 1.43% ✅
torchvision-vgg16_1 463.22 459.61 0.79% 🔆
cadene-dpn92_1 304.64 291.75 4.42% ✅
cadene-resnext101_1 236.86 235.24 0.69% ✅
slim-vgg16_1 64.01 63.65 0.56% ✅
slim-mobilenet_1 1,972.79 1,970.11 0.14% ✅
slim-inceptionv4_1 195.39 189.63 3.04% ✅
onnx-taau-downsample 258.93 258.21 0.28% ✅

Check results before merge 🔆

@migraphx-bot
Copy link
Collaborator

Test Rate new
721990		 Rate old
bb623d		 Diff Compare
torchvision-resnet50 2,222.44 2,219.72 0.12% ✅
torchvision-resnet50_fp16 4,742.36 4,740.18 0.05% ✅
torchvision-alexnet 4,973.81 4,967.98 0.12% ✅
torchvision-alexnet_fp16 26,151.59 26,169.23 -0.07% ✅
torchvision-densenet121 1,628.89 1,484.03 9.76% ✅
torchvision-densenet121_fp16 2,525.51 2,462.22 2.57% ✅
torchvision-inceptionv3 1,095.16 1,091.19 0.36% ✅
torchvision-inceptionv3_fp16 1,987.13 1,973.06 0.71% ✅
torchvision-vgg16 895.23 894.36 0.10% ✅
torchvision-vgg16_fp16 1,724.99 1,724.30 0.04% ✅
cadene-inceptionv4 527.37 522.92 0.85% ✅
cadene-resnext64x4 577.05 576.09 0.17% ✅
slim-mobilenet 6,395.97 6,385.19 0.17% ✅
slim-nasnetalarge 203.17 202.00 0.58% ✅
slim-resnet50v2 2,428.64 2,425.83 0.12% ✅
bert-mrpc-onnx 652.76 635.60 2.70% ✅
bert-mrpc-tf 295.33 290.37 1.71% ✅
pytorch-examples-wlang-gru 229.69 229.64 0.02% ✅
pytorch-examples-wlang-lstm 306.34 304.44 0.63% 🔆
torchvision-resnet50_1 513.00 509.62 0.66% 🔆
torchvision-inceptionv3_1 302.77 298.59 1.40% ✅
torchvision-vgg16_1 462.98 459.61 0.73% 🔆
cadene-dpn92_1 309.39 291.75 6.05% ✅
cadene-resnext101_1 237.25 235.24 0.86% ✅
slim-vgg16_1 64.00 63.65 0.55% ✅
slim-mobilenet_1 1,992.45 1,970.11 1.13% 🔆
slim-inceptionv4_1 195.42 189.63 3.05% ✅
onnx-taau-downsample 259.20 258.21 0.38% ✅

Check results before merge 🔆

@migraphx-bot
Copy link
Collaborator

Test Rate new
094f22		 Rate old
bb623d		 Diff Compare
torchvision-resnet50 2,223.98 2,219.72 0.19% ✅
torchvision-resnet50_fp16 4,742.22 4,740.18 0.04% ✅
torchvision-alexnet 4,974.24 4,967.98 0.13% ✅
torchvision-alexnet_fp16 26,189.04 26,169.23 0.08% ✅
torchvision-densenet121 1,633.53 1,484.03 10.07% ✅
torchvision-densenet121_fp16 2,531.10 2,462.22 2.80% ✅
torchvision-inceptionv3 1,095.96 1,091.19 0.44% ✅
torchvision-inceptionv3_fp16 2,009.34 1,973.06 1.84% ✅
torchvision-vgg16 895.18 894.36 0.09% ✅
torchvision-vgg16_fp16 1,724.62 1,724.30 0.02% ✅
cadene-inceptionv4 527.90 522.92 0.95% ✅
cadene-resnext64x4 577.04 576.09 0.16% ✅
slim-mobilenet 6,393.48 6,385.19 0.13% ✅
slim-nasnetalarge 203.18 202.00 0.58% ✅
slim-resnet50v2 2,429.34 2,425.83 0.14% ✅
bert-mrpc-onnx 637.94 635.60 0.37% ✅
bert-mrpc-tf 295.76 290.37 1.86% ✅
pytorch-examples-wlang-gru 230.04 229.64 0.17% ✅
pytorch-examples-wlang-lstm 306.05 304.44 0.53% ✅
torchvision-resnet50_1 511.69 509.62 0.41% ✅
torchvision-inceptionv3_1 303.30 298.59 1.58% 🔆
torchvision-vgg16_1 463.55 459.61 0.86% 🔆
cadene-dpn92_1 301.36 291.75 3.29% ✅
cadene-resnext101_1 237.78 235.24 1.08% ✅
slim-vgg16_1 63.99 63.65 0.54% ✅
slim-mobilenet_1 1,982.24 1,970.11 0.62% 🔆
slim-inceptionv4_1 197.24 189.63 4.01% ✅
onnx-taau-downsample 259.13 258.21 0.36% ✅

Check results before merge 🔆

umangyadav
umangyadav reviewed Aug 11, 2022
View reviewed changes
@migraphx-bot
Copy link
Collaborator

Test Rate new
a8eaf8		 Rate old
55cb7d		 Diff Compare
torchvision-resnet50 2,222.78 2,220.95 0.08% ✅
torchvision-resnet50_fp16 4,743.02 4,737.51 0.12% ✅
torchvision-alexnet 4,972.53 4,971.96 0.01% ✅
torchvision-alexnet_fp16 26,170.62 26,161.64 0.03% ✅
torchvision-densenet121 1,631.84 1,631.75 0.01% ✅
torchvision-densenet121_fp16 2,530.66 2,529.24 0.06% ✅
torchvision-inceptionv3 1,097.98 1,095.12 0.26% ✅
torchvision-inceptionv3_fp16 1,990.01 2,012.65 -1.12% ✅
torchvision-vgg16 895.34 893.77 0.18% ✅
torchvision-vgg16_fp16 1,726.21 1,723.79 0.14% ✅
cadene-inceptionv4 528.28 527.11 0.22% ✅
cadene-resnext64x4 577.88 576.65 0.21% ✅
slim-mobilenet 6,396.10 6,391.82 0.07% ✅
slim-nasnetalarge 203.28 202.92 0.18% ✅
slim-resnet50v2 2,427.78 2,426.08 0.07% ✅
bert-mrpc-onnx 653.32 638.13 2.38% 🔆
bert-mrpc-tf 295.49 295.23 0.09% ✅
pytorch-examples-wlang-gru 227.80 229.75 -0.85% 🔴
pytorch-examples-wlang-lstm 306.10 305.41 0.22% ✅
torchvision-resnet50_1 513.37 513.32 0.01% ✅
torchvision-inceptionv3_1 303.47 303.53 -0.02% ✅
torchvision-vgg16_1 463.40 462.74 0.14% ✅
cadene-dpn92_1 314.89 312.96 0.62% ✅
cadene-resnext101_1 237.84 233.54 1.84% ✅
slim-vgg16_1 64.03 63.99 0.06% ✅
slim-mobilenet_1 1,994.89 1,960.94 1.73% 🔆
slim-inceptionv4_1 193.99 195.53 -0.79% ✅
onnx-taau-downsample 259.15 258.45 0.27% ✅

This build is not recommended to merge 🔴

@migraphx-bot
Copy link
Collaborator

Test Rate new
6ce55a		 Rate old
cb5368		 Diff Compare
torchvision-resnet50 2,225.32 2,224.84 0.02% ✅
torchvision-resnet50_fp16 4,749.32 4,745.80 0.07% ✅
torchvision-alexnet 4,988.11 4,976.36 0.24% ✅
torchvision-alexnet_fp16 26,252.53 26,143.04 0.42% ✅
torchvision-densenet121 1,630.68 1,630.56 0.01% ✅
torchvision-densenet121_fp16 2,524.71 2,526.69 -0.08% ✅
torchvision-inceptionv3 1,098.27 1,096.31 0.18% ✅
torchvision-inceptionv3_fp16 1,980.31 1,984.36 -0.20% ✅
torchvision-vgg16 896.75 895.76 0.11% ✅
torchvision-vgg16_fp16 1,729.13 1,726.23 0.17% ✅
cadene-inceptionv4 529.00 528.10 0.17% ✅
cadene-resnext64x4 578.12 577.05 0.19% ✅
slim-mobilenet 6,412.85 6,396.87 0.25% ✅
slim-nasnetalarge 203.55 203.32 0.11% ✅
slim-resnet50v2 2,432.10 2,429.83 0.09% ✅
bert-mrpc-onnx 653.84 639.40 2.26% 🔆
bert-mrpc-tf 295.96 296.01 -0.02% ✅
pytorch-examples-wlang-gru 227.63 229.72 -0.91% ✅
pytorch-examples-wlang-lstm 306.02 306.02 -0.00% ✅
torchvision-resnet50_1 511.41 513.42 -0.39% ✅
torchvision-inceptionv3_1 302.83 303.19 -0.12% ✅
torchvision-vgg16_1 463.42 463.62 -0.04% ✅
cadene-dpn92_1 305.55 307.77 -0.72% ✅
cadene-resnext101_1 236.95 238.31 -0.57% ✅
slim-vgg16_1 64.02 64.01 0.01% ✅
slim-mobilenet_1 1,964.81 1,992.50 -1.39% 🔴
slim-inceptionv4_1 193.58 195.90 -1.18% ✅
onnx-taau-downsample 258.65 258.89 -0.09% ✅

This build is not recommended to merge 🔴

@causten causten merged commit 18e4a2c into develop Aug 17, 2022
@causten causten deleted the fuse-horiz-contiguous2 branch August 17, 2022 01:03
causten pushed a commit that referenced this pull request Aug 29, 2022
@pfultz2
Improve horizontal fusion of contiguous (#1292)
e3c9b63 
* Horizontally fuse contiguous
causten pushed a commit that referenced this pull request Aug 31, 2022
@pfultz2 @causten
Improve horizontal fusion of contiguous (#1292)
798c0a4 
* Horizontally fuse contiguous
@causten causten mentioned this pull request Aug 31, 2022
Merged
causten added a commit that referenced this pull request Aug 31, 2022
@causten
Final performance improvements for release (#1369)
a85b183 
* Improvements to handling and add constant passed to dot operator (#1280)
* Improve horizontal fusion of contiguous (#1292)
* Add pass to rewrite gelu as fast gelu (#1299)
* Add jit layernorm fusion (#1301)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@umangyadav umangyadav umangyadav approved these changes

@turneram turneram turneram approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@pfultz2 @causten @migraphx-bot @umangyadav @turneram