[quant] fix int16 quantization scale in conv weight #74665

terrychenism · 2022-03-24T05:53:53Z

Stack from ghstack (oldest at bottom):

-> [quant] fix int16 quantization scale in conv weight #74665

Summary:
fix int16 quantization scale in conv weight
before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0.
fixed by add qint32 support in set/get weight function for reference module

Test Plan:
python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D35106497

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

facebook-github-bot · 2022-03-24T05:53:59Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/74665
↩️ [fb-only] Re-run with SSH instructions
Need help or want to give feedback on the CI? Visit our office hours

💊 CI failures summary and remediations

As of commit b3a341c (more details on the Dr. CI page):

4/4 failures introduced in this PR

🕵️ 3 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pull / linux-xenial-cuda11.3-py3.7-gcc7 / build (1/3)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

2022-03-30T21:15:10.0444335Z �[36;1m echo "ERR...t available for the merge-base of your branch"�[0m

2022-03-30T21:15:10.0441517Z �[36;1mfi�[0m
2022-03-30T21:15:10.0441741Z �[36;1m# Covers the case where a previous tag doesn't exist for the tree�[0m
2022-03-30T21:15:10.0442083Z �[36;1m# this is only really applicable on trees that don't have `.circleci/docker` at its merge base, i.e. nightly�[0m
2022-03-30T21:15:10.0442391Z �[36;1mif ! git rev-parse "$MERGE_BASE:.circleci/docker"; then�[0m
2022-03-30T21:15:10.0442730Z �[36;1m  echo "Directory '.circleci/docker' not found in commit $MERGE_BASE, you should probably rebase onto a more recent commit"�[0m
2022-03-30T21:15:10.0443006Z �[36;1m  exit 1�[0m
2022-03-30T21:15:10.0443171Z �[36;1mfi�[0m
2022-03-30T21:15:10.0443390Z �[36;1mPREVIOUS_DOCKER_TAG=$(git rev-parse "$MERGE_BASE:.circleci/docker")�[0m
2022-03-30T21:15:10.0443713Z �[36;1m# If no image exists but the hash is the same as the previous hash then we should error out here�[0m
2022-03-30T21:15:10.0444011Z �[36;1mif [[ "${PREVIOUS_DOCKER_TAG}" = "${DOCKER_TAG}" ]]; then�[0m
2022-03-30T21:15:10.0444335Z �[36;1m  echo "ERROR: Something has gone wrong and the previous image isn't available for the merge-base of your branch"�[0m
2022-03-30T21:15:10.0444728Z �[36;1m  echo "       contact the PyTorch team to restore the original images"�[0m
2022-03-30T21:15:10.0445010Z �[36;1m  exit 1�[0m
2022-03-30T21:15:10.0445173Z �[36;1mfi�[0m
2022-03-30T21:15:10.0445357Z �[36;1mecho ::set-output name=rebuild::yes�[0m
2022-03-30T21:15:10.0455657Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2022-03-30T21:15:10.0455871Z env:
2022-03-30T21:15:10.0456014Z   IN_CI: 1
2022-03-30T21:15:10.0456172Z   IS_GHA: 1
2022-03-30T21:15:10.0456389Z   BASE_REVISION: c9b4a1edd9c241f0f9594cd5031f03c03f1d4d8c
2022-03-30T21:15:10.0456795Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda11.3-cudnn8-py3-gcc7:a2c09c6009bb8a10cbb45a8c5f7c573593289be0

pull / deploy-linux-xenial-cuda11.3-py3.7-gcc7 / build (2/3)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

2022-03-30T21:15:15.0436931Z �[36;1m echo "ERR...t available for the merge-base of your branch"�[0m

2022-03-30T21:15:15.0433716Z �[36;1mfi�[0m
2022-03-30T21:15:15.0433940Z �[36;1m# Covers the case where a previous tag doesn't exist for the tree�[0m
2022-03-30T21:15:15.0434275Z �[36;1m# this is only really applicable on trees that don't have `.circleci/docker` at its merge base, i.e. nightly�[0m
2022-03-30T21:15:15.0434760Z �[36;1mif ! git rev-parse "$MERGE_BASE:.circleci/docker"; then�[0m
2022-03-30T21:15:15.0435103Z �[36;1m  echo "Directory '.circleci/docker' not found in commit $MERGE_BASE, you should probably rebase onto a more recent commit"�[0m
2022-03-30T21:15:15.0435382Z �[36;1m  exit 1�[0m
2022-03-30T21:15:15.0435546Z �[36;1mfi�[0m
2022-03-30T21:15:15.0435901Z �[36;1mPREVIOUS_DOCKER_TAG=$(git rev-parse "$MERGE_BASE:.circleci/docker")�[0m
2022-03-30T21:15:15.0436231Z �[36;1m# If no image exists but the hash is the same as the previous hash then we should error out here�[0m
2022-03-30T21:15:15.0436604Z �[36;1mif [[ "${PREVIOUS_DOCKER_TAG}" = "${DOCKER_TAG}" ]]; then�[0m
2022-03-30T21:15:15.0436931Z �[36;1m  echo "ERROR: Something has gone wrong and the previous image isn't available for the merge-base of your branch"�[0m
2022-03-30T21:15:15.0437261Z �[36;1m  echo "       contact the PyTorch team to restore the original images"�[0m
2022-03-30T21:15:15.0437551Z �[36;1m  exit 1�[0m
2022-03-30T21:15:15.0437716Z �[36;1mfi�[0m
2022-03-30T21:15:15.0437901Z �[36;1mecho ::set-output name=rebuild::yes�[0m
2022-03-30T21:15:15.0448360Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2022-03-30T21:15:15.0448575Z env:
2022-03-30T21:15:15.0448718Z   IN_CI: 1
2022-03-30T21:15:15.0448877Z   IS_GHA: 1
2022-03-30T21:15:15.0449096Z   BASE_REVISION: c9b4a1edd9c241f0f9594cd5031f03c03f1d4d8c
2022-03-30T21:15:15.0449498Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda11.3-cudnn8-py3-gcc7:a2c09c6009bb8a10cbb45a8c5f7c573593289be0

pull / linux-xenial-cuda11.3-py3.7-gcc7-bazel-test / build-and-test (3/3)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

2022-03-30T21:15:15.6077944Z �[36;1m echo "ERR...t available for the merge-base of your branch"�[0m

2022-03-30T21:15:15.6074400Z �[36;1mfi�[0m
2022-03-30T21:15:15.6074640Z �[36;1m# Covers the case where a previous tag doesn't exist for the tree�[0m
2022-03-30T21:15:15.6075092Z �[36;1m# this is only really applicable on trees that don't have `.circleci/docker` at its merge base, i.e. nightly�[0m
2022-03-30T21:15:15.6075421Z �[36;1mif ! git rev-parse "$MERGE_BASE:.circleci/docker"; then�[0m
2022-03-30T21:15:15.6075786Z �[36;1m  echo "Directory '.circleci/docker' not found in commit $MERGE_BASE, you should probably rebase onto a more recent commit"�[0m
2022-03-30T21:15:15.6076440Z �[36;1m  exit 1�[0m
2022-03-30T21:15:15.6076710Z �[36;1mfi�[0m
2022-03-30T21:15:15.6076946Z �[36;1mPREVIOUS_DOCKER_TAG=$(git rev-parse "$MERGE_BASE:.circleci/docker")�[0m
2022-03-30T21:15:15.6077291Z �[36;1m# If no image exists but the hash is the same as the previous hash then we should error out here�[0m
2022-03-30T21:15:15.6077610Z �[36;1mif [[ "${PREVIOUS_DOCKER_TAG}" = "${DOCKER_TAG}" ]]; then�[0m
2022-03-30T21:15:15.6077944Z �[36;1m  echo "ERROR: Something has gone wrong and the previous image isn't available for the merge-base of your branch"�[0m
2022-03-30T21:15:15.6078387Z �[36;1m  echo "       contact the PyTorch team to restore the original images"�[0m
2022-03-30T21:15:15.6078635Z �[36;1m  exit 1�[0m
2022-03-30T21:15:15.6079018Z �[36;1mfi�[0m
2022-03-30T21:15:15.6079344Z �[36;1mecho ::set-output name=rebuild::yes�[0m
2022-03-30T21:15:15.6090871Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2022-03-30T21:15:15.6091206Z env:
2022-03-30T21:15:15.6091365Z   IN_CI: 1
2022-03-30T21:15:15.6091533Z   IS_GHA: 1
2022-03-30T21:15:15.6091726Z   GIT_DEFAULT_BRANCH: master
2022-03-30T21:15:15.6091967Z   BASE_REVISION: c9b4a1edd9c241f0f9594cd5031f03c03f1d4d8c

1 failure not recognized by patterns:

Job	Step	Action
^{pull / pytorch-xla-linux-bionic-py3.7-clang8 / test (xla, 1, 1, linux.2xlarge)}	^Test	🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: e45e5cf Pull Request resolved: #74665

terrychenism · 2022-03-24T05:55:57Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

vkuzo · 2022-03-24T12:45:35Z

fix int16 quantization scale in conv weight

In the PR summary, could we provide some context on what specifically was broken, and how you are fixing it?

vkuzo · 2022-03-24T12:55:11Z

test/quantization/eager/test_quantize_eager_ptq.py

+        data = torch.randn(*input_size, dtype=torch.float)
+
+        original_ref_m = RefM()
+        torch.quantization.engine = "qnnpack"


note: not all test environments will support qnnpack. You can check for this with "qnnpack" in torch.testing._internal.common_quantized.supported_qengines, and skip the test if this is false

jerryzh168 · 2022-03-24T17:03:19Z

torch/nn/quantized/_reference/modules/utils.py

@@ -98,11 +98,11 @@ def _quantize_weight(
        return weight

    if weight_qscheme == torch.per_tensor_affine:
-        if weight_dtype in [torch.quint8, torch.qint8, torch.qint32]:
+        if weight_dtype in [torch.quint8, torch.qint8, torch.qint32, torch.qint32]:


nit: torch.qint32 already included before for this one

vkuzo · 2022-03-24T17:04:46Z

torch/nn/quantized/_reference/modules/utils.py

@@ -16,7 +16,7 @@ def _init_weight_qparams(self, weight_qparams, device):
            None, torch.per_tensor_affine, torch.per_channel_affine,
            torch.per_channel_affine_float_qparams], \
            Exception(f"qscheme: {self.weight_qscheme} is not support in reference quantized {self._get_name()}")
-        if self.weight_dtype in [torch.quint8, torch.qint8, torch.quint4x2]:
+        if self.weight_dtype in [torch.quint8, torch.qint8, torch.quint4x2, torch.qint32]:


can we add a warning on L37 in the else branch to enable the user to find this easier next time?

I think we can add an assert, the else branch should not be executed, it is added just for torchscript

I added 'if-else' back due to one of fx test cases needs torch.float, so it will call 'else'

TestQuantizeFx.test_dynamic_with_fusion_multiple_uses

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 79b8048 Pull Request resolved: #74665

terrychenism · 2022-03-24T17:30:47Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

test/quantization/eager/test_quantize_eager_ptq.py

torch/nn/quantized/_reference/modules/utils.py

jerryzh168

looks good overall, had two inline comments, please address them before landing

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

terrychenism · 2022-03-25T21:00:56Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 8466547 Pull Request resolved: #74665

terrychenism · 2022-03-25T23:26:17Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

terrychenism · 2022-03-29T17:29:04Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

terrychenism · 2022-03-29T20:47:08Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 865027e Pull Request resolved: #74665

terrychenism · 2022-03-29T21:37:37Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

terrychenism · 2022-03-30T16:37:15Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

terrychenism · 2022-03-30T17:18:02Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: fix int16 quantization scale in conv weight before pr, ref_module.conv.get_quantized_weight() in qint32 dtype will get scale = 1.0. fixed by add qint32 support in set/get weight function for reference module Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D35106497](https://our.internmc.facebook.com/intern/diff/D35106497) [ghstack-poisoned]

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: c915dd4 Pull Request resolved: #74665

terrychenism · 2022-03-30T21:11:14Z

@terrychenism has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Pull Request resolved: #74665 fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Imported from OSS Reviewed By: mrshenli Differential Revision: D35106497 fbshipit-source-id: 61030786d20d845ef36ea40cdacdd7dcccf12ae9

github-actions · 2022-03-31T06:10:54Z

Hey @terrychenism.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

[quant] fix int16 quantization scale in conv weight

343ef88

Summary: fix int16 quantization scale in conv weight Test Plan: python3 test/test_quantization.py TestQuantizeEagerOps.test_int16_reference_module Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

terrychenism requested review from albanD and jbschlosser as code owners March 24, 2022 05:53

facebook-github-bot added the cla signed label Mar 24, 2022

terrychenism requested review from jerryzh168 and vkuzo and removed request for jbschlosser and albanD March 24, 2022 05:54

vkuzo reviewed Mar 24, 2022

View reviewed changes

jerryzh168 reviewed Mar 24, 2022

View reviewed changes

vkuzo reviewed Mar 24, 2022

View reviewed changes

jerryzh168 reviewed Mar 25, 2022

View reviewed changes

test/quantization/eager/test_quantize_eager_ptq.py Outdated Show resolved Hide resolved

jerryzh168 reviewed Mar 25, 2022

View reviewed changes

torch/nn/quantized/_reference/modules/utils.py Show resolved Hide resolved

jerryzh168 approved these changes Mar 25, 2022

View reviewed changes

pytorchmergebot closed this in 8c05f44 Mar 31, 2022

facebook-github-bot deleted the gh/terrychenism/24/head branch April 3, 2022 14:16

WBobby mentioned this pull request Aug 17, 2022

Add ROCm5.2.3/AMDGPU support for PyTorch WBobby/pytorch#2

Closed

[quant] fix int16 quantization scale in conv weight #74665

[quant] fix int16 quantization scale in conv weight #74665

Uh oh!

Conversation

terrychenism commented Mar 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Mar 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 3 new failures recognized by patterns

pull / linux-xenial-cuda11.3-py3.7-gcc7 / build (1/3)

pull / deploy-linux-xenial-cuda11.3-py3.7-gcc7 / build (2/3)

pull / linux-xenial-cuda11.3-py3.7-gcc7-bazel-test / build-and-test (3/3)

1 failure not recognized by patterns:

Uh oh!

terrychenism commented Mar 24, 2022

Uh oh!

vkuzo commented Mar 24, 2022

Uh oh!

vkuzo Mar 24, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Mar 24, 2022

Choose a reason for hiding this comment

Uh oh!

vkuzo Mar 24, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Mar 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

terrychenism Mar 25, 2022

Choose a reason for hiding this comment

Uh oh!

terrychenism Mar 25, 2022

Choose a reason for hiding this comment

Uh oh!

terrychenism commented Mar 24, 2022

Uh oh!

Uh oh!

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

terrychenism commented Mar 25, 2022

Uh oh!

terrychenism commented Mar 25, 2022

Uh oh!

terrychenism commented Mar 29, 2022

Uh oh!

terrychenism commented Mar 29, 2022

Uh oh!

terrychenism commented Mar 29, 2022

Uh oh!

terrychenism commented Mar 30, 2022

Uh oh!

terrychenism commented Mar 30, 2022

Uh oh!

terrychenism commented Mar 30, 2022

Uh oh!

github-actions bot commented Mar 31, 2022

Uh oh!

Uh oh!

terrychenism commented Mar 24, 2022 •

edited

Loading

facebook-github-bot commented Mar 24, 2022 •

edited

Loading

jerryzh168 Mar 24, 2022 •

edited

Loading