Skip to content

Add a test that compares the output of our quantized models against expected cached values #4597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Oct 13, 2021

Conversation

jdsgomes
Copy link
Contributor

@jdsgomes jdsgomes commented Oct 12, 2021

closes #4502

cc @datumbox @pmeier

Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @jdsgomes

Overall it looks great, I just left a couple of minor remarks for you. Let me know your thoughts.

Edit: Also there seems to be a conflict with latest main, could you please resolve?

Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are failing tests that seem related. It might be due to flakiness but worth checking the following:

@jdsgomes jdsgomes changed the title Add a test that compares the output of our quantized models against expected cached values [WIP] Add a test that compares the output of our quantized models against expected cached values Oct 12, 2021
@jdsgomes jdsgomes changed the title [WIP] Add a test that compares the output of our quantized models against expected cached values Add a test that compares the output of our quantized models against expected cached values Oct 12, 2021
Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @jdsgomes!

@datumbox datumbox merged commit 3855901 into pytorch:main Oct 13, 2021
@jdsgomes
Copy link
Contributor Author

Thanks for the review @datumbox !

@jdsgomes jdsgomes deleted the output-checks-on-quant-models branch October 13, 2021 08:22
facebook-github-bot pushed a commit that referenced this pull request Oct 14, 2021
…against expected cached values (#4597)

Summary:
* adding tests to check output of quantized models

* adding test quantized model weights

* merge test_new_quantized_classification_model with test_quantized_classification_model

* adding skipif removed by mistake

* addressing comments from PR

* removing unused argument

* fixing lint errors

* changing model to eval model and updating weights

* Update test/test_models.py

* enforce single test in circleci

* changing random seed

* updating weights for new seed

* adding missing empty line

* try 128 random seed

* try 256 random seed

* try 16 random seed

* disable inception_v3 input/output quantization tests

* removing ModelTester.test_inception_v3_quantized_expect.pkl

* reverting temporary ci run_test.sh changes

Reviewed By: fmassa

Differential Revision: D31649962

fbshipit-source-id: 35a0cb4d8d3564c88dabc09e750d5ad0a281431a

Co-authored-by: Vasilis Vryniotis <[email protected]>
Co-authored-by: Vasilis Vryniotis <[email protected]>
mszhanyi pushed a commit to mszhanyi/vision that referenced this pull request Oct 19, 2021
…xpected cached values (pytorch#4597)

* adding tests to check output of quantized models

* adding test quantized model weights

* merge test_new_quantized_classification_model with test_quantized_classification_model

* adding skipif removed by mistake

* addressing comments from PR

* removing unused argument

* fixing lint errors

* changing model to eval model and updating weights

* Update test/test_models.py

Co-authored-by: Vasilis Vryniotis <[email protected]>

* enforce single test in circleci

* changing random seed

* updating weights for new seed

* adding missing empty line

* try 128 random seed

* try 256 random seed

* try 16 random seed

* disable inception_v3 input/output quantization tests

* removing ModelTester.test_inception_v3_quantized_expect.pkl

* reverting temporary ci run_test.sh changes

Co-authored-by: Vasilis Vryniotis <[email protected]>
cyyever pushed a commit to cyyever/vision that referenced this pull request Nov 16, 2021
…xpected cached values (pytorch#4597)

* adding tests to check output of quantized models

* adding test quantized model weights

* merge test_new_quantized_classification_model with test_quantized_classification_model

* adding skipif removed by mistake

* addressing comments from PR

* removing unused argument

* fixing lint errors

* changing model to eval model and updating weights

* Update test/test_models.py

Co-authored-by: Vasilis Vryniotis <[email protected]>

* enforce single test in circleci

* changing random seed

* updating weights for new seed

* adding missing empty line

* try 128 random seed

* try 256 random seed

* try 16 random seed

* disable inception_v3 input/output quantization tests

* removing ModelTester.test_inception_v3_quantized_expect.pkl

* reverting temporary ci run_test.sh changes

Co-authored-by: Vasilis Vryniotis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a test that compares the output of our quantized models against expected cached values
2 participants