Skip to content

Conversation

cfgfung
Copy link
Contributor

@cfgfung cfgfung commented Dec 14, 2023

Type of Change

I am adding one more CV example (first segmentation foundation model) to demonstrate the usage of INC.

Description

SAM is the first segmentation foundation model in the CV domain. It is composed of ViT and several image decoding layers. By applying suitable quantizations on specific layers, it is found that there is only a slight drop in the dice metric (~0.01). This can be an example of CV users adapting INC in their use case.

Expected Behavior & Potential Risk

Users will base on the example code to experiment INC on SAM. No risk to the existing code base.

How has this PR been tested?

Replicate the steps according to the readme in a new environment.
It is tested on Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz

Dependency Change?

Users have to install the following packages in order to run the example. 'torchmetrics' is a new package to the user.

torchvision
tdqm
torchmetrics
neural-compressor

Copy link
Contributor

@violetch24 violetch24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please refer to torchvision example for a similar format and process:

  1. After quantization, q_model needs to be saved, then loaded for accuracy and benchmark.
  2. Users should be able to load either fp32 or int8 model.
  3. Need to add benchmark part using INC's BenchmarkConfig.
  4. The example should include bash scripts to run tuning and benchmark separately.

@cfgfung
Copy link
Contributor Author

cfgfung commented Jan 10, 2024

Hi,
I have aligned the codes and tested them on Endeavour (SPR node).
Would you please help to see if this is ready for PR?

@violetch24
Copy link
Contributor

Please update the flag name in comment as well as in shell scripts and README. There are also some spell errors in pre-ci. Others look good to me, thanks!

@violetch24 violetch24 requested a review from chensuyue January 17, 2024 01:51
@chensuyue
Copy link
Contributor

There are 2 new components used by this PR, segment-anything and torchmetrics, I will add into the BoM list.
And in order to track this example with our weekly test, I will submit another PR to unify the test interface.

@chensuyue chensuyue merged commit bd5e698 into intel:master Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants