Update palm detection model from MediaPipe (2023feb) #128
Conversation
|
What's the main difference between this one and the previous one? |
Thanks for reminding! I've updated the description of this PR. |
| @@ -34,7 +34,7 @@ Guidelines: | |||
| | [WeChatQRCode](./models/qrcode_wechatqrcode) | QR Code Detection and Parsing | 100x100 | 7.04 | 37.68 | --- | --- | --- | | |||
| | [DaSiamRPN](./models/object_tracking_dasiamrpn) | Object Tracking | 1280x720 | 36.15 | 705.48 | 76.82 | --- | --- | | |||
| | [YoutuReID](./models/person_reid_youtureid) | Person Re-Identification | 128x256 | 35.81 | 521.98 | 90.07 | 44.61 | --- | | |||
| | [MP-PalmDet](./models/palm_detection_mediapipe) | Palm Detection | 256x256 | 15.57 | 168.37 | 50.64 | 62.45 | --- | | |||
| | [MP-PalmDet](./models/palm_detection_mediapipe) | Palm Detection | 192x192 | 11.09 | 63.79 | 83.20 | 33.81 | --- | | |||
There was a problem hiding this comment.
After verification, this problem is caused by OpenCV version iteration. In version 4.6.0, time cost on cuda should change from 50ms to 30ms. But in version 4.7.0, time cost on cuda change from 200ms to 80ms.
This means that after the OpenCV version update, inference time of this palm detecion model is 4 times slower on cuda. Let's solved this problem firstly.
There was a problem hiding this comment.
Leave as-is and will be fixed in the next release.
There was a problem hiding this comment.
This problem is fixed by PR opencv/opencv#23243 and opencv/opencv#23255.
|
I tested the hand pose demo on Khadas VIM3. The FPS is the almost the same whatever I choose the TIMVX+NPU or Default+CPU. Could you check if everything is right? |
|
The accuracy of the INT8 palm detector is obviously lower than that of the float model. Should we release the int8 model? |
|
HandPose model still need to be updated after this PR. I think we can add more images to quantized this model so that the accuracy may be higher. I will try to improve this model. |
|
I think we can include @zihaomu for discussion on the speed. |
|
I have compared the new and old model. And the new model has lot of |
|
I re-quantified the model with 3900+ images and got a better model. The accuracy is obviously better. new model.zip
|
| @@ -1,10 +1,11 @@ | |||
| # Palm detector from MediaPipe Handpose | |||
|
|
|||
| This model detects palm bounding boxes and palm landmarks, and is converted from Tensorflow-JS to ONNX using following tools: | |||
| This model detects palm bounding boxes and palm landmarks, and is converted from TFLite to ONNX using following tools: | |||
There was a problem hiding this comment.
By the way, TFLite support in DNN has been merged. Dev branch can now load and run TFLite models.
* update to the latest palm detection model * quantize palm detection model * make palm aligned to center when resize * update benchmark for palm detection * update benchmark data * update readme * update a new quantized model * update readme
This PR aims to change the palm detection model from tfjs model to mediapipe model. Model downloaded from https://google.github.io/mediapipe/solutions/models.html#hands
The main updates between the previous mdoel and the current model are as follows.
Model Information:
Inference Time:
Model Accuracy
Current model has better performance by running demo. Also fixed the bug that the aspect ratio was not kept when resizing images.