Support for torchvision.ops.nms on pytorch mobile

[hal-314]

❓ Questions and Help

You can't deploy object detection models like yolo, FastRCNN, etc without nms op. There are several issues in the forum. https://discuss.pytorch.org/t/support-torchvision-ops-nms-on-mobile-cpu/69627 or in this repo #2050 .

So, there is any eta for this? I want to use YoloV5 to detect objects with a smartphone. I have read the code and it seems to have a fairly straight forward implementation. So, I don't know why Pytorch Mobile can't find this operator.

[fmassa]

Assigning this to @dreiss for comments

[chenbohua3]

Same issue in pytorch1.6.0 / torchvision 0.7.0. Since nms is a widely used op in detection, this may be an urgent problem:)

[fmassa]

FYI we are actively working on getting this working on mobile for the next major release.

[HasnainRaz]

@fmassa Any updates regarding this? Scripting a model that uses torchvision nms using pytorch 1.7.1 and torchvision 0.8.2 and running on iOS with Libtorch 1.7.1 still produces the same error.

Is there any update or relevant PR/ticket we can look at to keep track of the progress?

[zhiqwang]

I think that #2897 is aimed to resolve this problem.

[HasnainRaz]

#2897 appears to track the torchvision ops for android. Is there one that tracks something similar for iOS as well?

[zhiqwang]

Oops... I don't know either.

[fmassa]

@HasnainRaz we will also be looking into iOS for next release. We currently don't have an open PR for it yet though.

[fmassa]

iOS is supported as of #3582, and Android support has been added in the 0.9.0 release

[mzkaramat]

@fmassa, I've tested with compiling libtorch from source and with latest version from cocoapods, but still can't find torchvision::nms in iOS app. Am I missing some step?

[fmassa]

@mzkaramat please check #3629 for an end to end example on how to build an iOS app which leverages torchvision::nms via Faster R-CNN

[mzkaramat]

Thanks!

[elb3k]

@mzkaramat please check #3629 for an end to end example on how to build an iOS app which leverages torchvision::nms via Faster R-CNN

Thank you for a solution. But binaries from amazon is not available for x86_64 or even arm64.

[fmassa]

@elb3k we will be uploading the stable binaries in the next release.

In the mean time, you can get the nightlies from

https://ossci-ios-build.s3.amazonaws.com/libtorchvision_ops_ios_nightly_build.zip

[elb3k]

@elb3k we will be uploading the stable binaries in the next release.

In the mean time, you can get the nightlies from

https://ossci-ios-build.s3.amazonaws.com/libtorchvision_ops_ios_nightly_build.zip

Thank you. I ended up compiling torchvision with LibTorch cocoapads package. Will make pull request with cocoapads build package script.

[fmassa]

Hi,

We have some information on how to compile manually in https://github.com/pytorch/vision/blob/master/.circleci/unittest/ios/scripts/binary_ios_build.sh and https://github.com/pytorch/vision/blob/master/ios/build_ios.sh, let us know if further information is required

[elb3k]

Hi,

We have some information on how to compile manually in https://github.com/pytorch/vision/blob/master/.circleci/unittest/ios/scripts/binary_ios_build.sh and https://github.com/pytorch/vision/blob/master/ios/build_ios.sh, let us know if further information is required

Hi, these were useful to create LibTorch_vision_ops package on cocoapods. I currently published from fork, once PR is accepted, main repo can publish on cocoapods.