Colab model demo notebooks output is not human-readable, just a numeric tensor

[Tylersuard]

🐛 Bug

When I run the PyTorch computer vision demo notebooks, the model outputs a tensor rather than a human-readable result. These demos would be a lot more effective at showing the capability of each model if the code would output labels or a labeled image. I can fix it, I would just need a guarantee that someone with access would review my PR promptly so my changes can get accepted.

To Reproduce

Steps to reproduce the behavior:

1. Run any colab notebook from the Hub, including ResNest, etc.
   https://pytorch.org/hub/facebookresearch_WSL-Images_resnext/
   https://pytorch.org/hub/pytorch_vision_resnext/
   https://pytorch.org/hub/pytorch_vision_resnest/
   https://pytorch.org/hub/pytorch_vision_googlenet/
   https://pytorch.org/hub/pytorch_vision_hardnet/
   https://pytorch.org/hub/pytorch_vision_alexnet/

Expected behavior

A simple text label for the image, or the image itself with a label drawn onto it.

cc @vfdev-5

[datumbox]

Thanks for your feedback.

I assume you refer to the following snippet of the notebook:

The intention of this is to keep things focused on how one uses the pre-trained model to make predictions with the minimum lines of code. Though showing the image with a label would be many times more aesthetically pleasing than the vector of the probabilities, it might complicate the example with lots of extra code for the image generation and dilute the message.

I do like your proposal to print the text label for the image. Showing the top 5 classes with their probabilities is more useful and something that people are likely to use. One blocker for your PR is #484. Currently TorchVision does not store the mapping between the class ids and the labels in the library. It might not completely block you from doing what you propose but it surely makes it more complex and less "guaranteed" to be merged.

If you have any good ideas on how to programmatically fetch the labels and process them in the notebook, so that you keep things neat and short, please send a PR for one of the notebooks to discuss it. Or you could also outline your solution here and proceed to the PR later to increase its chances of being merged. Up to you! :)

[Tylersuard]

Thank you for the quick and detailed response.
My concern is that I use the notebooks to evaluate different models, to decide which ones I want to include in my project. If I can't get the classes and their probabilities, I can't tell whether the model was successful in predicting my image.

Idea for a solution:

1. Specifically for ImageNet in the issue What is the relationship between the output label of pretrained model in model zoo and wordnet synset id?  #484 link you sent, it looks like there are two documents: one with the indices in order but with no class description, and one with the indices out of order but with class descriptions.
   I would match the class names to the indices and create a new text document with the class descriptions in order, and include it in the repo for the notebooks.
   I would download that file into the notebook using !wget, then just do a short loop statement matching the tensor probability to the label.

[datumbox]

@Tylersuard thanks for drafting this. Looks like a reasonable approach and I think this will be useful to the users. :)

Could you please send a PR for one of the notebooks so that we can discuss any details and then consider porting this to the rest of the notebooks?

[Tylersuard]

Absolutely! I just created this PR: pytorch/hub#175 Thank you for the opportunity to contribute. Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

…

On Thursday, December 17, 2020 6:38 AM, Vasilis Vryniotis ***@***.***> wrote: ***@***.***(https://github.com/Tylersuard) thanks for drafting this. Looks like a reasonable approach and I think this will be useful to the users. :) Could you please send a PR for one of the notebooks so that we can discuss any details and then consider porting this to the rest of the notebooks? — You are receiving this because you were mentioned. Reply to this email directly, [view it on GitHub](#3176 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AJ6H6YLEAQZSWSXXBWUUNQDSVIJWTANCNFSM4U5TXVGQ).

[datumbox]

@Tylersuard We never brought your API to the other models so that we can close the ticket. Would you be interested in doing it?

[Tylersuard]

Do you mean this? If any other notebooks need changes, I would be happy to make them. pytorch/hub#179

[datumbox]

Yes, exactly. Seems like it's all great then. I'll close the ticket.