Multi pretrained weights: Cleanups and Refactoring

[jdsgomes]

Candidates:

• Reduce code duplication on model builders by refactoring. For example potentially push the weigh loading mechanism in the private builder methods currently inherited from torchvision.models. Also we could provide a pretrained deprecation method.
• Review recipe links and ensure they are accurate for all Community Contributed models. Check if possible for some of them to remap their training process to our current ref scripts.
• Decide how quantized models will link to their unquantized versions.
• Naming conventions:
  • The name of the Weights base Enum class and the naming convention of the inheriting classes. Also make sure that all classes are aligned with the model builder names. How are we going to handle _ in names (for example on inception_v3 and the resnext models)?
  • The name of the WeightEntry data class.
  • The name of Weights.verify() method (because it does more than just verification).
  • The names of the enum values; also their styles (all capital?). Example: ImageNet1K_RefV1
  • Currently the _RefV1 of one model doesn't have to be _RefV1 of another model. For example ResNet vs RegNets use different recipes with the same enum value. In other cases, we skip _RefV1 and go directly to _RefV2, for example the WideResNet50 weights. Is this a problem?
• Decide the signature of models._api.get_weight() and make it public. Ensure it's torchhub-friendly.

cc @datumbox @pmeier @bjuncek @NicolasHug

[jdsgomes]

Just opening the discussion here regarding the naming for Weights and WeightEntry. As This classes store metadata about weights and how they were generated (recipe + transforms) rather than the actual weights so I think we could try to find a better name.

Having said that I'm struggling to find a much better name, but want to leave some suggestions for discussion:

• PretrainedModelEnum and PretrainedModel
• PretrainedModel and PretrainedModelEntry
• Weights and WeightsInfo

Open for more suggestions. Naming IS hard :)

[jdsgomes]

Inside the Weights class we have a verify method that has some side effects depending on the input parameter. I suggest to change this to build or create it would be clear that we are trying to build an object from a string or from another object of the same type (in this case it will be an identity function). And this builder would throw an exception if the arguments were invalid.

[datumbox]

Concerning the WeightEntry name, Motto has advocated using the word Bundle. You can read more here: datumbox/dapi-model-versioning#1 (comment)

[NicolasHug]

Thanks for opening this discussion @jdsgomes , I agree naming is important! (and hard)

In this case, everything is private right? If we don't expect users to interact directly with these classes (i.e they won't be inheriting from them), I feel like there isn't too much risk in choosing an imperfect naming, because this will only really affect us (the devs).

I personally don't mind Weights and WeightEntry, they're not super obvious at first but it becomes clear when one sees the instanciation of e.g. the ResNet50Weights class. I think this will be the case for any naming we choose IMHO. My only note about the current naming is that WeightEntry could probably be WeightsEntry.

I think your suggestions above are good too. I'll just add another one: WeightsEnum and Weights. e.g.:

class ResNet50Weights(WeightsEnum):
    ImageNet1K_RefV1 = Weights(...)
    ImageNet1K_RefV2 = Weights(...)
    ...

[datumbox]

@NicolasHug Thanks for the thoughts. Another idea we discussed with @jdsgomes offline was the fact that the names of Enums are too long due to our naming conventions for some models. For example for fasterrcnn_resnet50_fpn() the enum is FasterRCNNResNet50FPNWeights which frankly is too long and too unreadable. One option would be to drop completly the "Weights" from the inheriting classes. For example: FasterRCNNResNet50FPN.

At any case, please everyone feel free to add candidates on the list. To unblock the multi-pretrained initiative we were lenient on the reviews concerning nitpicking, naming etc, so we should definitely do a more careful round before moving the code to main torchvision. :)