Skip to content

Reinstate and deprecate model_urls and quant_model_urls #5992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
May 12, 2022

Conversation

datumbox
Copy link
Contributor

@datumbox datumbox commented May 11, 2022

Resolves #5836

This PR:

  • Reinstates the internal model_urls/quant_model_urls dictionaries and deprecates its usages. We do this ONLY for the models that used dictionaries with names that didn't start with _ and existed in version v0.12. We don't include any new model builders or weights on the deprecation dictionary.
  • Updates all deprecation warnings of Multi-weights support API to add specific deletion versions.

Verified with:

import torchvision
import inspect


def fetch_model_urls_from_module(main_module, attribute):
    l = inspect.getmembers(main_module, inspect.ismodule)
    l = [x for x in l if hasattr(x[1], attribute)]
    return dict(
        (t[0], getattr(t[1], attribute)) for t in l
    )


model_urls = {
    "models": fetch_model_urls_from_module(torchvision.models, "model_urls"),
    "detection": fetch_model_urls_from_module(torchvision.models.detection, "model_urls"),
    "optical_flow": fetch_model_urls_from_module(torchvision.models.optical_flow, "model_urls"),
    "quantization": fetch_model_urls_from_module(torchvision.models.quantization, "quant_model_urls"),
    "segmentation": fetch_model_urls_from_module(torchvision.models.segmentation, "model_urls"),
    "video": fetch_model_urls_from_module(torchvision.models.video, "model_urls"),
}

# produced exactly as above by checking out branch release/0.12
model_urls_v012 = {'models': {'densenet': {'densenet121': 'https://download.pytorch.org/models/densenet121-a639ec97.pth', 'densenet169': 'https://download.pytorch.org/models/densenet169-b2777c0a.pth', 'densenet201': 'https://download.pytorch.org/models/densenet201-c1103571.pth', 'densenet161': 'https://download.pytorch.org/models/densenet161-8d451a50.pth'}, 'efficientnet': {'efficientnet_b0': 'https://download.pytorch.org/models/efficientnet_b0_rwightman-3dd342df.pth', 'efficientnet_b1': 'https://download.pytorch.org/models/efficientnet_b1_rwightman-533bc792.pth', 'efficientnet_b2': 'https://download.pytorch.org/models/efficientnet_b2_rwightman-bcdf34b7.pth', 'efficientnet_b3': 'https://download.pytorch.org/models/efficientnet_b3_rwightman-cf984f9c.pth', 'efficientnet_b4': 'https://download.pytorch.org/models/efficientnet_b4_rwightman-7eb33cd5.pth', 'efficientnet_b5': 'https://download.pytorch.org/models/efficientnet_b5_lukemelas-b6417697.pth', 'efficientnet_b6': 'https://download.pytorch.org/models/efficientnet_b6_lukemelas-c76e70fd.pth', 'efficientnet_b7': 'https://download.pytorch.org/models/efficientnet_b7_lukemelas-dcc49843.pth'}, 'inception': {'inception_v3_google': 'https://download.pytorch.org/models/inception_v3_google-0cc3c7bd.pth'}, 'mobilenetv2': {'mobilenet_v2': 'https://download.pytorch.org/models/mobilenet_v2-b0353104.pth'}, 'mobilenetv3': {'mobilenet_v3_large': 'https://download.pytorch.org/models/mobilenet_v3_large-8738ca79.pth', 'mobilenet_v3_small': 'https://download.pytorch.org/models/mobilenet_v3_small-047dcff4.pth'}, 'regnet': {'regnet_y_400mf': 'https://download.pytorch.org/models/regnet_y_400mf-c65dace8.pth', 'regnet_y_800mf': 'https://download.pytorch.org/models/regnet_y_800mf-1b27b58c.pth', 'regnet_y_1_6gf': 'https://download.pytorch.org/models/regnet_y_1_6gf-b11a554e.pth', 'regnet_y_3_2gf': 'https://download.pytorch.org/models/regnet_y_3_2gf-b5a9779c.pth', 'regnet_y_8gf': 'https://download.pytorch.org/models/regnet_y_8gf-d0d0e4a8.pth', 'regnet_y_16gf': 'https://download.pytorch.org/models/regnet_y_16gf-9e6ed7dd.pth', 'regnet_y_32gf': 'https://download.pytorch.org/models/regnet_y_32gf-4dee3f7a.pth', 'regnet_x_400mf': 'https://download.pytorch.org/models/regnet_x_400mf-adf1edd5.pth', 'regnet_x_800mf': 'https://download.pytorch.org/models/regnet_x_800mf-ad17e45c.pth', 'regnet_x_1_6gf': 'https://download.pytorch.org/models/regnet_x_1_6gf-e3633e7f.pth', 'regnet_x_3_2gf': 'https://download.pytorch.org/models/regnet_x_3_2gf-f342aeae.pth', 'regnet_x_8gf': 'https://download.pytorch.org/models/regnet_x_8gf-03ceed89.pth', 'regnet_x_16gf': 'https://download.pytorch.org/models/regnet_x_16gf-2007eb11.pth', 'regnet_x_32gf': 'https://download.pytorch.org/models/regnet_x_32gf-9d47f8d0.pth'}, 'resnet': {'resnet18': 'https://download.pytorch.org/models/resnet18-f37072fd.pth', 'resnet34': 'https://download.pytorch.org/models/resnet34-b627a593.pth', 'resnet50': 'https://download.pytorch.org/models/resnet50-0676ba61.pth', 'resnet101': 'https://download.pytorch.org/models/resnet101-63fe2227.pth', 'resnet152': 'https://download.pytorch.org/models/resnet152-394f9c45.pth', 'resnext50_32x4d': 'https://download.pytorch.org/models/resnext50_32x4d-7cdf4587.pth', 'resnext101_32x8d': 'https://download.pytorch.org/models/resnext101_32x8d-8ba56ff5.pth', 'wide_resnet50_2': 'https://download.pytorch.org/models/wide_resnet50_2-95faca4d.pth', 'wide_resnet101_2': 'https://download.pytorch.org/models/wide_resnet101_2-32ee1156.pth'}, 'shufflenetv2': {'shufflenetv2_x0.5': 'https://download.pytorch.org/models/shufflenetv2_x0.5-f707e7126e.pth', 'shufflenetv2_x1.0': 'https://download.pytorch.org/models/shufflenetv2_x1-5666bf0f80.pth', 'shufflenetv2_x1.5': None, 'shufflenetv2_x2.0': None}, 'squeezenet': {'squeezenet1_0': 'https://download.pytorch.org/models/squeezenet1_0-b66bff10.pth', 'squeezenet1_1': 'https://download.pytorch.org/models/squeezenet1_1-b8a52dc0.pth'}, 'vgg': {'vgg11': 'https://download.pytorch.org/models/vgg11-8a719046.pth', 'vgg13': 'https://download.pytorch.org/models/vgg13-19584684.pth', 'vgg16': 'https://download.pytorch.org/models/vgg16-397923af.pth', 'vgg19': 'https://download.pytorch.org/models/vgg19-dcbb9e9d.pth', 'vgg11_bn': 'https://download.pytorch.org/models/vgg11_bn-6002323d.pth', 'vgg13_bn': 'https://download.pytorch.org/models/vgg13_bn-abd245e5.pth', 'vgg16_bn': 'https://download.pytorch.org/models/vgg16_bn-6c64b313.pth', 'vgg19_bn': 'https://download.pytorch.org/models/vgg19_bn-c79401a0.pth'}, 'vision_transformer': {'vit_b_16': 'https://download.pytorch.org/models/vit_b_16-c867db91.pth', 'vit_b_32': 'https://download.pytorch.org/models/vit_b_32-d86f8d99.pth', 'vit_l_16': 'https://download.pytorch.org/models/vit_l_16-852ce7e3.pth', 'vit_l_32': 'https://download.pytorch.org/models/vit_l_32-c7638314.pth'}}, 'detection': {'faster_rcnn': {'fasterrcnn_resnet50_fpn_coco': 'https://download.pytorch.org/models/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth', 'fasterrcnn_mobilenet_v3_large_320_fpn_coco': 'https://download.pytorch.org/models/fasterrcnn_mobilenet_v3_large_320_fpn-907ea3f9.pth', 'fasterrcnn_mobilenet_v3_large_fpn_coco': 'https://download.pytorch.org/models/fasterrcnn_mobilenet_v3_large_fpn-fb6a3cc7.pth'}, 'fcos': {'fcos_resnet50_fpn_coco': 'https://download.pytorch.org/models/fcos_resnet50_fpn_coco-99b0c9b7.pth'}, 'keypoint_rcnn': {'keypointrcnn_resnet50_fpn_coco_legacy': 'https://download.pytorch.org/models/keypointrcnn_resnet50_fpn_coco-9f466800.pth', 'keypointrcnn_resnet50_fpn_coco': 'https://download.pytorch.org/models/keypointrcnn_resnet50_fpn_coco-fc266e95.pth'}, 'mask_rcnn': {'maskrcnn_resnet50_fpn_coco': 'https://download.pytorch.org/models/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth'}, 'retinanet': {'retinanet_resnet50_fpn_coco': 'https://download.pytorch.org/models/retinanet_resnet50_fpn_coco-eeacb38b.pth'}, 'ssd': {'ssd300_vgg16_coco': 'https://download.pytorch.org/models/ssd300_vgg16_coco-b556d3b4.pth'}, 'ssdlite': {'ssdlite320_mobilenet_v3_large_coco': 'https://download.pytorch.org/models/ssdlite320_mobilenet_v3_large_coco-a79551df.pth'}}, 'optical_flow': {}, 'quantization': {'inception': {'inception_v3_google_fbgemm': 'https://download.pytorch.org/models/quantized/inception_v3_google_fbgemm-71447a44.pth'}, 'mobilenetv2': {'mobilenet_v2_qnnpack': 'https://download.pytorch.org/models/quantized/mobilenet_v2_qnnpack_37f702c5.pth'}, 'mobilenetv3': {'mobilenet_v3_large_qnnpack': 'https://download.pytorch.org/models/quantized/mobilenet_v3_large_qnnpack-5bcacf28.pth'}, 'resnet': {'resnet18_fbgemm': 'https://download.pytorch.org/models/quantized/resnet18_fbgemm_16fa66dd.pth', 'resnet50_fbgemm': 'https://download.pytorch.org/models/quantized/resnet50_fbgemm_bf931d71.pth', 'resnext101_32x8d_fbgemm': 'https://download.pytorch.org/models/quantized/resnext101_32x8_fbgemm_09835ccf.pth'}, 'shufflenetv2': {'shufflenetv2_x0.5_fbgemm': 'https://download.pytorch.org/models/quantized/shufflenetv2_x0.5_fbgemm-00845098.pth', 'shufflenetv2_x1.0_fbgemm': 'https://download.pytorch.org/models/quantized/shufflenetv2_x1_fbgemm-db332c57.pth'}}, 'segmentation': {'deeplabv3': {'deeplabv3_resnet50_coco': 'https://download.pytorch.org/models/deeplabv3_resnet50_coco-cd0a2569.pth', 'deeplabv3_resnet101_coco': 'https://download.pytorch.org/models/deeplabv3_resnet101_coco-586e9e4e.pth', 'deeplabv3_mobilenet_v3_large_coco': 'https://download.pytorch.org/models/deeplabv3_mobilenet_v3_large-fc3c493d.pth'}, 'fcn': {'fcn_resnet50_coco': 'https://download.pytorch.org/models/fcn_resnet50_coco-1167a1af.pth', 'fcn_resnet101_coco': 'https://download.pytorch.org/models/fcn_resnet101_coco-7ecb50ca.pth'}, 'lraspp': {'lraspp_mobilenet_v3_large_coco': 'https://download.pytorch.org/models/lraspp_mobilenet_v3_large-d234d4ea.pth'}}, 'video': {'resnet': {'r3d_18': 'https://download.pytorch.org/models/r3d_18-b3b3357e.pth', 'mc3_18': 'https://download.pytorch.org/models/mc3_18-a90a0ba3.pth', 'r2plus1d_18': 'https://download.pytorch.org/models/r2plus1d_18-91a641e6.pth'}}}


assert model_urls == model_urls_v012
print("All good")

Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @datumbox , LGTM. Just some nits

@datumbox datumbox changed the title Reinstate and deprecate model_urls and quant_model_urls [WIP] Reinstate and deprecate model_urls and quant_model_urls May 11, 2022
@datumbox datumbox changed the title [WIP] Reinstate and deprecate model_urls and quant_model_urls Reinstate and deprecate model_urls and quant_model_urls May 12, 2022
@datumbox datumbox added module: models module: models.quantization Issues related to the quantizable/quantized models deprecation labels May 12, 2022
@datumbox
Copy link
Contributor Author

All failures are unrelated. Merging.

@datumbox datumbox merged commit e722e9c into pytorch:main May 12, 2022
@datumbox datumbox deleted the models/model_urls branch May 12, 2022 12:43
facebook-github-bot pushed a commit that referenced this pull request May 24, 2022
…5992)

Summary:
* Reinstate and deprecate `model_urls` and `quant_model_urls`

* Apply suggestions from code review

* Move todo location

* Add alexnet

* Add densenet

* Add efficientnet

* Add googlenet.

* Add inception.

* Add mobilenetv3

* Add regnet

* Add resnet

* Add shufflenetv2

* Fix linter

* Add squeezenet

* Add vgg

* Add vit

* Add quantized googlenet

* Add quantized inceptionv3

* Add quantized mobilenet_v3

* Add quantized resnet

* Add quantized shufflenetv2

* Fix incorrect imports

* Add faster_rcnn

* Add fcos

* Add keypoint rcnn

* Add mask rcnn

* Add retinanet

* Add ssd

* Add ssdlite

* Add deeplabv3

* Add fcn

* Add lraspp.

* Add video resnet

* Removing weights for shufflenetv2_x1.5 and shufflenetv2_x2.0

* Update the comments

Reviewed By: datumbox

Differential Revision: D36413362

fbshipit-source-id: 56d6855a77e17db457542cb1c12373a1a881b46b

Co-authored-by: Nicolas Hug <[email protected]>
Co-authored-by: Nicolas Hug <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed deprecation module: models.quantization Issues related to the quantizable/quantized models module: models
Projects
None yet
Development

Successfully merging this pull request may close these issues.

new model weights loading breaks backwards compatibility for existing models
3 participants