torchvision models no longer respect strict=False when loading

[d4l3k]

🐛 Describe the bug

After the recent changes to model loading in #5444 it's impossible to specify strict=False when loading a preexisting checkpoint. This is important since often when I'm trying a new submodel I'll replace just one piece and load the others from a checkpoint but the new piece from scratch.

import torchvision

model = torchvision.models.detection.fasterrcnn_mobilenet_v3_large_320_fpn()
model.load_state_dict({}, strict=False)

Traceback (most recent call last):
  File "/home/tristanr/Developer/torchvision-repro/repro.py", line 10, in <module>
    model.load_state_dict({}, strict=False)
  File "/home/tristanr/venvs/torchvision/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1566, in load_state_dict
    load(self)
  File "/home/tristanr/venvs/torchvision/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1564, in load
    load(child, prefix + name + '.')
  File "/home/tristanr/venvs/torchvision/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1564, in load
    load(child, prefix + name + '.')
  File "/home/tristanr/venvs/torchvision/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1560, in load
    module._load_from_state_dict(
  File "/home/tristanr/venvs/torchvision/lib/python3.10/site-packages/torchvision/ops/feature_pyramid_network.py", line 131, in _load_from_state_dict
    state_dict[new_key] = state_dict.pop(old_key)
KeyError: 'backbone.fpn.inner_blocks.0.weight'

Versions

PyTorch version: 1.12.0.dev20220419+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Arch Linux (x86_64)
GCC version: (GCC) 11.2.0
Clang version: Could not collect
CMake version: version 3.22.2
Libc version: glibc-2.35

Python version: 3.10.2 (main, Mar 15 2022, 12:43:00) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.16.11-arch1-1-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: 11.6.55
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.3.1
/usr/lib/libcudnn_adv_infer.so.8.3.1
/usr/lib/libcudnn_adv_train.so.8.3.1
/usr/lib/libcudnn_cnn_infer.so.8.3.1
/usr/lib/libcudnn_cnn_train.so.8.3.1
/usr/lib/libcudnn_ops_infer.so.8.3.1
/usr/lib/libcudnn_ops_train.so.8.3.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.12.0.dev20220419+cu113
[pip3] torchaudio==0.12.0.dev20220419+cu113
[pip3] torchvision==0.13.0.dev20220419+cu113
[conda] blas 1.0 mkl
[conda] magma-cuda110 2.5.2 1 pytorch
[conda] mkl 2021.4.0 h06a4308_640
[conda] mkl-include 2021.4.0 h06a4308_640
[conda] mkl-service 2.4.0 py39h7f8727e_0
[conda] mkl_fft 1.3.1 py39hd3c417c_0
[conda] mkl_random 1.2.2 py39h51133e4_0
[conda] mypy_extensions 0.4.3 py39h06a4308_0
[conda] numpy 1.20.3 py39hf144106_0
[conda] numpy-base 1.20.3 py39h74d4b33_0
[conda] numpydoc 1.1.0 pyhd3eb1b0_1

cc @datumbox

[datumbox]

@d4l3k Thanks fore reporting.

I confirm this is indeed a problem and it is related to #5444. I'll send a PR soon to fix.