Skip to content

Clarification for training resnext101_32x8d on ImageNet #4390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 10, 2021
Merged

Clarification for training resnext101_32x8d on ImageNet #4390

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3033c9d
Fix training resuming in references/segmentation
netw0rkf10w Apr 24, 2020
0d7913d
resolve conflict when pulling upstream
netw0rkf10w Jul 7, 2021
0a05682
Merge branch 'master' of https://github.com/pytorch/vision
netw0rkf10w Sep 10, 2021
181f81a
Clarification for training resnext101_32x8d
netw0rkf10w Sep 10, 2021
d64f46c
Update references/classification/README.md
netw0rkf10w Sep 10, 2021
5eafd72
Merge branch 'main' into references_doc
NicolasHug Sep 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion references/classification/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,17 @@ python -m torch.distributed.launch --nproc_per_node=8 --use_env train.py\

### ResNext-101 32x8d

On 8 nodes, each with 8 GPUs (for a total of 64 GPUS)
```
python -m torch.distributed.launch --nproc_per_node=8 --use_env train.py\
--model resnext101_32x8d --epochs 100
```

Note that the above command corresponds to a single node with 8 GPUs. If you use
a different number of GPUs and/or a different batch size, then the learning rate
should be scaled accordingly. For example, the pretrained model provided by
`torchvision` was trained on 8 nodes, each with 8 GPUs (for a total of 64 GPUs),
with `--batch_size 16` and `--lr 0.4`, instead of the current defaults
which are respectively batch_size=32 and lr=0.1

### MobileNetV2
```
Expand Down