-
Notifications
You must be signed in to change notification settings - Fork 100
Closed
Description
I see that the ImageNet-C evaluation uses the preprocessing: Resize(256)+CenterCrop(224)+ToTensor()
.
robustbench/robustbench/data.py
Lines 146 to 154 in 61ce9e9
def load_imagenetc( | |
n_examples: Optional[int] = 5000, | |
severity: int = 5, | |
data_dir: str = './data', | |
shuffle: bool = False, | |
corruptions: Sequence[str] = CORRUPTIONS, | |
prepr: str = 'Res256Crop224' | |
) -> Tuple[torch.Tensor, torch.Tensor]: | |
transforms_test = PREPROCESSINGS[prepr] |
This causes discrepancies with the scores reported in the original papers (DeepAugment, AugMix, Standard RN-50). The ImageNet-C dataset already contains 224x224 images and hence only ToTensor()
should be used for consistency.
Fixing prepr='none'
in load_imagenetc
should solve the issue (assuming all the models are capable of handling 224x224 images as input).
Metadata
Metadata
Assignees
Labels
No labels