Add iNaturalist dataset

[vfdev-5]

🚀 Feature

A feature request to discuss about about adding iNaturalist dataset wrapper of ImageFolder (maybe, even without downloading tools).

Motivation

We can see in the research code custom dataset implementation which can be also in torchvision. For example, here:

https://github.com/facebookresearch/deit/blob/ab5715372db8c6cad5740714b2216d55aeae052e/datasets.py#L13

Being also aware of our not-yet-defined procedure on which datasets to add, feel free to close the FR if we can not move forward.

cc @pmeier

[pmeier]

It seems to be cited 159 times. Given that this is not a general purpose classification dataset, I think it is reasonable popular. @vfdev-5 do you have insights if this is still used in contemporary publications or is even picking up in popularity?

[vfdev-5]

According to scholar, there are already 9 papers citing the dataset in 2021. Code cited above is for Deit (12/2020) paper.
Looks like a popular dataset. Let's see other opinions here before going further.

[pmeier]

Just to be clear: I'm in favor of adding this.

[arch-user-france1]

There is a mistake in the documentation.

The module is spelled lowercase (torchvision.datasets.inaturalist). There is no torchvision.datasets.INaturalist.

Wait what? I can not import INaturalist anymore at all. Maybe it was spelled INaturalist but has been removed.

    dataset = datasets.inaturalist(
              ^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'torchvision.datasets' has no attribute 'inaturalist'

    dataset = datasets.INaturalist(
              ^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'torchvision.datasets' has no attribute 'INaturalist'

Is INaturalist still there? I have been able to use it before.

[pmeier]

@arch-user-france1 There is no mistake in the documentation since it doesn't describe the module, but rather the class:

vision/torchvision/datasets/inaturalist.py

Line 31 in 120e7af

class INaturalist(VisionDataset):

Seeing the AttributeError's above, I guess something is wrong with your environment. Please open a new issue following the steps in the form so we can have a look.

[arch-user-france1]

@arch-user-france1 There is no mistake in the documentation since it doesn't describe the module, but rather the class:

vision/torchvision/datasets/inaturalist.py

Line 31 in 120e7af

class INaturalist(VisionDataset):

Seeing the AttributeError's above, I guess something is wrong with your environment. Please open a new issue following the steps in the form so we can have a look.

Ah, forget it. I had set up a buggy environment. Fixed it... Weird, though.

[arch-user-france1]

I think there is a memory leak in the dataset or DataLoader - is it supposed to use more than 30GB of RAM? It does not free any memory, so a dataset of a this size would be unusable. Nobody has 200GB RAM...
I have not spotted any issue in my code, and I have seen a stackoverflow question describing the same problem.

[pmeier]

@arch-user-france1 This has nothing to do with this issue. Please open a new one following the steps in the form.

[nhewadehigah]

Hi Do anyone know how to access the supercategory label in 2021 version. In tensorflow it has supercategory label but not seen in pytorch.