Skip to content

add EuroSAT prototype dataset #5452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 3, 2022
Merged

Conversation

Dbhasin1
Copy link
Contributor

fixes #5338

@facebook-github-bot
Copy link

facebook-github-bot commented Feb 21, 2022

💊 CI failures summary and remediations

As of commit 5b34400 (more details on the Dr. CI page):


💚 💚 Looks good so far! There are no failures yet. 💚 💚


This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Copy link
Collaborator

@pmeier pmeier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR @Dbhasin1! I only have some style nit inline. Apart from that, we should also add a test for the dataset. To do that you add a

@register_mock
def eurosat(info, root, config):
    ...

to the file test/builtin_dataset_mocks.py. The body of this function should generate mock data in the same structure of the original dataset, but with only a few samples. Luckily, we already have this functionality for the old dataset in

vision/test/test_datasets.py

Lines 2176 to 2190 in c530b62

def inject_fake_data(self, tmpdir, config):
data_folder = os.path.join(tmpdir, "eurosat", "2750")
os.makedirs(data_folder)
num_examples_per_class = 3
classes = ("AnnualCrop", "Forest")
for cls in classes:
datasets_utils.create_image_folder(
root=data_folder,
name=cls,
file_name_fn=lambda idx: f"{cls}_{idx}.jpg",
num_examples=num_examples_per_class,
)
return len(classes) * num_examples_per_class

Thus, you only need to port this as well to the new architecture. Let me know if you face any issues with that.

Copy link
Collaborator

@pmeier pmeier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor things left. Otherwise LGTM when CI is green. Thanks a lot @Dbhasin1!

Copy link
Collaborator

@pmeier pmeier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor thing left. Otherwise LGTM! Thanks a ton @Dbhasin1!

@pmeier pmeier requested a review from NicolasHug March 1, 2022 18:25
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stamping, thanks @Dbhasin1

@NicolasHug NicolasHug merged commit 97385df into pytorch:main Mar 3, 2022
facebook-github-bot pushed a commit that referenced this pull request Mar 15, 2022
Summary:
* add eurosat

* revert formatting

* port test and make style changes

* add eurosat to __init__

* fix pathlib error

* create dataset zipfile and revert pre commit changes

* remove unecessary variable in resources

* revert auto formatter changes and modify ufmt version

* revert change to contributing guide

Reviewed By: vmoens

Differential Revision: D34878968

fbshipit-source-id: 845347b981c85fff13111f77a33119181f863829

Co-authored-by: Dbhasin1 <[email protected]>
Co-authored-by: Nicolas Hug <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EuroSAT
4 participants