Skip to content

_find_classes() underperforming for large numbers of classes #557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
da-wad opened this issue Jul 25, 2018 · 2 comments
Closed

_find_classes() underperforming for large numbers of classes #557

da-wad opened this issue Jul 25, 2018 · 2 comments

Comments

@da-wad
Copy link

da-wad commented Jul 25, 2018

When training with large (>10^5) numbers of classes (Ref. https://arxiv.org/abs/1805.01978) the _find_classes() method is sloooow. Replacing the first line with: classes = [d.name for d in os.scandir(dir) if d.is_dir()] works correctly for me and is a lot faster (see Python docs: https://docs.python.org/3/library/os.html#os.listdir ). Thought others might appreciate this...

@soumith
Copy link
Member

soumith commented Jul 27, 2018

thanks a lot. would you be kind enough to open a PR for this (with a if python3 guard)

da-wad pushed a commit to da-wad/vision that referenced this issue Jul 27, 2018
soumith pushed a commit that referenced this issue Jul 27, 2018
* Use scandir in _find_classes (#557)

* fix lint
@soumith
Copy link
Member

soumith commented Jul 27, 2018

closed via #559

@soumith soumith closed this as completed Jul 27, 2018
rajveerb pushed a commit to rajveerb/vision that referenced this issue Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants