Skip to content

GH-119169: Speed up os.walk(topdown=False) #119186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 12 commits into from

Conversation

barneygale
Copy link
Contributor

@barneygale barneygale commented May 19, 2024

Add entries to the stack while iterating over os.scandir() results, rather than afterwards.

Handle errors from `os.scandir()` and `ScandirIterator` similarly, which
lets us loop over directory entries with `for`. In top-down mode, call
`os.path.join()` at most once per iteration.
@vstinner
Copy link
Member

cc @serhiy-storchaka

@barneygale barneygale marked this pull request as draft May 23, 2024 08:46
@barneygale
Copy link
Contributor Author

#119473 fixes the test failure.

@barneygale barneygale changed the title GH-119169: Speed up os.walk() GH-119169: Speed up os.[f]walk() May 30, 2024
barneygale added a commit to barneygale/cpython that referenced this pull request Jul 6, 2024
When `os.walk()` traverses into subdirectories in top-down mode, call
`os.path.join()` once to add a trailing slash, and use string concatenation
thereafter to generate child paths.
@barneygale barneygale changed the title GH-119169: Speed up os.[f]walk() GH-119169: Speed up os.walk() Jul 6, 2024
barneygale added a commit to barneygale/cpython that referenced this pull request Jul 6, 2024
Handle errors from `os.scandir()` and `ScandirIterator` similarly, which
lets us loop over directory entries with `for`.
@barneygale barneygale changed the title GH-119169: Speed up os.walk() GH-119169: Speed up os.walk(topdown=False) Jul 6, 2024
barneygale added a commit to barneygale/cpython that referenced this pull request Jul 8, 2024
@barneygale
Copy link
Contributor Author

Closing because reversing sibling traversal order isn't kosher - see discussion on #119473.

@barneygale barneygale closed this Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance or resource usage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants