Skip to content

Fix incremental speed regression #4331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 8, 2017

Conversation

gvanrossum
Copy link
Member

Fixes #4135.

@gvanrossum gvanrossum force-pushed the fix-4135-incremental-speed branch from 2421d80 to 47b29de Compare December 6, 2017 19:39
@gvanrossum gvanrossum requested a review from JukkaL December 6, 2017 21:07
@gvanrossum
Copy link
Member Author

Don't merge yet -- this has an O(N**2) component. We can do better by having an "errors" bit per file. (Though it's really per SCC that counts.)

@gvanrossum gvanrossum force-pushed the fix-4135-incremental-speed branch from 3ecc740 to 257a9d1 Compare December 8, 2017 05:30
@gvanrossum
Copy link
Member Author

gvanrossum commented Dec 8, 2017

What do you think of this version? It sets a flag transitive_error when there's an error in either the current module or any of its dependencies. (Except it does it it per SCC, so if any dependency of any module in an SCC has this flag set, all modules in the SCC get it set.)

There's one bit of strangeness (see the last local commit) -- one test decided to have a different [stale] list.

(If you'd rather see the previous O(N**2) version, I can probably get it back using git reflog. :-)

@JukkaL
Copy link
Collaborator

JukkaL commented Dec 8, 2017

The approach looks good to me. I'm a little worried about the difference in the test case though -- might be worth figuring out why it has changed. Feel free to merge this once you are happy with this (and have fixed Travis).

@gvanrossum
Copy link
Member Author

Ah, I understand why that test (testIncrementalWithSilentImportsAndIgnore) had to be modified. There are no errors in c/__init__.py, and it does not depend on modules with errors. However it is processed after a module with errors (c/submodule.py), and with the old approach that meant no cache file would be written for c/__init__.py (in fact that was the bug we're fixing here). But with the new approach the file is not considered to have a (transitive) error, and the logic in write_cache() decides it can write the cache file. So it's a good thing.

@gvanrossum gvanrossum merged commit 01de89f into python:master Dec 8, 2017
@gvanrossum gvanrossum deleted the fix-4135-incremental-speed branch December 8, 2017 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants