Two pass analysis with lazy types #1756
Description
Currently dependency analysis creates full graph and walks it. Instead, we could use linker-type approach.
When analysis discovers import, it loads module, parses and do a first pass collecting symbols. This should be quick, like indexing. Symbols go to a global symbol resolution. Some types can be created as lazy similar to #1728 avoiding analysis of some of the modules until needed. When first pass completes, second pass then resolves symbols across loaded modules.
Loaded modules don't need to be fully analyzed, symbol resolution may be delayed until module member is requested as in #1757. The member should also be lazy since its 'deep' data like function return types or signatures may never be needed.
This is already done within single module for resolution of forward references and circular dependencies between types inside the module. Similar approach should work across modules. This should simplify and shorted analysis pass and simplify handling of circular dependencies.
Related
- Long analysis on changes with large file in dependency graph #1502
- Python Language server takes a huge amount of RAM (more than 10+GB) #1426
- High memory usage in 0.3.40(memory leak?) #1298
- Occasional failures in tests when dependency analysis claims it is done, but some tasks are still running.