Delay astroid_bootstrapping() until instantiating AstroidBuilder

[jacobtylerwalls]

Type of Changes

	Type
✓	🔨 Refactoring

Description

Reduce time to import astroid by delaying astroid_bootstrapping() until the first instantiation of AstroidBuilder.

Closes #2161

Benchmark

python3 -X importtime -c 'import astroid'

main@ 4493399

import time:     34604 |     315816 | astroid

pr

import time:     14397 |      75213 | astroid

[codecov]

Codecov Report

Merging #2210 (8f78173) into main (1fbbf25) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2210   +/-   ##
=======================================
  Coverage   92.68%   92.69%           
=======================================
  Files          94       94           
  Lines       10828    10835    +7     
=======================================
+ Hits        10036    10043    +7     
  Misses        792      792           

Flag	Coverage Δ
linux	92.45% <100.00%> (+<0.01%)	⬆️
pypy	87.65% <100.00%> (+<0.01%)	⬆️
windows	92.28% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
astroid/brain/brain_builtin_inference.py	91.64% <100.00%> (+0.01%)	⬆️
astroid/brain/brain_nose.py	95.00% <100.00%> (+0.12%)	⬆️
astroid/builder.py	94.19% <100.00%> (+0.05%)	⬆️
astroid/raw_building.py	94.96% <100.00%> (+0.04%)	⬆️

[Pierre-Sassoulas]

Wow, 4 Time faster ? Or at least 2 times ? Not sure if I read the data corectly. It seems you did not have to add lazy anything inside the astroid manager either ? I'm going to check if that's yet another way to make it faster in the future once I'm not on mobile. Exciting change either ways ! The impact will be enormous when pylint is launched in parallele on small files by an external tool like pre-commit !

[DanielNoord]

Doesn't it make more sense for the bootstrapping to occur when AstroidManager is created? Its name (to me) implies that I would be responsible for this instead of the Builder.

[jacobtylerwalls]

Doesn't it make more sense for the bootstrapping to occur when AstroidManager is created? Its name (to me) implies that I would be responsible for this instead of the Builder.

This would create an infinite loop, since calling _astroid_bootstrapping() from AstroidManager.__init__() would create a Builder, which creates a Manager:

astroid/astroid/raw_building.py

Lines 597 to 602 in 1fbbf25

	def _astroid_bootstrapping() -> None:
	"""astroid bootstrapping the builtins module"""
	# this boot strapping is necessary since we need the Const nodes to
	# inspect_build builtins, and then we can proxy Const
	builder = InspectBuilder()
	astroid_builtin = builder.inspect_build(builtins)

astroid/astroid/raw_building.py

Lines 423 to 431 in 1fbbf25

	class InspectBuilder:
	"""class for building nodes from living object
	this is actually a really minimal representation, including only Module,
	FunctionDef and ClassDef nodes and some others as guessed.
	"""
	def __init__(self, manager_instance: AstroidManager | None = None) -> None:
	self._manager = manager_instance or AstroidManager()

Also, I like the idea of keeping this guard closer to what it actually needs to guard--the building of the stdlib ast by the Builder.

[DanielNoord]

Doesn't it make more sense for the bootstrapping to occur when AstroidManager is created? Its name (to me) implies that I would be responsible for this instead of the Builder.

This would create an infinite loop, since calling _astroid_bootstrapping() from AstroidManager.__init__() would create a Builder, which creates a Manager:

astroid/astroid/raw_building.py

Lines 597 to 602 in 1fbbf25

	def _astroid_bootstrapping() -> None:
	"""astroid bootstrapping the builtins module"""
	# this boot strapping is necessary since we need the Const nodes to
	# inspect_build builtins, and then we can proxy Const
	builder = InspectBuilder()
	astroid_builtin = builder.inspect_build(builtins)

astroid/astroid/raw_building.py

Lines 423 to 431 in 1fbbf25

	class InspectBuilder:
	"""class for building nodes from living object
	this is actually a really minimal representation, including only Module,
	FunctionDef and ClassDef nodes and some others as guessed.
	"""
	def __init__(self, manager_instance: AstroidManager | None = None) -> None:
	self._manager = manager_instance or AstroidManager()

Also, I like the idea of keeping this guard closer to what it actually needs to guard--the building of the stdlib ast by the Builder.

Sounds fine! Thanks for explaining.

[Pierre-Sassoulas]

Just tested the changes and it look like it's almost 3 time faster, this is great work !