Description
🐞 bug report
Affected Rule
Its within the pip extension.
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse( ...
)
Is this a regression?
This works without modules, but doesn't now modules are enabled.
Description
Using the module extension to download whl files adds significantly to the path.
rules_python~~pip~example_pip_312_charset_normalizer
vs:
example_pip_charset_normalizer
when running on windows with --nobuild_python_zip you can easily exceed MAX_PATH, and any library which uses extensions will fail to load the extensions.
🔬 Minimal Reproduction
Reproduction attached:
pip_path_length.zip
Run: bazel build //example/... you get the following error:
INFO: Invocation ID: 19b20afe-a6fa-4573-bac9-f74bccb40807
INFO: Analyzed 3 targets (85 packages loaded, 3896 targets configured).
ERROR: D:/bazel_bugs/pip_path_length/example/BUILD.bazel:19:8: Executing genrule //example:run failed: (Exit 1): sh.exe failed: error executing Genrule command (from target //example:run) C:\Program Files\Git\bin\sh.exe -c ... (remaining 1 argument skipped)
Traceback (most recent call last):
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_requests\site-packages\requests\compat.py", line 11, in <module>
import chardet
ModuleNotFoundError: No module named 'chardet'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\_main\example\test.py", line 2, in <module>
import requests
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_requests\site-packages\requests\__init__.py", line 45, in <module>
from .exceptions import RequestsDependencyWarning
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_requests\site-packages\requests\exceptions.py", line 9, in <module>
from .compat import JSONDecodeError as CompatJSONDecodeError
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_requests\site-packages\requests\compat.py", line 13, in <module>
import charset_normalizer as chardet
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_charset_normalizer\site-packages\charset_normalizer\__init__.py", line 26, in <module>
from .api import from_bytes, from_fp, from_path, is_binary
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_charset_normalizer\site-packages\charset_normalizer\api.py", line 7, in <module>
from .cd import (
File "c:\dev\_bazel\x3mr6iom\execroot\_main\bazel-out\x64_windows-opt-exec-st-d57f47055a04\bin\example\a_moderatly_long_tool_path.exe.runfiles\rules_python~~pip~example_pip_312_charset_normalizer\site-packages\charset_normalizer\cd.py", line 16, in <module>
from .md import is_suspiciously_successive_range
ImportError: DLL load failed while importing md__mypyc: The filename or extension is too long.
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.440s, Critical Path: 0.93s
INFO: 7 processes: 6 internal, 1 local.
ERROR: Build did NOT complete successfully
You can open up the directories and you can see its trying to open a path greater than MAX_PATH.
To see what its old behaviour: bazel build //example/... --noenable_bzlmod
🌍 Your Environment
Operating System:
WINDOWS 11
Output of bazel version:
7.5.0
Rules_python version:
1.1.0
Anything else relevant?
We enable build --nobuild_python_zip on windows in our repo because the performance without is enabled it too slow. We use python to process many GB of assets using the data construct, zipping that all up is unfeasible.
Note: if you comment out:
https://github.com/bazelbuild/rules_python/blob/e7d2f09394dd14816310c4c661d2fefab33b2b1b/python/private/python_bootstrap_template.txt#L55-L56
then we are presented with the UNC paths and LoadLibrary in python works.