[BUG] ensure_local_distutils seems to fail inside PyInstaller .exe (for new setuptools versions >= 60)

[xjcl]

setuptools version

60.5.0 (>= 60.0.0 seems affected)

Python version

3.7.7 on win32 (3.7 - 3.10 seem affected)

OS

Windows 10

Additional environment information

No response

Description

I described this issue in detail on StackOverflow: https://stackoverflow.com/q/71027006/2111778 Below a copy:

---

I am trying to convert some Python code into an .exe with PyInstaller. My code uses distutils, which has already caused me some head scratching in the past as it seems to duplicate setuptools functionality. It also requires e.g. an unused import of setuptools to work properly which seems very unpythonic to me.

My first attempt to create an exe failed with the error message Module not found: 'setuptools' because my code only does import distutils explicitly (this works fine, but not inside the exe build). But knowing about the "unused import trick" I changed that to essentially import setuptools; import distutils, which basically instructs PyInstaller to include the setuptools module as well.

My script runs fine but after I turn it into an exe I get a traceback inside of the suspicious _distutils_hack submodule of setuptools. And yes it just prints a file name with no context.

Traceback (most recent call last):
  <18 lines omitted>
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "_distutils_hack\override.py", line 71, in <module>
  File "_distutils_hack\__init__.py", line 71, in do_override
  File "_distutils_hack\__init__.py", line 59, in ensure_local_distutils
AssertionError: C:\Users\<omitted>\AppData\Local\Temp\_MEI294562\distutils\core.pyc 

I am using

• Python 3.7
• pyinstaller==4.8 (Jan 2022) for Windows
• distutils==3.7.7 (built-in)
• setuptools==60.5.0 (Jan 2022)

Apparently, setuptools is listed on PyPI and thus upgradable, but distutils is not listed on PyPI and thus not upgradable (the version is bundled with Python).

A workaround I found was downgrading to any version before 60

pip install --upgrade setuptools==59.8.0

Expected behavior

See above

How to Reproduce

See above

Output

See above

[jaraco]

Sorry for the trouble. There's a lot going on here that we'll need to tease apart.

My guess is that the newest setuptools just is not compatible with Python 3.7 but mistakenly labeled as such.

That's not the case. The latest releases of Setuptools still support Python 3.7.

It is the case that starting with Setuptools 60, however, Setuptools is adopting distutils (because stdlib distutils is expected to go away in Python 3.12), supplying its own copy internally and using the _distutils_hack to ensure that version of distutils is used. It's not a big surprise that Pyinstaller is tripping up on some of this behavior.

Can you put together a more complete reproducer, preferably a minimum one that encounters the error?

Are you using setuptools to build your project or are you using setuptools inside your project (or both)?

I notice you're using import setuptools for its side effect. Instead, you should use setuptools directly, i.e. import setuptools; setuptools.setup(...).

[xjcl]

Thanks a lot for the quick reply.

I notice you're using import setuptools for its side effect. Instead, you should use setuptools directly

I agree. distutils is deprecated in favor of setuptools (I incorrectly remembered it the other way around), so we should migrate our code to setuptools anyway. Correct me if I'm wrong, but this means the StackOverflow answer I linked above is also misguided and should be changed from the top into the bottom?:

import setuptools
from distutils.core import setup

from setuptools import setup

Are you using setuptools to build your project or are you using setuptools inside your project (or both)?

The project is a utility for deploying apps on our org's local server, so distutils comes in handy there. The project used distutils only and did not import setuptools.

As I tried to explain, the exe threw an error that 'setuptools' could not be imported, specifically when the line distutils.core.run_setup(setup_py_file, stop_after="init") is executed. So my hypothesis at this point was that distutils requires and imports setuptools somehow, but in a hacky way that is undetectable to PyInstaller, hence why it is missing in the exe. Now during reproduction I actually realized this was a misunderstanding and that the error occurs because the setup_py_file imports setuptools directly.

So my next step was explicitly adding the line "import setuptools" into our project to ensure that PyInstaller will bundle setuptools==60.5.0 inside of the exe. That lead to the described error inside ensure_local_distutils.

So in summary the project/exe builds other projects that import setuptools in their setup.py. Hope this was helpful.

Can you put together a more complete reproducer, preferably a minimum one that encounters the error?

Will do, give me some time.

[xjcl]

I was able to repro this on 3.7 - 3.10 and as you suggested this issue was introduced in setuptools==60.0.0

Below is a PowerShell script which downloads and installs Python, then installs setuptools and pyinstaller, and then creates an exe out of a file which just imports the two packages.

# Run me as administrator
#     https://stackoverflow.com/q/5944180/2111778
#     powershell -ExecutionPolicy Bypass .\repro_setuptools_60_bug.ps1  

$pythonVersion = @("3.7.7", "3.8.10", "3.9.10", "3.10.2")[3]
$pythonUrl = "https://www.python.org/ftp/python/$pythonVersion/python-$pythonVersion.exe"

$pythonInstallDirDir = "C:\Temp\Test1"
$pythonDownloadPath = "$pythonInstallDirDir\python-installer-$pythonVersion.exe"
$pythonInstallDir = "$pythonInstallDirDir\Python$pythonVersion"

if (Test-Path $pythonInstallDirDir) {
    Remove-Item -Recurse -Force $pythonInstallDirDir
    sleep 10  # removals are not instant on Windows
}

mkdir $pythonInstallDirDir
cd $pythonInstallDirDir

(New-Object Net.WebClient).DownloadFile($pythonUrl, $pythonDownloadPath)

# For some reason the installer exists right away so sleep until the install appears done
& $pythonDownloadPath /quiet InstallAllUsers=0 Include_test=0 DefaultJustForMeTargetDir=$pythonInstallDir
echo "Python installer exited"
while (-Not (Test-Path "$pythonInstallDir\Scripts\pip.exe")) {
    sleep 1
}
echo "Pip install done. Waiting 20 seconds for Python install to finish"
sleep 20

$pythonExe = "$pythonInstallDir\python.exe"
& $pythonExe -m pip install setuptools==60.0.0 pyinstaller==4.8

mkdir "$pythonInstallDirDir\project1"
Set-Content "$pythonInstallDirDir\project1\repro_exe.py" "import setuptools; import distutils"
& "$pythonInstallDir\Scripts\pyinstaller.exe" --onefile "$pythonInstallDirDir\project1\repro_exe.py"

# This line reproduces the error (AssertionError inside ensure_local_distutils)
& .\dist\repro_exe.exe

& $pythonDownloadPath /quiet /uninstall
sleep 40

Results:

# Repro table:
#           setuptools==59.8.0  setuptools==60.0.0  setuptools==60.8.1 (latest)
# 3.7.7     Issue absent        Issue present       Issue present
# 3.8.10    Issue absent        Issue present       Issue present
# 3.9.10    Issue absent        Issue present       Issue present
# 3.10.2    Issue absent        Issue present       Issue present

It might be worth trying different versions of PyInstaller. I know it extracts packages in some sort of Temp folder (as you can see in the original traceback), and the reason an AssertionError is thrown is that there is an assert that "_distutils" is somewhere in the path (which it isn't, the underscore is missing)

[xjcl]

On second thought my PowerShell script might overcomplicate things, plus the Python installer fails with exit code 0x666 if there is already a newer version on the system. So here a short manual guide assuming a pre-existing Python install.

1. Install via pip

setuptools==60.0.0 pyinstaller==4.8

2. Create file repro.py with content

import setuptools; import distutils

3. Run (the pyinstaller.exe should be in the Scripts directory of your Python install)

pyinstaller.exe --onefile repro.py

This should produce a file .\dist\repro.exe relative to the current working directory.

4. Run the file, this produces the AssertionError inside ensure_local_distutils on the command line

.\dist\repro.exe

  File "_distutils_hack\__init__.py", line 59, in ensure_local_distutils
AssertionError: C:\Users\<omitted>\AppData\Local\Temp\_MEI294562\distutils\core.pyc 

---

2a) I just found this. Note that the file

import setuptools

produces a different traceback:

  File "_distutils_hack\__init__.py", line 92, in create_module                                      
  File "importlib\__init__.py", line 126, in import_module                                        
ModuleNotFoundError: No module named 'setuptools._distutils' 

2b) Note that the file

import distutils; import setuptools

produces the same error as 2a but with an additional warning:

_distutils_hack\__init__.py:23: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.    

[xjcl]

I tried to look at this myself a bit and it appears the expected file location is

<venv_dir>\python\Lib\site-packages\setuptools\_distutils\core.py

i.e. setuptools replaces distutils with its own bundled version. While the actual value we get in the exe is the original distutils

C:\Users\<omitted>\AppData\Local\Temp\_MEI294562\distutils\core.pyc 

I tried to follow the call chain unsuccessfully

ensure_local_distutils in <venv_dir>\Lib\site-packages\_distutils_hack\__init__.py
    import_module in <host_python_dir>\Lib\importlib\__init__.py
        _gcd_import in <host_python_dir>\Lib\importlib\_bootstrap.py
            _find_and_load in <host_python_dir>\Lib\importlib\_bootstrap.py

but sadly the trace runs dry there, in fact my print statements stop working inside of _boostrap.py. I assume by the name there is some dark magic going on, presumably _bootstrap.py is overwritten with an import from _frozen_importlib which PyCharm cannot step into :/ (BTW, by "host Python" I mean the original Python install used to create the venv.)

[xjcl]

I was able to do some additional debugging by replacing

import _frozen_importlib as _bootstrap

with

from . import _bootstrap
__bootstrap._setup(sys, _imp)

inside of importlib/__init__.py

---

Findings:

In Python, sys.meta_path is a list of importer objects (BuiltinImporter, FrozenImporter, SourceFileLoader, etc.) which are walked in order to find a module.

• _distutils_hack defines a DistutilsMetaFinder and appends it to the front of this list to ensure setuptools._distutils is loaded in place of distutils
• PyInstaller defines its own pyimod03_importers.FrozenImporter which is appended near the back of this list. This is what ends up being used, I assume the earlier importer get skipped because the source code files are not available to the exe. This is why a bytecode (.pyc) file appears in the assert.

So a potential fix would extend DistutilsMetaFinder to also be able to find the relevant .pyc file.

[xjcl]

I'm noticing another issue. The _distutils_hack.DistutilsMetaFinder is actually at the front of the list TWICE in a normal run and ONCE during an exe run. Also any print statements I add inside of _distutils_hack\__init__.py do not get executed in the exe, only the normal run.

So I am perplexed how the _distutils_hack.DistutilsMetaFinder ends up at the front of the sys.meta_path.

I guess it could be a leftover from the exe build process? But then why don't my changes make it?

[FFY00]

Are you sure the code isn't being loaded from a cache? What is the value of _distutils_hack.__spec__.origin?

[xjcl]

Are you sure the code isn't being loaded from a cache? What is the value of _distutils_hack.__spec__.origin?

The value is C:\Users\<omitted>\AppData\Local\Temp\_MEI57922\_distutils_hack\__init__.pyc. The problem with PyInstaller is you get these temp files and don't know where they are from. I already tried deleting _distutils_hack\__pycache__\*.pyc files earlier with no success.

[xjcl]

I printed a trace now based on print statements.

TL;DR when importing distutils, it hits the DistutilsMetaFinder and tries to load setuptools._distutils instead (mod = importlib.import_module('setuptools._distutils')).

Loading said module fails in the exe. In vanilla Python this module is found by the _frozen_importlib_external.PathFinder.

[xjcl]

There is a function _find_spec inside of _bootstrap.py. And this is called with a path argument, which is an array of str. In vanilla Python this is

['<venv_dir>\Lib\site-packages\setuptools']

as expected. Inside the exe the path also seems correct

['C:\Users\<omitted>\AppData\Local\Temp\_MEI<numbers>\setuptools']

The only problem is that the directory does not exist by the time _find_spec is entered.

Which makes no sense to me because I thought our call stack looked like this

main.py
    setuptools\__init__.py
        _distutils_hack\__init__.py

so the setuptools directory should definitely exist at the time that the _distutils_hack occurs. This is also the case in vanilla Python, but inside the exe I cannot verify this as my print statements inside of setuptools\__init__.py do not have any effect.

Maybe this is a PyInstaller issue after all where it has access to multiple versions of setuptools or something?

[xjcl]

Yeah I'm giving up on this now. Nothing here makes sense to me.

[dennisvang]

Reproduced this on Windows 10 in a fresh conda environment with the following:

• python 3.8.13
• setuptools 62.6.0
• pyinstaller 5.1

Traceback (most recent call last):
  File "repro.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "setuptools\__init__.py", line 8, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "_distutils_hack\override.py", line 1, in <module>
  File "_distutils_hack\__init__.py", line 77, in do_override
  File "_distutils_hack\__init__.py", line 64, in ensure_local_distutils
AssertionError: C:\Users\me\AppData\Local\Temp\_MEI124442\distutils\core.pyc
[10700] Failed to execute script 'repro' due to unhandled exception!

In my case, step 2a (i.e. only import setuptools, no import distutils) yields the same AssertionError with the same traceback.

[xjcl]

Thanks a lot for noticing my report and implementing a workaround in PyInstaller>=5.4.0 @dennisvang! 😄

I no longer rely on setuptools inside of my exe, but it will surely be useful for someone else.

I am not sure if this issue could have other effects (maybe in py2exe?) so I'm waiting for the setuptools team to decide if this issue can be closed. Thanks for your help everybody!

[jaraco]

I regret having lost track of this issue. I appreciate all the work that went into the investigation and for the PyInstaller maintainers to provide support for this hack. I don't believe there's any more that's needed at this stage. If py2exe needs guidance, feel free to ping this issue or open a new one (and mention me).