Upgrading from 23.0.1 to 23.1 causes 'User site-packages are not visible in this virtualenv'

[abramhemphill]

Description

With pip 23.0.1 this worked

python -m venv .venv
.venv\Scripts\python -m pip install --upgrade pip
.venv\Scripts\activate
(.venv) > pip install -r requirements.txt

With pip 23.1

python -m venv .venv
.venv\Scripts\python -m pip install --upgrade pip
.venv\Scripts\activate
(.venv) > pip install -r requirements.txt
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

We've never had site-packages visible, something else has changed. The pyvenv.cfg has not changed.

Expected behavior

With pip 23.0.1 this worked. I reverted pip this morning and it still works with 23.0.1

python -m venv .venv
.venv\Scripts\python -m pip install --upgrade pip
.venv\Scripts\activate
(.venv) > pip install -r requirements.txt
...success

pip version

23.1

Python version

3.9.13

OS

win11

How to Reproduce

python -m venv .venv
.venv\Scripts\python -m pip install --upgrade pip
.venv\Scripts\activate
(.venv) > pip install -r requirements.txt

This should work. Heck any pip install should work and not show the --user error.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

Why is pip trying to do a --user install? Is there some configuration you're not showing here? Also, can you show your requirements.txt (and ideally trim it down to a minimal reproducer)?

[abramhemphill]

it's not just -r requirements.txt. -e . on a dev repo is the same issue. Really any pip install command give the same error message. 23.0.1 is fine though.

[uranusjr]

Can you run pip config list?

[luciansmith]

This seems similar to the same problem we ran into today with 23.1: installing things using github actions now fails. Weirdly, it claims that the pip module doesn't exist. From https://github.com/biosimulators/Biosimulators/actions/runs/4726006706/jobs/8385242346 :

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [28 lines of output]
      /opt/hostedtoolcache/Python/3.8.16/x64/bin/python: No module named pip

EDIT: probably a simpler example: https://github.com/biosimulations/biomodels-qc/actions/runs/4641088733/jobs/8376894388

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [28 lines of output]
      /opt/hostedtoolcache/Python/3.9.16/x64/bin/python: No module named pip

[abramhemphill]

[pfmoore]

I get exactly the same error with 23.0.1 and 23.1. And it's what I expect - you cannot use --user in a virtual environment that doesn't have --system-site-packages set, as the message says. The problem here is that somehow you're not getting that error in 23.0.1. You'll need to do some more investigation to work out why - there's nothing in what you've shown us yet that explains that behaviour.

But the behaviour you are seeing in 23.1 is correct, and is not a bug.

@luciansmith the behaviour you describe is different. If you can produce a reproducible example, please raise a separate issue for it.

[abramhemphill]

23.0.1
(.venv) C:\sanbox\1src\idf\abi3_2\package>pip config list
:env:.extra-index-url='https://url/simple'

(.venv) C:\sanbox\1src\idf\abi3_2\package>pip list
Package Version

---

pip 22.0.4
setuptools 58.1.0
WARNING: You are using pip version 22.0.4; however, version 23.1 is available.
You should consider upgrading via the 'C:\sanbox\1src\idf\abi3_2\package.venv\Scripts\python.exe -m pip install --upgrade pip' command.

(.venv) C:\sanbox\1src\idf\abi3_2\package>python -m pip install --upgrade pip==23.0.1
Looking in indexes: https://pypi.org/simple, https://url/simple
Collecting pip==23.0.1
Using cached https://url/pip/pip-23.0.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.0.4
Uninstalling pip-22.0.4:
Successfully uninstalled pip-22.0.4
Successfully installed pip-23.0.1

(.venv) C:\sanbox\1src\idf\abi3_2\package>pip config list
:env:.extra-index-url='https://url/simple'

(.venv) C:\sanbox\1src\idf\abi3_2\package>pip install pytest
Looking in indexes: https://pypi.org/simple, https://url/simple
Collecting pytest
Downloading pytest-7.3.1-py3-none-any.whl (320 kB)
---------------------------------------- 320.5/320.5 kB 2.5 MB/s eta 0:00:00
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pluggy<2.0,>=0.12
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting iniconfig
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)

Vs 23.1
C:\sanbox\1src\idf\abi3_2\package>python --version
Python 3.9.13

C:\sanbox\1src\idf\abi3_2\package>pip list
Package Version

---

pip 23.1

C:\sanbox\1src\idf\abi3_2\package>python -m pip list
Package Version

---

pip 23.1

C:\sanbox\1src\idf\abi3_2\package>pip config list
:env:.extra-index-url='https://url/simple'
freeze.user='yes'
install.user='yes'
list.user='yes'
uninstall.user='yes'

C:\sanbox\1src\idf\abi3_2\package>python -m venv .venv

C:\sanbox\1src\idf\abi3_2\package>.venv\Scripts\activate

(.venv) C:\sanbox\1src\idf\abi3_2\package>pip list
Package Version

---

pip 22.0.4
setuptools 58.1.0
WARNING: You are using pip version 22.0.4; however, version 23.1 is available.
You should consider upgrading via the 'C:\sanbox\1src\idf\abi3_2\package.venv\Scripts\python.exe -m pip install --upgrade pip' command.

(.venv) C:\sanbox\1src\idf\abi3_2\package>python -m pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://url/simple
Requirement already satisfied: pip in c:\sanbox\1src\idf\abi3_2\package.venv\lib\site-packages (22.0.4)
Collecting pip
Using cached pip-23.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.0.4
Uninstalling pip-22.0.4:
Successfully uninstalled pip-22.0.4
Successfully installed pip-23.1

(.venv) C:\sanbox\1src\idf\abi3_2\package>pip install pytest
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

(.venv) C:\sanbox\1src\idf\abi3_2\package>python -m pip install --upgrade pip==23.0.1
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

(.venv) C:\sanbox\1src\idf\abi3_2\package>pip config list
:env:.extra-index-url='https://url/simple'
freeze.user='yes'
install.user='yes'
list.user='yes'
uninstall.user='yes'

(.venv) C:\sanbox\1src\idf\abi3_2\package>deactivate

[pfmoore]

So in 23.1 you have a pip configuration setting --user globally, but in 23.0.1 you do not. That would explain the behaviour you are seeing.

Thanks for the confirmation.

[uranusjr]

Also note that you can use pip config debug to find where the configs originate from and modify them as appropriate.

[pfmoore]

Note that this may actually be #11982 - the pip config list in the failing case looks suspiciously like the config added by the Windows Store installation of Python.