Alpine: ERROR: Cannot uninstall 'distlib' 0.3.0

[hugovk]

Issue

On Alpine, installation of newest virtualenv 20.0.28 fails:

Collecting virtualenv
  Downloading virtualenv-20.0.28-py2.py3-none-any.whl (4.9 MB)
Requirement already satisfied: appdirs<2,>=1.4.3 in /usr/lib/python3.8/site-packages (from virtualenv) (1.4.4)
Requirement already satisfied: six<2,>=1.9.0 in /usr/lib/python3.8/site-packages (from virtualenv) (1.15.0)
Collecting filelock<4,>=3.0.0
  Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting distlib<1,>=0.3.1
  Downloading distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Installing collected packages: filelock, distlib, virtualenv
  Attempting uninstall: distlib
    Found existing installation: distlib 0.3.0
ERROR: Cannot uninstall 'distlib'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

PR #1880 bumped the distutils dependency:

-distlib>=0.3.0,<1
+distlib>=0.3.1,<1

Which was released in virtualenv 20.0.26. Pinning to virtualenv < 20.0.26 installs correctly.

More info

This happens with this Dockerfile:

# This is a sample Dockerfile to build Pillow on Alpine Linux
# with all/most of the dependencies working.
#
# Tcl/Tk isn't detecting
# Freetype has different metrics so tests are failing.
# sudo and bash are required for the webp build script.

FROM alpine

RUN apk --no-cache add python3 \
                       build-base \
                       python3-dev \
                       # dev dependencies
                       git \
                       bash \
                       sudo \
                       py3-pip

RUN /usr/sbin/adduser -D pillow && \
    python3 -c "import fcntl; fcntl.fcntl(1, fcntl.F_SETFL, 0)" && \
    pip3 list && \
    pip3 install virtualenv && virtualenv /vpy3 -vvv --with-traceback && \
    /vpy3/bin/pip install --upgrade pip && \
    /vpy3/bin/pip install olefile pytest pytest-cov && \
    /vpy3/bin/pip install numpy --only-binary=:all: || true && \
    chown -R pillow:pillow /vpy3

USER pillow
CMD ["depends/test.sh"]

#docker run -v $TRAVIS_BUILD_DIR:/Pillow pythonpillow/alpine

Failing build: https://travis-ci.org/github/hugovk/docker-images/builds/711483204

Pinning virtualenv < 20.0.26 installs and creates the virtualenv fine (ignore the later test failures, due to a stripped down Dockerfile):

https://travis-ci.org/github/hugovk/docker-images/builds/711485321#L365

Environment

Provide at least:

• OS: Alpine
• Python 3.6.7
• pip 20.1.1
• pip list of the host python where virtualenv is installed:

Package      Version
------------ ----------
appdirs      1.4.4
CacheControl 0.12.6
certifi      2020.4.5.1
chardet      3.0.4
colorama     0.4.3
contextlib2  0.6.0
distlib      0.3.0
distro       1.5.0
html5lib     1.0.1
idna         2.9
lockfile     0.12.2
msgpack      1.0.0
ordered-set  4.0.1
packaging    20.4
pep517       0.8.2
pip          20.1.1
progress     1.5
pyparsing    2.4.7
pytoml       0.1.21
requests     2.23.0
retrying     1.3.3
setuptools   47.0.0
six          1.15.0
toml         0.10.1
urllib3      1.25.9
webencodings 0.5.1

Output of the virtual environment creation

n/a, fails during pip3 install virtualenv

[hugovk]

This is NOT happening with our other Dockerfiles:

• amazon-1-amd64
• amazon-2-amd64
• arch
• archive
• centos-6-amd64
• centos-7-amd64
• centos-8-amd64
• debian-10-buster-x86
• debian-9-stretch-x86
• fedora-31-amd64
• fedora-32-amd64
• ubuntu-16.04-xenial-amd64
• ubuntu-18.04-bionic-amd64
• ubuntu-20.04-focal-amd64

[pfmoore]

Package      Version
------------ ----------
appdirs      1.4.4
CacheControl 0.12.6
certifi      2020.4.5.1
chardet      3.0.4
colorama     0.4.3
contextlib2  0.6.0
distlib      0.3.0

How was that copy of distlib installed? It's not obvious (to me, at least) from the dockerfile you post.

I wonder if py3-pip installs an OS-managed, unbundled copy of pip with distlib installed as a separate library, rather than being vendored in pip?

[gaborbernat]

This is NOT happening with our other Dockerfiles:

This well could be but the issue is OSs fault not ours. Distutils is already installed on the operating system before you invoke the pip install command. An old version of it. However, pip not instaleld it, it's either the OS or easy_install installed, and as such pip refuses to remove it and just fails. The solution would be to uninstall the already existing distutils either via easy_install or OS before you invoke the install command, or install virtualenv into a virtual environment, or install it at --user level, or use virtualenv as zipapp.

[tdensmore]

@hugovk Did you ever solve this issue? I am running into the same thing, and my search for a solution led me here.

[hugovk]

If I remember, it was to do with setuptools and the ongoing work to deprecate distutils. We did two things: upgrade to setuptools>=49.3.2, and replaced/removed all use of distutils.