Improve error message for python 3.5 EOL experience

[nipunn1313]

Hiya - we ran into this debugging challenge a couple of times at internally Dropbox, as well as on the mypy CI (see python/mypy@e1b34b5)

I want to acknowledge python3.5 is EOL and we want to minimize having to deal with it. However, currently, we have several systems still on ubuntu 16.04 - and it's challenging to get off it - we're working on it, but wanted to file this as I'm optimistic there's a low cost workaround. It took many eng hours to isolate the problem to this issue =\

python3.5 ships out with pip==8 - and pip dropped support for py3.5 in pip==21 (which launched Jan 2021)

pip IS intelligent enough such that when you run pip install --upgrade pip from a python3.5 installation - it will not upgrade to pip==21, due to metadata from python_requires (https://www.python.org/dev/peps/pep-0345/)

However, pip learned this python-version-metadata logic in pip==9, so the pip that ships with py3.5 will happily update itself to pip==21 and irreversibly break (since pip 21 uses fstrings)

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.5/dist-packages/pip/__main__.py", line 21, in <module>
    from pip._internal.cli.main import main as _main
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}")
                                   ^
SyntaxError: invalid syntax

The workaround is to either pin pip==20

pip install --upgrade pip==20

OR first upgrade to smarter pip (anything between 9-20) and then upgrade

pip install --upgrade pip==9  # learn python_requires metadata
pip install --upgrade pip  # then upgrade

The former workaround is the more-obvious solution and I suspect many people are doing it now. Esp if it's on common code that runs across multiple python versions (like the mypy CI linked above) - this is probably bad.
The latter is harder to figure out.

I can think of a couple ideas to solve this

1. Ship a new python3.5 w/ pip >= 9 (seems unlikely since python3.5 is EOL)
2. Check for python3.5 at the top of pip - and recommend that folks the latter solution (and/or get off python 3.5)
3. Somehow patch the pip==8 to know about metadata? I'm not sure if this will work since python3.5 may still ship w/ the older version.
   I don't know pip very well, so I'm of course totally open to other solutions as well.

[nipunn1313]

Potentially another idea - might be to have pypi refuse to return pip==21 to python3.5 clients who are asking. I'm not sure whether pypi actually listens/hears the python version - though that solution seems too complex.

Given that python3.5 is EOL, I'm mostly in favor of a better error message that suggests getting off python3.5 or pinning pip==20

[pfmoore]

I guess we could add an interpreter version check to pip/__main__.py and fail early with a more friendly message if the user is on an unsupported Python version.

The problem is that by the time the user sees that version, it's mostly too late - they already have the wrong pip version installed, and it won't work (so they can't use it to uninstall/downgrade pip).

[uranusjr]

Fortunately Python 3.5 has ensurepip so we can suggest something like

{sys.executable} -m ensurepip
{sys.executable} -m pip install "pip<21"

I would welcome a PR adding an import-time check to either __main__.py or __init__.py.

[pradyunsg]

This has been fixed by pypa/get-pip#85; and other users are recommended to first update to a version of pip that shipped with their interpreter.

[nipunn1313]

Hi @pradyunsg - I took a look at the diff and it's nonobvious how it fixes the issue. Could you help clarify?

The issue arises here because when a user uses the recommended instructions to self-update pip pip install --upgrade pip from python3.5 - it updates to a version of pip that is broken. How does the change to get-pip solve this?

It's not clear to me how a developer stuck on python3.5 (eg stuck on ubuntu 16.04) would see an improved error message from this change to get-pip. I may be missing something about how it works.

[pradyunsg]

We don't control the version of pip that Ubuntu 16.04 ships with. That is so old that the enhancements we've made for improving experience around "minimum Python version required" are not included in it. We can't provide the up-to-date experience to users using really old versions of pip. :)

Overall, the degraded experience is fixed in the current version of pip and across all of our supported distribution mechanisms (ensurepip will just-work, and get-pip.py got updated in that PR). I don't think there's anything actionable for pip here TBH.

[nipunn1313]

I believe the suggestion in the above post was to action it w/ a check early in main for python version - w/ a helpful error message.

To be clear, not trying to say that we should continue to support python3.5 - but rather - try to dodge the situation where folks who are stuck w/ some environments with python3.5 - pinning their pip to 20 (eg mypy CI).
May be too late to solve that anyway.

It doesn't sound like pypa/get-pip#85 fixed the issue, but it does sound like you're maybe recommending not actioning this task?

[uranusjr]

There’s really no good way to show a human-friendly error message in this situation though. The syntax error happens on import time, so pip does not even have a chance to execute anything before it happens. The only way to achieve what you want is for pip to avoid using any new syntax features, which I hope is understanably a lot to ask from maintainers. The get-pip.py solution is the best pip can do.

[nipunn1313]

Yeah - if it were to suggest a better error message - it would have to be some kind of check at the top of __main__.py above the imports in latest pip.

I suspect folks stuck in this situation are likely just putting pip == 20 in their requirements, but perhaps it's ok to just let those folks do that and not try to optimize for their quality of life.