TypeError: bad operand type for unary -: 'NoneType'

[alex]

Not quite sure what provokes this error, I've seen it a few times when pip is invoked under tox:

py33 installdeps: flake8, mock, sqlalchemy, Flask
ERROR: invocation failed, logfile: /Users/alex_gaynor/projects/project/.tox/py33/log/py33-1.log
ERROR: actionid=py33
msg=getenv
cmdargs=[local('/Users/alex_gaynor/projects/project/.tox/py33/bin/pip'), 'install', '--pre', 'flake8', 'mock', 'sqlalchemy', 'Flask']
env={'PYTHONIOENCODING': 'utf_8', 'VERSIONER_PYTHON_PREFER_32_BIT': 'no', 'TMPDIR': '/var/folders/fm/57qdjdrx6gzgcq1zlym0xwfc0000gn/T/', 'LOGNAME': 'alex_gaynor', 'USER': 'alex_gaynor', 'HOME': '/Users/alex_gaynor', 'PATH': '/Users/alex_gaynor/projects/project/.tox/py33/bin:/Users/alex_gaynor/.virtualenvs/c8ee7ddf8807579c/bin:/usr/local/heroku/bin:/Users/alex_gaynor/.rbenv/shims:/Users/alex_gaynor/Library/Python/2.7/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin', 'PS1': '(c8ee7ddf8807579c)\\h:\\W \\u\\$ ', 'DISPLAY': '/tmp/launch-EyMZ56/org.macosforge.xquartz:0', 'TERM_PROGRAM': 'iTerm.app', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm-256color', 'Apple_PubSub_Socket_Render': '/tmp/launch-Ds4n39/Render', 'COLORFGBG': '7;0', 'VERSIONER_PYTHON_VERSION': '2.7', 'SHLVL': '1', 'SECURITYSESSIONID': '186a6', 'ITERM_SESSION_ID': 'w0t4p0', 'EDITOR': 'subl -w', 'HISTFILESIZE': '100000000000', 'WORKON_HOME': '/Users/alex_gaynor/.virtualenvs', 'SSH_AUTH_SOCK': '/tmp/launch-f04gW3/Listeners', 'VIRTUAL_ENV': '/Users/alex_gaynor/projects/project/.tox/py33', 'SHELL': '/bin/bash', 'ITERM_PROFILE': 'Default', '_': '/Users/alex_gaynor/.virtualenvs/c8ee7ddf8807579c/bin/tox', 'VIRTUALENVWRAPPER_PROJECT_FILENAME': '.project', 'VIRTUALENVWRAPPER_HOOK_DIR': '/Users/alex_gaynor/.virtualenvs', 'OLDPWD': '/Users/alex_gaynor/.virtualenvs/c8ee7ddf8807579c', '__CF_USER_TEXT_ENCODING': '0x1F5:0:0', 'Apple_Ubiquity_Message': '/tmp/launch-7G7rbX/Apple_Ubiquity_Message', 'PWD': '/Users/alex_gaynor/projects/project', 'COMMAND_MODE': 'unix2003'}
Downloading/unpacking flake8
Downloading/unpacking mock
  Using download cache from /Users/alex_gaynor/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fm%2Fmock%2Fmock-1.0.1.tar.gz
  Running setup.py egg_info for package mock

    warning: no files found matching '*.png' under directory 'docs'
    warning: no files found matching '*.css' under directory 'docs'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no files found matching '*.js' under directory 'docs'
Downloading/unpacking sqlalchemy
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/index.py", line 287, in find_requirement
    applicable_versions = self._sort_versions(applicable_versions)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/index.py", line 180, in _sort_versions
    return sorted(applicable_versions, key=self._link_sort_key, reverse=True)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/index.py", line 167, in _link_sort_key
    pri = -(link.wheel.support_index_min())
TypeError: bad operand type for unary -: 'NoneType'

[alex]

I've now got this to reproduce reliably:

Alexanders-MacBook-Pro:pretend alex_gaynor$ .tox/py26/bin/pip install --pre pytest
Downloading/unpacking pytest
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/index.py", line 287, in find_requirement
    applicable_versions = self._sort_versions(applicable_versions)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/index.py", line 180, in _sort_versions
    return sorted(applicable_versions, key=self._link_sort_key, reverse=True)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/index.py", line 167, in _link_sort_key
    pri = -(link.wheel.support_index_min())
TypeError: bad operand type for unary -: 'NoneType'

Storing complete log in /Users/alex_gaynor/.pip/pip.log

I think this is somehow related to the fact that I put a bunch of stuff in my wheelhouse for py27?

[dstufft]

Looks like this stems from this line:

pip/pip/wheel.py

Line 269 in 9b4b651

return min(indexes) if indexes else None

You can't do -None.

[alex]

So I believe the set of circumstances this occurs in is: if you have something in your wheelhouse, but that item is not universal, and is for some other python version

[qwcode]

the assumption of this section of the sorting code, is that the wheel links it's sorting are supported, so wheel.support_index_min() won't be None. the comment above this line says "all wheel links are known to be supported at this stage" but obviously that's not correct. so looking.....

[qwcode]

about 'wheelhouse', it's the default build location for pip wheel. It won't be used automatically during installs, unless --find-links or PIP_FIND_LINKS is pointing at it, but I don't see that in your examples?

[alex]

Yup, here is my pip.conf:

[global]
timeout = 15
download_cache = ~/.pip/cache
allow-all-external = false
no-allow-insecure = false
use-wheel = true
wheel-dir = /Users/alex_gaynor/.pip/wheels
find-links = /Users/alex_gaynor/.pip/wheels

[qwcode]

ok, you mind listing your wheels dir for me? the names will help.

[alex]

It's pretty huge so I put it in a paste: http://bpaste.net/show/136231/

[qwcode]

As it turns out, the root issue here is general.
Our link matching code thinks any pytest-xdist archive url fulfills the requirement pytest.
When not using --pre, the version (determined as xdist-X.Y) gets skipped as a pre-release.
When using --pre (like in the example above in tox), it gets passed on as a valid url.

In the case of wheels, there's some additional sorting that operates on the filename, which is not expecting unsupported archives, for the wrong project to be passed in.

e.g. download https://pypi.python.org/packages/source/p/pytest-xdist/pytest-xdist-1.8.zip into a local archives folder and do this:

pip install --no-deps --no-index --pre --find-links=archives pytest

it will happily install pytest-xdist for you.

Our matching routine right now tries to get away with not determining the project name, and matching the whole filename with <requirement>-. This shortcoming has come up before in other issues (#804 (comment))

I think the solution is to change this routine so that it matches against the project name. distlib has a function for determining the project name (pip.vendor.distlib.util.split_filename)

[qwcode]

I had a PR with tests ready to go, but there's a snag with using pip.vendor.distlib.util.split_filename. it doesn't parse when 'dev' is the version, but pip supports locating and installing a 'dev' version. So, I'm inclined to copy the split_filename function up into pip and modify it's regexp (and submit an issue to distlib about it, unless vinay could fix this quickly) cc @vsajip

[dstufft]

I temporarily switched PyPI to using the split_filename function and it broke in a decent number of conditions, I ended up pulling it out immediately.

[qwcode]

what does PyPI have now?

[dstufft]

Something that pip can't do; I looked it up by filename in the database

[qwcode]

donald, do you mind posting the ones that fail.

for the sake of 1.4.2, the conservative thing I think it just to fix this for wheel archives, and leave fixing sdist parsing to 1.5, which will provide beta testing, which a major change like this really needs.

I'll post a PR probably tomorrow.

[dstufft]

Um, I think django-altuser-v0.6.8.tar.gz was one of them, I believe youtube_dl_server-alpha.1.tar.gz was another. I don't remember the others.