TypeError: bad operand type for unary -: 'NoneType' #1192

alex · 2013-09-07T15:42:41Z

Not quite sure what provokes this error, I've seen it a few times when pip is invoked under tox:

py33 installdeps: flake8, mock, sqlalchemy, Flask
ERROR: invocation failed, logfile: /Users/alex_gaynor/projects/project/.tox/py33/log/py33-1.log
ERROR: actionid=py33
msg=getenv
cmdargs=[local('/Users/alex_gaynor/projects/project/.tox/py33/bin/pip'), 'install', '--pre', 'flake8', 'mock', 'sqlalchemy', 'Flask']
env={'PYTHONIOENCODING': 'utf_8', 'VERSIONER_PYTHON_PREFER_32_BIT': 'no', 'TMPDIR': '/var/folders/fm/57qdjdrx6gzgcq1zlym0xwfc0000gn/T/', 'LOGNAME': 'alex_gaynor', 'USER': 'alex_gaynor', 'HOME': '/Users/alex_gaynor', 'PATH': '/Users/alex_gaynor/projects/project/.tox/py33/bin:/Users/alex_gaynor/.virtualenvs/c8ee7ddf8807579c/bin:/usr/local/heroku/bin:/Users/alex_gaynor/.rbenv/shims:/Users/alex_gaynor/Library/Python/2.7/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin', 'PS1': '(c8ee7ddf8807579c)\\h:\\W \\u\\$ ', 'DISPLAY': '/tmp/launch-EyMZ56/org.macosforge.xquartz:0', 'TERM_PROGRAM': 'iTerm.app', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm-256color', 'Apple_PubSub_Socket_Render': '/tmp/launch-Ds4n39/Render', 'COLORFGBG': '7;0', 'VERSIONER_PYTHON_VERSION': '2.7', 'SHLVL': '1', 'SECURITYSESSIONID': '186a6', 'ITERM_SESSION_ID': 'w0t4p0', 'EDITOR': 'subl -w', 'HISTFILESIZE': '100000000000', 'WORKON_HOME': '/Users/alex_gaynor/.virtualenvs', 'SSH_AUTH_SOCK': '/tmp/launch-f04gW3/Listeners', 'VIRTUAL_ENV': '/Users/alex_gaynor/projects/project/.tox/py33', 'SHELL': '/bin/bash', 'ITERM_PROFILE': 'Default', '_': '/Users/alex_gaynor/.virtualenvs/c8ee7ddf8807579c/bin/tox', 'VIRTUALENVWRAPPER_PROJECT_FILENAME': '.project', 'VIRTUALENVWRAPPER_HOOK_DIR': '/Users/alex_gaynor/.virtualenvs', 'OLDPWD': '/Users/alex_gaynor/.virtualenvs/c8ee7ddf8807579c', '__CF_USER_TEXT_ENCODING': '0x1F5:0:0', 'Apple_Ubiquity_Message': '/tmp/launch-7G7rbX/Apple_Ubiquity_Message', 'PWD': '/Users/alex_gaynor/projects/project', 'COMMAND_MODE': 'unix2003'}
Downloading/unpacking flake8
Downloading/unpacking mock
  Using download cache from /Users/alex_gaynor/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fm%2Fmock%2Fmock-1.0.1.tar.gz
  Running setup.py egg_info for package mock

    warning: no files found matching '*.png' under directory 'docs'
    warning: no files found matching '*.css' under directory 'docs'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no files found matching '*.js' under directory 'docs'
Downloading/unpacking sqlalchemy
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/index.py", line 287, in find_requirement
    applicable_versions = self._sort_versions(applicable_versions)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/index.py", line 180, in _sort_versions
    return sorted(applicable_versions, key=self._link_sort_key, reverse=True)
  File "/Users/alex_gaynor/projects/project/.tox/py33/lib/python3.3/site-packages/pip/index.py", line 167, in _link_sort_key
    pri = -(link.wheel.support_index_min())
TypeError: bad operand type for unary -: 'NoneType'

The text was updated successfully, but these errors were encountered:

alex · 2013-09-08T17:58:18Z

I've now got this to reproduce reliably:

Alexanders-MacBook-Pro:pretend alex_gaynor$ .tox/py26/bin/pip install --pre pytest
Downloading/unpacking pytest
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/index.py", line 287, in find_requirement
    applicable_versions = self._sort_versions(applicable_versions)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/index.py", line 180, in _sort_versions
    return sorted(applicable_versions, key=self._link_sort_key, reverse=True)
  File "/Users/alex_gaynor/projects/pretend/.tox/py26/lib/python2.6/site-packages/pip/index.py", line 167, in _link_sort_key
    pri = -(link.wheel.support_index_min())
TypeError: bad operand type for unary -: 'NoneType'

Storing complete log in /Users/alex_gaynor/.pip/pip.log

I think this is somehow related to the fact that I put a bunch of stuff in my wheelhouse for py27?

dstufft · 2013-09-08T18:04:45Z

Looks like this stems from this line:

pip/pip/wheel.py

Line 269 in 9b4b651

return min(indexes) if indexes else None

You can't do -None.

alex · 2013-09-26T17:35:09Z

So I believe the set of circumstances this occurs in is: if you have something in your wheelhouse, but that item is not universal, and is for some other python version

qwcode · 2013-09-28T18:23:05Z

the assumption of this section of the sorting code, is that the wheel links it's sorting are supported, so wheel.support_index_min() won't be None. the comment above this line says "all wheel links are known to be supported at this stage" but obviously that's not correct. so looking.....

qwcode · 2013-09-28T18:46:09Z

about 'wheelhouse', it's the default build location for pip wheel. It won't be used automatically during installs, unless --find-links or PIP_FIND_LINKS is pointing at it, but I don't see that in your examples?

alex · 2013-09-28T18:47:07Z

Yup, here is my pip.conf:

[global]
timeout = 15
download_cache = ~/.pip/cache
allow-all-external = false
no-allow-insecure = false
use-wheel = true
wheel-dir = /Users/alex_gaynor/.pip/wheels
find-links = /Users/alex_gaynor/.pip/wheels

qwcode · 2013-09-28T18:47:52Z

ok, you mind listing your wheels dir for me? the names will help.

alex · 2013-09-28T18:48:27Z

It's pretty huge so I put it in a paste: http://bpaste.net/show/136231/

qwcode · 2013-09-28T20:20:55Z

As it turns out, the root issue here is general.
Our link matching code thinks any pytest-xdist archive url fulfills the requirement pytest.
When not using --pre, the version (determined as xdist-X.Y) gets skipped as a pre-release.
When using --pre (like in the example above in tox), it gets passed on as a valid url.

In the case of wheels, there's some additional sorting that operates on the filename, which is not expecting unsupported archives, for the wrong project to be passed in.

e.g. download https://pypi.python.org/packages/source/p/pytest-xdist/pytest-xdist-1.8.zip into a local archives folder and do this:

pip install --no-deps --no-index --pre --find-links=archives pytest

it will happily install pytest-xdist for you.

Our matching routine right now tries to get away with not determining the project name, and matching the whole filename with <requirement>-. This shortcoming has come up before in other issues (#804 (comment))

I think the solution is to change this routine so that it matches against the project name. distlib has a function for determining the project name (pip.vendor.distlib.util.split_filename)

qwcode · 2013-09-30T04:42:46Z

I had a PR with tests ready to go, but there's a snag with using pip.vendor.distlib.util.split_filename. it doesn't parse when 'dev' is the version, but pip supports locating and installing a 'dev' version. So, I'm inclined to copy the split_filename function up into pip and modify it's regexp (and submit an issue to distlib about it, unless vinay could fix this quickly) cc @vsajip

dstufft · 2013-09-30T04:44:37Z

I temporarily switched PyPI to using the split_filename function and it broke in a decent number of conditions, I ended up pulling it out immediately.

qwcode · 2013-09-30T04:53:00Z

what does PyPI have now?

dstufft · 2013-09-30T04:53:39Z

Something that pip can't do; I looked it up by filename in the database

qwcode · 2013-09-30T16:49:16Z

donald, do you mind posting the ones that fail.

for the sake of 1.4.2, the conservative thing I think it just to fix this for wheel archives, and leave fixing sdist parsing to 1.5, which will provide beta testing, which a major change like this really needs.

I'll post a PR probably tomorrow.

dstufft · 2013-09-30T16:53:09Z

Um, I think django-altuser-v0.6.8.tar.gz was one of them, I believe youtube_dl_server-alpha.1.tar.gz was another. I don't remember the others.

qwcode · 2013-09-30T17:10:14Z

the distlib function is only able to be better at parsing project names like "pytest-xdist" because it assumes a more "rigid" versioning scheme (versions starting with [0-9]. Is that right? (except for not allowing "dev")

Breaking parsing for pre-existing distributions is bad, but something has to give here. staying with what we have for sdists is pretty bad too.

vsajip · 2013-09-30T18:20:44Z

With the latest update to distlib, split_filename does this:

split_filename('django-altuser-v0.6.8') -> ('django-altuser', 'v0.6.8', None)
split_filename('youtube_dl_server-alpha.1') -> ('youtube_dl_server', 'alpha.1', None)

I can't say whether it will correctly deal with all combinations on PyPI, though - there are some pretty varied version and project naming styles there. By all means log an issue if a reasonable-looking project name/version combination isn't handled.

vsajip · 2013-09-30T18:22:38Z

Oh, and

split_filename('pytest-xdist-dev') -> ('pytest-xdist', 'dev', None)

qwcode · 2013-09-30T18:46:37Z

oh, good

qwcode · 2013-09-30T19:59:02Z

hey, @vsajip, can you add additional tests for that diff? https://bitbucket.org/pypa/distlib/commits/22c38563185bf13bfd267efc88d147a51e9880e3

vsajip · 2013-09-30T20:36:46Z

@qwcode: Done, see https://bitbucket.org/pypa/distlib/commits/95acdc93081748494fda1383c18d0f574f19389c

qwcode · 2013-09-30T20:51:29Z

@vsajip, thanks, one case I see missing in your suite is the wheel filename case where both the project name and version have "-" replaced with "_", e.g. "pytest_xdist-0.1_myfork"

vsajip · 2013-09-30T20:58:47Z

@qwcode, see https://bitbucket.org/pypa/distlib/commits/0033e59c8cfb6426b3bd52f73355a4934d2feeab

fix for #1192

This was referenced Oct 2, 2013

a "name1" requirement can be fulfilled by a project called "name1-name2" (for sdists) #1217

Closed

fix for #1192 #1218

Merged

qwcode added a commit that referenced this issue Oct 6, 2013

Merge pull request #1218 from qwcode/issue-1192

3aafbab

fix for #1192

qwcode closed this as completed Oct 9, 2013

lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 6, 2019

lock bot locked as resolved and limited conversation to collaborators Jun 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError: bad operand type for unary -: 'NoneType' #1192

TypeError: bad operand type for unary -: 'NoneType' #1192

alex commented Sep 7, 2013

alex commented Sep 8, 2013

dstufft commented Sep 8, 2013

alex commented Sep 26, 2013

qwcode commented Sep 28, 2013

qwcode commented Sep 28, 2013

alex commented Sep 28, 2013

qwcode commented Sep 28, 2013

alex commented Sep 28, 2013

qwcode commented Sep 28, 2013

qwcode commented Sep 30, 2013

dstufft commented Sep 30, 2013

qwcode commented Sep 30, 2013

dstufft commented Sep 30, 2013

qwcode commented Sep 30, 2013

dstufft commented Sep 30, 2013

qwcode commented Sep 30, 2013

vsajip commented Sep 30, 2013

vsajip commented Sep 30, 2013

qwcode commented Sep 30, 2013

qwcode commented Sep 30, 2013

vsajip commented Sep 30, 2013

qwcode commented Sep 30, 2013

vsajip commented Sep 30, 2013

TypeError: bad operand type for unary -: 'NoneType' #1192

TypeError: bad operand type for unary -: 'NoneType' #1192

Comments

alex commented Sep 7, 2013

alex commented Sep 8, 2013

dstufft commented Sep 8, 2013

alex commented Sep 26, 2013

qwcode commented Sep 28, 2013

qwcode commented Sep 28, 2013

alex commented Sep 28, 2013

qwcode commented Sep 28, 2013

alex commented Sep 28, 2013

qwcode commented Sep 28, 2013

qwcode commented Sep 30, 2013

dstufft commented Sep 30, 2013

qwcode commented Sep 30, 2013

dstufft commented Sep 30, 2013

qwcode commented Sep 30, 2013

dstufft commented Sep 30, 2013

qwcode commented Sep 30, 2013

vsajip commented Sep 30, 2013

vsajip commented Sep 30, 2013

qwcode commented Sep 30, 2013

qwcode commented Sep 30, 2013

vsajip commented Sep 30, 2013

qwcode commented Sep 30, 2013

vsajip commented Sep 30, 2013