appdirs: return paths relative to effective user's homedir

[terminalmage]

When executing a pip install from within a Python process in which the euid has been changed, the appdirs helpers which use expanduser() return paths relative to the home directory of the actual user, rather than the effective user. This causes the install to fail when the effective user cannot write to the cachedir:

>>> import subprocess
>>> import os
>>> os.geteuid()
0
>>> os.seteuid(1000)
>>> os.geteuid()
1000
>>> p = subprocess.Popen(['/home/erik/virtualenv/pipdev/bin/pip', 'install', 'pudb'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> stdout, stderr = p.communicate()
>>> p.returncode
1
>>> stderr
"Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/root/.cache/pip/wheels/ec/bb/79/09cd8a9982b637b251208ac3400f9702c72e63accc8e4b69e3'\nConsider using the `--user` option or check the permissions.\n\n"
>>>

With this bugfix, the installation proceeds as expected, and the sources/wheels are downloaded to the effective user's cachedir:

>>> import subprocess
>>> import os
>>> os.geteuid()
0
>>> os.seteuid(1000)
>>> os.geteuid()
1000
>>> p = subprocess.Popen(['/home/erik/virtualenv/pipdev/bin/pip', 'install', 'pudb'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> stdout, stderr = p.communicate()
>>> p.returncode
0
>>> print(stdout)
Collecting pudb
Collecting urwid>=1.1.1 (from pudb)
Collecting pygments>=1.0 (from pudb)
  Using cached https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl
Installing collected packages: urwid, pygments, pudb
Successfully installed pudb-2017.1.4 pygments-2.2.0 urwid-2.0.1

>>>

With regard to testing, I'm not certain how this would work with the existing setup, as the code appears to just be patching function calls, and in the absence of something like a MagicMock where you can do more complicated patching, I'm unsure how we'd confirm that we're pulling from the effective user's environment. I would welcome any guidance offered in this respect.

Resolves #5359.

[terminalmage]

Unit test failures are clearly related, I'll take a look on Thursday or Friday.

[benoit-pierre]

Note: there are 2 copies of appdirs: ./src/pip/_internal/utils/appdirs.py and ./src/pip/_vendor/appdirs.py. IMHO this should be reported to the appdirs project, and pip can get the fix by re-vendoring the new version and updating its copy. See previous discussion on whether pip's internal copy should be dropped or not.

[terminalmage]

Ahh, thanks for catching (this is what I get when I don't read the docstring!). I can resubmit upstream.

[terminalmage]

Submitted upstream as ActiveState/appdirs#105

[benoit-pierre]

I'm also not sure this is a bug and not normal behavior if HOME or one of the XDG.. variable is set. Is that the case? Shouldn't you pass a patched-up environment to subprocess?

[pradyunsg]

Closing this PR since the fix has to happen upstream and this hasn't been shown to be significant and/or urgent enough that pip carries a patch for it.

[terminalmage]

I agree, it's not terribly urgent. It's true that one can pass a HOME to subprocess via the env, though in my opinion one shouldn't have to do so. But given that there is a simple workaround, getting this fix from upstream doesn't seem like a high priority.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.