colon (:) support for virtualenv directory path #395

rrauenza · 2013-02-11T23:02:33Z

I'm guessing paths get screwed up since a path delimiter is usually colon. Emitting an error may help diagnose this for others:

[fedora:/tmp/a:b]$ virtualenv -v -v -v -v -v env
Creating env/lib/python2.7
This system uses lib64; symlinking lib64 to lib
Symlinking env/lib64
Symlinking Python bootstrap modules
Symlinking env/lib/python2.7/config
Symlinking env/lib/python2.7/lib-dynload
Symlinking env/lib64/python2.7/os.py
Ignoring built-in bootstrap module: posix
Symlinking env/lib64/python2.7/posixpath.py
Cannot import bootstrap module: nt
Symlinking env/lib64/python2.7/ntpath.py
Symlinking env/lib64/python2.7/genericpath.py
Symlinking env/lib64/python2.7/fnmatch.py
Symlinking env/lib64/python2.7/locale.py
Symlinking env/lib64/python2.7/encodings
Symlinking env/lib64/python2.7/codecs.py
Symlinking env/lib64/python2.7/stat.py
Symlinking env/lib64/python2.7/UserDict.py
File env/lib64/python2.7/lib-dynload/readline.so already exists
Symlinking env/lib64/python2.7/copy_reg.py
Symlinking env/lib64/python2.7/types.py
Symlinking env/lib64/python2.7/re.py
Symlinking env/lib64/python2.7/sre.py
Symlinking env/lib64/python2.7/sre_parse.py
Symlinking env/lib64/python2.7/sre_constants.py
Symlinking env/lib64/python2.7/sre_compile.py
File env/lib64/python2.7/lib-dynload/zlibmodule.so already exists
Symlinking env/lib64/python2.7/warnings.py
Symlinking env/lib64/python2.7/linecache.py
Symlinking env/lib64/python2.7/_abcoll.py
Symlinking env/lib64/python2.7/abc.py
Symlinking env/lib64/python2.7/_weakrefset.py
Creating env/lib/python2.7/site-packages
Writing env/lib64/python2.7/site.py
Writing env/lib64/python2.7/orig-prefix.txt
Writing env/lib64/python2.7/no-global-site-packages.txt
Creating parent directories for env/include
Symlinking env/include/python2.7
Creating env/bin
New python executable in env/bin/python
Changed mode of env/bin/python to 0755
Testing executable with env/bin/python -c "
import sys
prefix = sys.prefix
if sys.version_info[0] == 3:
prefix = prefix.encode('utf8')
if hasattr(sys.stdout, 'detach'):
sys.stdout = sys.stdout.detach()
elif hasattr(sys.stdout, 'buffer'):
sys.stdout = sys.stdout.buffer
sys.stdout.write(prefix)
"
ImportError: No module named site
ERROR: The executable env/bin/python is not functioning
ERROR: It thinks sys.prefix is u'/tmp/a:b' (should be u'/tmp/a:b/env')
ERROR: virtualenv is not compatible with this system or executable
[fedora:/tmp/a:b]$

[fedora:/tmp/a-b]$ virtualenv env
New python executable in env/bin/python
Installing setuptools............done.
Installing pip...............done.
[fedora:/tmp/a-b]$

[fedora:/tmp/a-b]$ python --version
Python 2.7.3
[fedora:/tmp/a-b]$ virtualenv --version
1.7.1.2
[fedora:/tmp/a-b]$

jvrsantacruz · 2013-04-30T08:42:28Z

I'm also affected by this behaviour.

~/application-2013-04-30T10:23:47$ virtualenv env
New python executable in env/bin/python
ImportError: No module named site
ERROR: The executable env/bin/python is not functioning
ERROR: It thinks sys.prefix is u'/home/deployer/application-2013-04-30T10:23:47' (should be u'/home/deployer/application-2013-04-30T10:23:47/env')
ERROR: virtualenv is not compatible with this system or executable

while it perfectly works by just removing the colons from the directory path.

~/application-2013-04-30$ virtualenv env
New python executable in env/bin/python
Please make sure you remove any previous custom paths from your /home/deployer/.pydistutils.cfg file.
Installing distribute...........................................................................................................................................................................................................................done.
Installing pip................done.

My versions are:

$ virtualenv --version
1.9.1
$ python --version
Python 2.7.2+

alekstorm · 2013-12-05T00:12:08Z

Reporting in with virtualenv==1.10.1 and python==2.7.3. Was trying to embed a datetime in the directory name too :).

srikanth-viswanathan · 2016-05-20T02:31:29Z

Chiming in to report this issue as well. I actually encountered this while trying to run a virtualenv inside chronos. The path to the sandbox has colons in it which breaks virtualenv.

erikw · 2018-05-16T15:41:13Z

After hours of debugging, thinking I had some problems with macOS default python vs homebrew python vs pyenv python, I finally discovered that virtualenv works perfect, but not when you are in a directory which has a colon in it's name!

This bug should really be fixed, as it causes very hard to debug problems.

Just for the reference, the exact error I had, so people with same stracktrace can land here from Google:

$ mkdir ~/github.com:22
$ cd !$
$ virtualenv -p python3 .                                                                                                                                                                                                            3.3
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6'
New python executable in /home/erikw/github.com:22/bin/python3.6
Also creating executable in /home/erikw/github.com:22/bin/python
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fff9fb6f380 (most recent call first):
Installing setuptools, pip, wheel...
  Complete output from command /home/erikw/github.com:22/bin/python3.6 - setuptools pip wheel:
  Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fff9fb6f380 (most recent call first):
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/virtualenv.py", line 2349, in <module>
    main()
  File "/usr/local/lib/python3.6/site-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/usr/local/lib/python3.6/site-packages/virtualenv.py", line 953, in create_environment
    download=download,
  File "/usr/local/lib/python3.6/site-packages/virtualenv.py", line 904, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "/usr/local/lib/python3.6/site-packages/virtualenv.py", line 796, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/erikw/github.com:22/bin/python3.6 - setuptools pip wheel failed with error code -6

$ cd ..
$ mv github.com:22 github.com
$ cd !$
$ virtualenv -p python3 .                                                                                                                                                                                                            3.3
[..] # all fine, works now!

I'm using virtualenv version 15.2.0 installed with pip3 and python version 3.6.5.

Why do I have a directory with a colon in it you might ask? Because tools that create directories based on URLs, which is a not too uncommon scenario, e.g. the git clone manager ghq.

stale · 2019-04-14T08:27:29Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.

erikw · 2019-04-14T08:37:33Z

This is not stale.

rrauenza · 2019-05-20T06:29:27Z

Given the shell uses the colon as a path separator ... I'm not sure this could ever be fixed. How would you add a path to PATH that includes a colon?

https://stackoverflow.com/questions/21641294/if-there-is-a-colon-in-the-directory-name-how-could-i-add-it-to-path

I think the only option is for virtualenv to error out with an explanation if the destination directory has a colon in it? Which would be somewhat simple to add and save some folks a lot of time...

erikw · 2019-05-20T08:36:41Z

If we can't fix it, at least a warning/error message making debugging easier would be good.

gaborbernat · 2019-05-20T10:16:10Z

The path separator is platform dependent (see https://docs.python.org/3.7/library/os.html#os.pathsep). We should hard error if the path contains the path separator, but allow going ahead otherwise. A pull request achieving this would be accepted and merged.

rrauenza · 2019-05-23T18:46:47Z

Pull Request:

#1360

gaborbernat · 2019-05-24T12:07:07Z

Resolved via #1360

Given a git url, ctool clones the repository to a target directory named as the url with everything up the last '/' removed, so: https://github.com/proj/repo is cloned to 'repo' lp:~user/repo is cloned to 'repo' This doesn't work when the url doesn't contain a '/', something that can happen when using the 'lp:' shorthand, e.g. lp:project is cloned to 'lp:project' This happens to break the virtualenv Python module in some test environments because of [1]. This commit constructs the `git clone` target directory first by trimming the git url up to the first ':', and then up to the last '/'. [1] pypa/virtualenv#395

Given a git url, ctool clones the repository to a target directory named as the url with everything up the last '/' removed, so: https://github.com/user/repo is cloned to 'repo' lp:~user/repo is cloned to 'repo' This doesn't work when the url doesn't contain a '/', something that can happen when using the 'lp:' shorthand, e.g. lp:project is cloned to 'lp:project' This happens to break the virtualenv Python module in some test environments because of [1]. This commit constructs the `git clone` target directory first by trimming the git url up to the first ':', and then up to the last '/', accounting for 'slashless' URLs. [1] pypa/virtualenv#395

bmw mentioned this issue Jul 7, 2016

Creating Virtual Environment - "No module named site" on Ubuntu 14.04 certbot/certbot#3220

Closed

pawelad mentioned this issue Mar 18, 2017

tox fails when ":" is in the current directory path tox-dev/tox#203

Closed

erikw mentioned this issue May 16, 2018

remove port number in local repository path x-motemen/ghq#26

Closed

gaborbernat added bug help-wanted labels Jan 14, 2019

gaborbernat changed the title ~~virtualenv fails if colon (:) is in directory path~~ colon (:) support for virtualenv directory path Jan 14, 2019

stale bot added the wontfix label Apr 14, 2019

stale bot removed the wontfix label Apr 14, 2019

gaborbernat closed this as completed May 24, 2019

paride mentioned this issue Nov 22, 2019

ctool: ensure there are no colons in the 'git clone' target directory canonical/uss-tableflip#21

Merged

pypa locked and limited conversation to collaborators Jan 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

colon (:) support for virtualenv directory path #395

colon (:) support for virtualenv directory path #395

rrauenza commented Feb 11, 2013

jvrsantacruz commented Apr 30, 2013

Uh oh!

alekstorm commented Dec 5, 2013

Uh oh!

srikanth-viswanathan commented May 20, 2016

Uh oh!

erikw commented May 16, 2018 •

edited

Loading

Uh oh!

stale bot commented Apr 14, 2019

Uh oh!

erikw commented Apr 14, 2019

Uh oh!

rrauenza commented May 20, 2019 •

edited

Loading

Uh oh!

erikw commented May 20, 2019 •

edited

Loading

Uh oh!

gaborbernat commented May 20, 2019 •

edited

Loading

Uh oh!

rrauenza commented May 23, 2019 •

edited

Loading

Uh oh!

gaborbernat commented May 24, 2019

Uh oh!

Uh oh!

colon (:) support for virtualenv directory path #395

colon (:) support for virtualenv directory path #395

Comments

rrauenza commented Feb 11, 2013

jvrsantacruz commented Apr 30, 2013

Uh oh!

alekstorm commented Dec 5, 2013

Uh oh!

srikanth-viswanathan commented May 20, 2016

Uh oh!

erikw commented May 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stale bot commented Apr 14, 2019

Uh oh!

erikw commented Apr 14, 2019

Uh oh!

rrauenza commented May 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erikw commented May 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gaborbernat commented May 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rrauenza commented May 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gaborbernat commented May 24, 2019

Uh oh!

erikw commented May 16, 2018 •

edited

Loading

rrauenza commented May 20, 2019 •

edited

Loading

erikw commented May 20, 2019 •

edited

Loading

gaborbernat commented May 20, 2019 •

edited

Loading

rrauenza commented May 23, 2019 •

edited

Loading