pytest has issues with nodeid and symlinks

[ghost]

I believe this is similar to #4321, but seeing as that was closed I thought I would open up a new issue with some of the details that I have gathered. I was able to reproduce this with both Python 2.7 with pytest 4.6.9 and Python 3.8.2 with pytest 5.4.1.

Here is the pip list for the Python 3 environment

root@9008da3b5067:/# pip list
Package        Version
-------------- -------
attrs          19.3.0 
more-itertools 8.2.0  
packaging      20.3   
pip            20.0.2 
pluggy         0.13.1 
py             1.8.1  
pyparsing      2.4.6  
pytest         5.4.1  
setuptools     45.2.0 
six            1.14.0 
wcwidth        0.1.8  
wheel          0.34.2 

Here is the pip list in the Python 2 environment

root@15e2b2ea0ffd:~# pip list
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Package             Version    
------------------- -----------
appdirs             1.4.3      
atomicwrites        1.3.0      
attrs               19.3.0     
configparser        4.0.2      
contextlib2         0.6.0.post1
distlib             0.3.0      
filelock            3.0.12     
funcsigs            1.0.2      
importlib-metadata  1.5.0      
importlib-resources 1.0.2      
more-itertools      5.0.0      
packaging           20.3       
pathlib2            2.3.5      
pip                 20.0.2     
pluggy              0.13.1     
py                  1.8.1      
pyparsing           2.4.6      
pytest              4.6.9      
scandir             1.10.0     
setuptools          44.0.0     
six                 1.14.0     
typing              3.7.4.1    
virtualenv          20.0.7     
wcwidth             0.1.8      
wheel               0.34.2     
zipp                1.2.0   

I am not sure you need the Python 2 information, but thought it might help to show that it existed in previous versions as well.

In both situations I used a docker container to run the test:

1. docker pull python:2
2. docker pull python:3

Here is what I did to reproduce the issue

1. mkdir /test_pytest
2. Create test within directory. I created test_random.py with the following contents:

import pytest

def test_thing():
    assert 2 + 2 == 4

3. Change to user's home directory. Example: cd /root
4. Symlink the directory from step 1 to the user's home directory. As I was doing this in a docker container I did the following: ln -s /test_pytest test_pytest
5. Run pytest -sv /root/test_pytest/test_random.py
   Output is as follows:

root@9008da3b5067:~# pytest -sv /root/test_pytest/test_random.py 
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /root
collected 1 item                                                                                                                                                                                                  

::test_thing PASSED

================================================================================================ 1 passed in 0.01s ================================================================================================

Notice how the nodeid is represented as ::test_thing::. I believe this is incorrect, but maybe I am mistaken.

I would think that it would at least be similar across different runs of the same test; however, running this in different directories produces different results.

For example:

root@9008da3b5067:~# pwd
/root
root@9008da3b5067:~# pytest -sv test_pytest/test_random.py 
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /root
collected 1 item                                                                                                                                                                                                  

::test_thing PASSED

================================================================================================ 1 passed in 0.01s ================================================================================================

Also shows the issue.

This does not though:

root@9008da3b5067:~# pytest -sv test_pytest
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /root
collected 1 item                                                                                                                                                                                                  

test_random.py::test_thing PASSED

================================================================================================ 1 passed in 0.01s ================================================================================================

Also going into the directory itself also shows what I would expect the output to be:

root@9008da3b5067:~# cd test_pytest
root@9008da3b5067:~/test_pytest# pwd
/root/test_pytest
root@9008da3b5067:~/test_pytest# pytest -sv .
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /test_pytest
collected 1 item                                                                                                                                                                                                  

test_random.py::test_thing PASSED

================================================================================================ 1 passed in 0.01s ================================================================================================
root@9008da3b5067:~/test_pytest# pytest -sv test_random.py 
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /test_pytest
collected 1 item                                                                                                                                                                                                  

test_random.py::test_thing PASSED

================================================================================================ 1 passed in 0.00s ================================================================================================

Anyway I apologize if the information is jumbled, but wanted to try to capture as much info as possible.

Let me know if there is any other information you may need to help with this issue.

[gnikonorov]

@haapaojas I can't reproduce this on Ubuntu. Could this be specific to docker?

I attached a file showing a bunch of invocations of pytest with a symlink. In all cases the symlink was translated in the pytest output. Does the issue go away with pytest v 5.4.2?

pytest_issue_6941.txt

Package versions:

(virtual_env) gnikonorov:~$ which pip
/home/gnikonorov/testing/test_pytest/virtual_env/bin/pip
(virtual_env) gnikonorov:~$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
attrs (19.3.0)
importlib-metadata (1.6.0)
more-itertools (8.3.0)
packaging (20.4)
pip (9.0.1)
pkg-resources (0.0.0)
pluggy (0.13.1)
py (1.8.1)
pyparsing (2.4.7)
pytest (5.4.2)
setuptools (39.0.1)
six (1.15.0)
wcwidth (0.1.9)
zipp (3.1.0)
(virtual_env) gnikonorov:~$

[ghost]

I just retested with pytest-5.4.2 and got the same behavior as before. With that being said your tests also worked for me.
I believe it has something to do with symlinks that show as the root part of the directory. So for instance the example that I have the symlink looks like this:
test_pytest -> /test_pytest
The symlink in your example does not have the beginning '/'. That is important for a reproduction. I attempted to try to reproduce this myself within the pytest unit testing environment, but I was not able to figure out how to replicate this particular symlink hierarchy within the testdir environment.
The examples in test_collection.py do not replicate the exact issue seen here. Also I can try to show a repro in a non-docker environment, but I have seen this at very least on CentOS.

[gnikonorov]

Thanks for the additional information @haapaojas, I'll look into this some more this weekend