tests inside package : conftest loaded multiple times ?

[asmodehn]

Repository with project and commit causing the problem : pyros-dev/pyros-setup@81a4c6e

Symptom :

(pyros-setup) alexv@asmodehn:~/Projects/pyros-setup$ pyros_setup --pytest --distro=indigo
========================================================================================================= test session starts =========================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/alexv/Projects/pyros-setup, inifile: 
collected 0 items / 1 errors 

=============================================================================================================== ERRORS ================================================================================================================
__________________________________________________________________________________________________________ ERROR collecting  __________________________________________________________________________________________________________
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/py/_path/common.py:332: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/py/_path/common.py:368: in gen
    if p.check(dir=1) and (rec is None or rec(p))])
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/main.py:672: in _recurse
    ihook = self.gethookproxy(path)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/main.py:575: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/config.py:315: in _getconftestmodules
    mod = self._importconftest(conftestpath)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/config.py:351: in _importconftest
    self.consider_conftest(mod)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/config.py:374: in consider_conftest
    if self.register(conftestmodule, name=conftestmodule.__file__):
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/config.py:226: in register
    ret = super(PytestPluginManager, self).register(plugin, name)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:370: in register
    hook._maybe_apply_history(hookimpl)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:747: in _maybe_apply_history
    res = self._hookexec(self, [method], kwargs)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:338: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:333: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:596: in execute
    res = hook_impl.function(*args)
../../.virtualenvs/pyros-setup/lib/python2.7/site-packages/pyros_setup/tests/conftest.py:5: in pytest_addoption
    help="distro: indigo or jade or kinetic")
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/config.py:482: in addoption
    self._anonymous.addoption(*opts, **attrs)
../../.virtualenvs/pyros-setup/local/lib/python2.7/site-packages/_pytest/config.py:707: in addoption
    raise ValueError("option names %s already added" % conflict)
E   ValueError: option names set(['--distro']) already added
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================================================= 1 error in 0.17 seconds =======================================================================================================

Recipe :

• Checkout that commit from pyros-setup.
• Notice how tests using pytest are included with the package, and allow passing an option.
• Create a virtual environment to work on it.
• Install with pip install -e . Launching with pyros_setup --pytest --distro=indigo should work fine (test fail or succeed depending on your system state and on that option)
• Remove with pip uninstall pyros_setup
• Install with pip install . Launching with pyros_setup --pytest --distro=indigo fail with the error ValueError: option names set(['--distro']) already added

On Ubuntu Trusty, here is my virtual env pip list (using system packages) :

(pyros-setup) alexv@asmodehn:~/Projects/pyros-setup$ pip list
alembic (0.6.2)
amqp (1.3.3)
ansible (2.1.1.0)
anyjson (0.3.3)
apt-xapian-index (0.45)
argh (0.26.1)
args (0.1.0)
autopep8 (0.9.1)
Babel (1.3)
backports.ssl-match-hostname (3.5.0.1)
beautifulsoup4 (4.2.1)
billiard (3.3.0.15)
binaryornot (0.2.0)
blinker (1.3)
bloom (0.5.21)
bzr (2.1.4)
catkin-pkg (0.2.10)
catkin-sphinx (0.2.2)
catkin-tools (0.4.2)
celery (3.1.6)
chardet (2.0.1)
Cheetah (2.4.4)
cl (0.0.3)
clint (0.5.1)
cobbler (2.4.1)
colorama (0.2.5)
command-not-found (0.3)
configobj (4.7.2)
cookiecutter (0.6.4)
coverage (3.7.1)
debtagshw (0.1)
defer (1.0.6)
dirspec (13.10)
distro-info (0.12)
Django (1.6.1)
docker-py (1.8.1)
dockerpty (0.3.4)
docopt (0.6.2)
docutils (0.11)
dulwich (0.9.4)
empy (3.1)
enum34 (0.9.23)
epydoc (3.0.1)
fastimport (0.9.2)
fig (1.0.1)
Flask (0.10.1)
futures (2.1.6)
gbp (0.6.9)
git-remote-helpers (0.1.0)
gitchangelog (2.3.0)
gunicorn (17.5)
html5lib (0.999)
httplib2 (0.8)
importlib (1.0.3)
iotop (0.6)
ipaddress (1.0.16)
itsdangerous (0.22)
Jinja2 (2.7.2)
jsonpickle (0.9.2)
keyring (3.5)
kitchen (1.1.1)
kombu (3.0.7)
launchpadlib (1.10.2)
lazr.restfulclient (0.13.3)
lazr.uri (1.0.3)
libvirt-python (1.2.2)
lxml (3.3.3)
mailer (0.7)
Mako (0.9.1)
MarkupSafe (0.18)
matplotlib (1.3.1)
meld3 (0.6.10)
mercurial (1.4.2)
mock (1.0.1)
mod-python (3.3.1)
MySQL-python (1.2.3)
netaddr (0.7.10)
netifaces (0.8)
nose (1.3.1)
numpy (1.8.2)
oauth (1.0.1)
oauthlib (0.6.1)
oneconf (0.3.7.14.4.1)
osrf-pycommon (0.1.2)
PAM (0.4.2)
paramiko (1.10.1)
passlib (1.5.3)
pathtools (0.1.2)
pep8 (1.4.6)
pexpect (3.1)
Pillow (2.3.0)
pip (8.1.2)
piston-mini-client (0.7.5)
pkginfo (1.3.2)
pluggy (0.3.1)
progressbar (2.3)
protobuf (2.5.0)
psutil (1.2.1)
py (1.4.31)
pyasn1 (0.1.7)
pycrypto (2.6.1)
pycups (1.9.66)
pycurl (7.19.3)
pydot (1.0.28)
pyflakes (0.8.1)
Pygments (1.6)
pygobject (3.12.0)
pygpgme (0.3)
pygraphviz (1.2)
pyinotify (0.9.4)
pymongo (2.6.3)
PyOpenGL (3.0.2)
pyOpenSSL (0.13)
pyparsing (2.0.1)
pyros-config (0.1.99, /home/alexv/Projects/pyros-config)
pyros-setup (0.1.99)
pyserial (2.6)
pysmbc (1.0.14.1)
pytest (3.0.1)
python-apt (0.9.3.5ubuntu2)
python-dateutil (1.5)
python-debian (0.1.21-nmu2ubuntu2)
python-memcached (1.53)
python-networkmanager (1.2.1)
pytz (2012rc0)
pyxdg (0.25)
PyYAML (3.10)
pyzmq (14.0.1)
redis (2.7.2)
reportlab (3.0)
requests (2.10.0)
requests-toolbelt (0.7.0)
roman (2.0.0)
ros-buildfarm (1.1.0)
rosdep (0.11.5)
rosdistro (0.4.7)
rospkg (1.0.39)
SecretStorage (2.0.0)
sessioninstaller (0.0.0)
setuptools (26.1.1)
simplejson (3.3.1)
six (1.10.0)
software-center-aptd-plugins (0.0.0)
Sphinx (1.2.2)
sphinx-rtd-theme (0.1.7)
sphinxcontrib-plantuml (0.6)
SQLAlchemy (0.8.4)
ssh-import-id (3.21)
stevedore (0.14.1)
supervisor (3.0b2)
termcolor (1.1.0)
testfixtures (4.7.0)
texttable (0.8.3)
tornado (3.1.1)
tox (2.3.1)
transitions (0.2.7)
trollius (2.1)
twine (1.8.1)
Twisted (13.2.0)
Twisted-Conch (13.2.0)
Twisted-Core (13.2.0)
Twisted-Lore (13.2.0)
Twisted-Mail (13.2.0)
Twisted-Names (13.2.0)
Twisted-News (13.2.0)
Twisted-Runner (13.2.0)
Twisted-Web (13.2.0)
Twisted-Words (13.2.0)
urlgrabber (3.9.1)
urllib3 (1.7.1)
vboxapi (1.0)
vcstools (0.1.38)
virtinst (0.600.4)
virtualenv (15.0.2)
virtualenv-clone (0.2.4)
virtualenvwrapper (4.1.1)
VTK (5.8.0)
wadllib (1.3.2)
watchdog (0.8.3)
websocket-client (0.37.0)
Werkzeug (0.9.4)
wheel (0.29.0)
wstool (0.1.13)
wstools (0.4.3)
WTForms (1.0.1)
wxPython (2.8.12.1)
wxPython-common (2.8.12.1)
xdot (0.5)
yujin-ansible-playbooks (0.1.7)
yujin-tools (0.4.33)
zenmap (6.40)
zope.interface (4.0.5)

[asmodehn]

Travis job illustrating the problem : https://travis-ci.org/asmodehn/pyros-setup/jobs/156150350

Note this happens only when launching pytest from the package itself (as part of __main__.py)

When launching pytest directly it doesnt happen : https://travis-ci.org/asmodehn/pyros-setup/jobs/156141458

So there is likely a problem with the way pytest is run from main...

I'd appreciate even a dirty workaround if there is no immediate fix, as it s really important for user of this package to run this test from installation... I have tried a few (like inspecting the option list before adding) but none seems to make any difference so far...

Thanks a lot !

[asmodehn]

I put a few print statements in pytest's config.py, and it does seems like pytest_addoption in conftest.py is indeed called multiple times in that usecase :

(pyros-setup) alexv@asmodehn:~/Projects/pyros-setup$ pyros_setup --pytest
Existing options in general : ['-k', '-m']
Existing options in collect : []
Existing options in collect : ['--collectonly', '--collect-only']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest']
Existing options in debugconfig : []
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s']
Existing options in debugconfig : ['--basetemp']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin', '--junitxml', '--junit-xml']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin', '--junitxml', '--junit-xml', '--junitprefix', '--junit-prefix']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules', '--doctest-report']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules', '--doctest-report', '--doctest-glob']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed', '--ff', '--failed-first']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed', '--ff', '--failed-first', '--cache-show']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert', '--setuponly', '--setup-only']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert', '--setuponly', '--setup-only', '--setupshow', '--setup-show']
options detected : []
Parser.addoption CALLED WITH opts : ('--distro',)
Existing options in custom options : []
========================================================================================================= test session starts =========================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/alexv/Projects/pyros-setup, inifile: 
options detected : ['--distro']
Parser.addoption CALLED WITH opts : ('--distro',)
Existing options in custom options : ['--distro']
collected 0 items / 1 errors 

[asmodehn]

While fiddling with it, it seems the first call is because of a conftest.py visible from my current directory.
The second call however is from the conftest.py inside the package.

Changing my current dir to not have the source of the packege under the current directory fixes the problem...

Is there a way for pytest to prevent collecting from current directory ?

[The-Compiler]

Maybe the confcutdir setting helps somehow? Unfortunately I'mnot sure what's going on, and currently lack the time to investigate...

[RonnyPfannschmidt]

im experiencing something similar and will investigate at work

[asmodehn]

More information about this issue...

On the other hand, having a conftest.py file in an installed package is sometimes detected :

(pyros-setup) alexv@asmodehn:~/Projects/pyros-setup/testing$ pyros_setup --pytest
Existing options in general : ['-k', '-m']
Existing options in collect : []
Existing options in collect : ['--collectonly', '--collect-only']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest']
Existing options in debugconfig : []
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s']
Existing options in debugconfig : ['--basetemp']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin', '--junitxml', '--junit-xml']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin', '--junitxml', '--junit-xml', '--junitprefix', '--junit-prefix']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules', '--doctest-report']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules', '--doctest-report', '--doctest-glob']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed', '--ff', '--failed-first']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed', '--ff', '--failed-first', '--cache-show']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert', '--setuponly', '--setup-only']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert', '--setuponly', '--setup-only', '--setupshow', '--setup-show']
========================================================================================================= test session starts =========================================================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/alexv/Projects/pyros-setup, inifile: 
Parser.addoption CALLED WITH opts : ('--distro',)
Existing options in custom options : []
collected 1 items 
[...]

And sometimes not :

(pyros-setup) alexv@asmodehn:~/Projects/pyros-setup/testing$ pyros_setup --pytest --distro indigo
Existing options in general : ['-k', '-m']
Existing options in collect : []
Existing options in collect : ['--collectonly', '--collect-only']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest']
Existing options in debugconfig : []
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s']
Existing options in debugconfig : ['--basetemp']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin', '--junitxml', '--junit-xml']
Existing options in terminal reporting : ['-v', '--verbose', '-q', '--quiet', '-r', '--disable-pytest-warnings', '-l', '--showlocals', '--tb', '--fulltrace', '--full-trace', '--color', '--durations', '--pastebin', '--junitxml', '--junit-xml', '--junitprefix', '--junit-prefix']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules', '--doctest-report']
Existing options in collect : ['--collectonly', '--collect-only', '--pyargs', '--ignore', '--confcutdir', '--noconftest', '--keepduplicates', '--keep-duplicates', '--doctest-modules', '--doctest-report', '--doctest-glob']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed', '--ff', '--failed-first']
Existing options in general : ['-k', '-m', '--markers', '-x', '--exitfirst', '--maxfail', '--strict', '-c', '--continue-on-collection-errors', '--fixtures', '--funcargs', '--fixtures-per-test', '--import-mode', '--pdb', '--pdbcls', '--capture', '-s', '--runxfail', '--lf', '--last-failed', '--ff', '--failed-first', '--cache-show']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert', '--setuponly', '--setup-only']
Existing options in debugconfig : ['--basetemp', '--version', '-h', '--help', '-p', '--traceconfig', '--trace-config', '--debug', '-o', '--override-ini', '--assert', '--setuponly', '--setup-only', '--setupshow', '--setup-show']
usage: pyros_setup [options] [file_or_dir] [file_or_dir] [...]
pyros_setup: error: unrecognized arguments: --distro indigo
  inifile: None
  rootdir: /home/alexv/Projects/pyros-setup
(pyros-setup) alexv@asmodehn:~/Projects/pyros-setup/testing$ 

Seems we fail here, even before getting the conftest.py

Both commands were run in a virtualenv where the package was installed with pip install .

For reference, the code doing the py.test run is (in __main__.py):

def main():
    if len(sys.argv) > 1:
        if sys.argv[1] == '--pytest':
            import pytest  # import only if needed
            errno = pytest.main(['-s', '--pyargs', 'pyros_setup'] + sys.argv[2:])
            sys.exit(errno)

[asmodehn]

From memory, the behavior I described earlier (with conftest.py parsed two times) was happening when the package was installed with pip install -e .
So the editable / egg-link feature likely plays a role there...

[RonnyPfannschmidt]

now im confident im facing the same issue

[RonnyPfannschmidt]

@asmodehn i managed to revisit this one, and the isuse seems to be that the local conftest and the pyargs triggered conftest are mismatched wrt the fs location thus they are eligible for both types of import

[RonnyPfannschmidt]

after reading the custom sys.path setup im reasonably sure this is self inflicted, i suggest debugging with capure disabled and printing name of the conftests

[nicoddemus]

Closing for now; we will be happy to re-open this once we get more information.