Silently eats DistributionNotFound errors

[matthijskooijman]

I was running into an issue with pluggy, because it eats DistributionNotFound errors. In my case, this prevented most of my installed pytest plugins from being used, without any indication as to why.

The root cause of the DistributionNotFound error was a missing dependency. I installed pytest from a [Pipfile.lock] that was (presumably) generated on a Unix system, where pytest does not depend on the colorama package. However, I was running on a Windows system, so pytest was installed without all of its dependencies. When pluggy tries to load pytest plugins, which (I presume) depend again on pytest, pkg_resources notices this missing dependency and raises a DistributionNotFound error, which is silently eaten by pluggy.

For reference, here is the traceback that's being eaten (by adding a traceback.print_exc() inside pluggy):

Traceback (most recent call last):
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pluggy\manager.py", line 254, in load_setuptools_entrypoints
    plugin = ep.load()
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pkg_resources\__init__.py", line 2320, in load
    self.require(*args, **kwargs)
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pkg_resources\__init__.py", line 2343, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pkg_resources\__init__.py", line 777, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'colorama; sys_platform == "win32"' distribution was not found and is required by pytest

Why does pluggy eat this error? I traced the history back to the pytest repo and the change was introduced in this commit: pytest-dev/pytest@1bc444d, but that does not really tell me why it was introduced....

[RonnyPfannschmidt]

@matthijskooijman its absolutely unclear why @hpk42 keept that "diaper" thing in

[nicoddemus]

It is unclear indeed. I propose that we drop the try/except and let the error propagate; I can't think we would ever want to silently discard plugin errors.

[RonnyPfannschmidt]

@nicoddemus just letting them bubble without some extra intervention in pytest will be a breaking api change

[nicoddemus]

Can we capture that in pytest ourselves and show a warning perhaps? Em dom, 19 de ago de 2018 04:01, Ronny Pfannschmidt < [email protected]> escreveu:

…

@nicoddemus <https://github.com/nicoddemus> just letting them bubble without some extra intervention in pytest will be a breaking api change — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#174 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABCO_LS8PDdcwpsrv0U7BicAyglNBOwtks5uSQ1cgaJpZM4V9Ji8> .

[matthijskooijman]

One problem with catching that error in pytest, is that the exception will have broken the loop, so any entry points that come after the problematic one, will be skipped.

[RonnyPfannschmidt]

just hit that since pip and setuptools normalize distribution names differently and a requirement that worked fine with pip triggered a hard setuptools failure in not finding the package name since pip finds foo_bar as foo.bar and setuptools doesnt

[khvn26]

Apparently pkg_resources swallows ImportErrors so should plugin code contain import errors, pytest will exit 1 in utter and complete silence. We use our own plugins located in the tested codebase in our CI pipelines and this issue has just caused a long and confusing troubleshooting session since our linter didn't catch the import errors.