Skip to content

remove the pytest_namespace hook in pytest itself without removing support in general #2280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

RonnyPfannschmidt
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt commented Feb 28, 2017

motivation: the pytest namespace hook requires to import the plugin system at import time in a unreasonably bad manner just to monkeypatch the namespace

this makes the pytest startup quite a mess and makes details of xdist unreliable and unfixable wrt initial node configuration

additionally the pytest_namespace hook is absolutely and completely broken for nested pytest setups that happen when testing pytest plugins

@RonnyPfannschmidt RonnyPfannschmidt changed the base branch from master to features February 28, 2017 13:11
@coveralls
Copy link

Coverage Status

Coverage increased (+0.003%) to 92.775% when pulling 5796182 on RonnyPfannschmidt:namespaces-hook into 0f3d7ac on pytest-dev:features.

@RonnyPfannschmidt RonnyPfannschmidt changed the title [wip] remove the pytest_namespace hook in pytest itself without removing support in general remove the pytest_namespace hook in pytest itself without removing support in general Mar 1, 2017
@RonnyPfannschmidt RonnyPfannschmidt changed the title remove the pytest_namespace hook in pytest itself without removing support in general [wip] remove the pytest_namespace hook in pytest itself without removing support in general Mar 4, 2017
Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with the motivation behind this PR!

Aside from a few comments I'm OK with the PR as well, just have to fix CI.

if config.option.strict:
pytest.mark._config = config
MARK_GEN._config = config
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will only support two levels, but I think it is enough (pytest "main" calling a test which uses a "testdir")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicoddemus thats a bug i only half-fixed - pytest-unconfigure unsets, pytest_configure sets, whats needed is a better wrapper

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a stack then should do the trick, if you think it is worthwhile.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still plan to tackle this @RonnyPfannschmidt?

@@ -335,3 +339,6 @@ def add_mark(self, mark):
def __iter__(self):
""" yield MarkInfo objects each relating to a marking-call. """
return imap(MarkInfo, self._marks)


MARK_GEN = MarkGenerator()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps mark_generator_factory would be better than MarkGenerator?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i#d love a better name for that object in general, nothing good comes to mind tho

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I wrote that wrong, it was supposed to be:

perhaps mark_generator_factory would be better than MARK_GEN?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i see, we can def rename the class and the constant

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I was proposing just the variable:

mark_generator = MarkGenerator()

But if you are willing, I think "factory" conveys a better name here (that word is also used in the docstring in fact). So perhaps:

mark_factory = MarkFactory()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still plan to tackle this @RonnyPfannschmidt?

)


def _setup_collect_fakemodule():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easier to just have a collect.py module which imports the symbols then, no? Less magic here would be better IMHO. Some tools might be confused/fail/warn that they can't find the file for pytest.collect, for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicoddemus that breaks python -m pytest on python2.6 - else i could just make it a file, move the main hack and be happy

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear, just creating _pytest/collect.py which imports those simbols, and add a line from _pytest import collect in pytest.py breaks Python2.6? Just trying to clarify in case I didn't make my point across.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i misunderstood, as turning pytest into a package and putting a collect filr there

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still plan to tackle this @RonnyPfannschmidt?

@flub
Copy link
Member

flub commented Mar 4, 2017

What would the effect be if the pytest_namespace hook got called later, i.e. right at the beginning of the pytest_configure call? Would that solve the initialisation problems as well?

@RonnyPfannschmidt
Copy link
Member Author

@flub then it would no longer work as intended and break users

personally i just want it gone because it cant be implemented correctly while retaining a sane life-cycle for the pluginmanager instances

@flub
Copy link
Member

flub commented Mar 5, 2017

Sure, I understand your motivation. I was thinking as a way forward for plugin authors who still want to do this, we would not be able to stop them doing this in pytest_configure anyway so why not move where it gets invoked instead of completely removing it once the deprecation cycle is gone. OTOH that could just cause more confusion, but my guess is that most plugins won't notice or care that this hook would be called slightly later.

@RonnyPfannschmidt
Copy link
Member Author

@flub with this pr i dont want to go about making it unusable or moving it, i just want to make pytest itself stop needing it

wrt moving it to pytest_configure, unless we do remove the added elements in pytest_unconfigure, it will just be a different mess, and im not willing to write the correct version of this because its error prone and complex - the gains don't justify the cost to make this work correct

@flub
Copy link
Member

flub commented Mar 5, 2017

@RonnyPfannschmidt Yeah, fair enough

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.2%) to 92.557% when pulling 3d36d57 on RonnyPfannschmidt:namespaces-hook into 0f3d7ac on pytest-dev:features.

@RonnyPfannschmidt RonnyPfannschmidt changed the title [wip] remove the pytest_namespace hook in pytest itself without removing support in general remove the pytest_namespace hook in pytest itself without removing support in general Mar 15, 2017
Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.09%) to 92.563% when pulling e4d6af1 on RonnyPfannschmidt:namespaces-hook into 5482dfe on pytest-dev:features.

modfile.strpath,
])
if res:
pytest.fail("command result %s" % res)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a test that ensures the official API works:

def test_pytest_api():
    import pytest
    pytest.Module
    pytest.Item
    ...

This will ensure we are not breaking anything now or in the future (it is easy for someone to remove an unused import somewhere and break plugins by accident).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

__all__ and flake8 will find that one

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.09%) to 92.563% when pulling ffc26af on RonnyPfannschmidt:namespaces-hook into 5482dfe on pytest-dev:features.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.09%) to 92.563% when pulling ffc26af on RonnyPfannschmidt:namespaces-hook into 5482dfe on pytest-dev:features.

@RonnyPfannschmidt RonnyPfannschmidt deleted the namespaces-hook branch March 16, 2017 18:32
@RonnyPfannschmidt
Copy link
Member Author

due to a git push misstake i killed this pr, will reopen a new

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants