-
Notifications
You must be signed in to change notification settings - Fork 234
Allow richer expressions in -n auto #477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
im -10 on this - not going to add a mini pseudo programming language into a option |
This can be done directly in shell, no reason to implement it in xdist.
There's probably similar for Powershell. |
I agree, closing as "won't fix" |
auto_detect_cpus is now a bit more complicated: pytest-xdist/src/xdist/plugin.py Lines 8 to 28 in 59caed8
pytest --numprocesses=$(python -c 'print(__import__("pytest.plugin").plugin.auto_detect_cpus() - 1)') |
that didn't work. here is a fixed version
|
nicoddemus
added a commit
to nicoddemus/pytest-xdist
that referenced
this issue
Aug 24, 2020
…hook This reverts the behavior we had before psutil was a hard dependency, but opens up the possibility of customization through the pytest_xdist_auto_num_workers hook, making things like pytest-dev#477 possible. Fix pytest-dev#585
nicoddemus
added a commit
to nicoddemus/pytest-xdist
that referenced
this issue
Aug 25, 2020
This makes using psutil optional and opens up the possibility of customization through the pytest_xdist_auto_num_workers hook, making things like pytest-dev#477 possible. Fix pytest-dev#585
toofar
added a commit
to toofar/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works.
toofar
added a commit
to toofar/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works.
toofar
added a commit
to toofar/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works.
toofar
added a commit
to qutebrowser/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works. Not sure about using "jobs" as a matrix arg because it kinda shadows the "jobs" keys at the top level. Maybe for the sake of making it easy to change it could be defined as an `env` value at the top level.
toofar
added a commit
to qutebrowser/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works. Not sure about using "jobs" as a matrix arg because it kinda shadows the "jobs" keys at the top level. Maybe for the sake of making it easy to change it could be defined as an `env` value at the top level.
toofar
added a commit
to qutebrowser/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works. Not sure about using "jobs" as a matrix arg because it kinda shadows the "jobs" keys at the top level. Maybe for the sake of making it easy to change it could be defined as an `env` value at the top level.
toofar
added a commit
to qutebrowser/qutebrowser
that referenced
this issue
May 25, 2024
From https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories runners have 4 cores (apart from macos-11). We could use `-n auto`, but I hypothesis that for the e2e tests since we have one worker process per test process maybe we don't want to actually use up all the cores with test processes? It doesn't look like it's possible to tell pytest-xdist to use all core but one, or half the cores, or anything (issue herehttps://github.com/pytest-dev/pytest-xdist/issues/477). So lets just set it to 2 for now which should be a lot faster than 1, if it works.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It would be nice if we could put things like
pytest -n auto-1
and have it correctly interpret that as "all CPUs except 1". This ensures that the processor is not hogged by tests, but also that it runs at a very fast clip.Could be done something like:
The text was updated successfully, but these errors were encountered: