remove py._pydir as rsync candidate #823
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
asottile
force-pushed
the
remove-pydir
branch
from
63ff6a6 to
42024f3
Compare
nicoddemus
approved these changes
Oct 22, 2022
Can you elaborate @RonnyPfannschmidt ? I don't know what |
|
Basically this ensures a blank remote python has a working pytest If pylib is available it has to be copied, same for the py file new pytest ships |
|
What about all the other pytest dependencies, such as (Sorry I'm not familiar with the rsync implementation in xdist at all) |
|
It's possible that the current tests don't correctly capture the requirements in a way that finds breakage If that's the case we should evaluate that and consider dropping rsync over fixing it |
Closed
|
Created #825 then, let me know if you are OK with it, otherwise feel free to post more information there if needed. |
|
Sounds good, I'll give it a try later |
|
Cool, thanks. |
folex
referenced
this pull request
in fluencelabs/spell
Jun 6, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [pytest-xdist](https://togithub.com/pytest-dev/pytest-xdist) ([changelog](https://pytest-xdist.readthedocs.io/en/latest/changelog.html)) | `==2.5.0` -> `==3.3.1` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>pytest-dev/pytest-xdist</summary> ### [`v3.3.1`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-331-2023-05-19) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.3.0...v3.3.1) \=============================== ## Bug Fixes - `#​907 <https://github.com/pytest-dev/pytest-xdist/issues/907>`\_: Avoid remote calls during startup as `execnet` by default does not ensure remote affinity with the main thread and might accidentally schedule the pytest worker into a non-main thread, which breaks numerous frameworks, for example `asyncio`, `anyio`, `PyQt/PySide`, etc. A more safe correction will require thread affinity in `execnet` (`pytest-dev/execnet#​96 <https://github.com/pytest-dev/execnet/issues/96>`\__). ### [`v3.3.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-330-2023-05-12) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.2.1...v3.3.0) \=============================== ## Features - `#​555 <https://github.com/pytest-dev/pytest-xdist/issues/555>`\_: Improved progress output when collecting nodes to be less verbose. ### [`v3.2.1`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-321-2023-03-12) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.2.0...v3.2.1) \=============================== ## Bug Fixes - `#​884 <https://github.com/pytest-dev/pytest-xdist/issues/884>`\_: Fixed hang in `worksteal` scheduler. ### [`v3.2.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-320-2023-02-07) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.1.0...v3.2.0) \=============================== ## Improved Documentation - `#​863 <https://github.com/pytest-dev/pytest-xdist/issues/863>`\_: Document limitations for debugging due to standard I/O of workers not being forwarded. Also, mention remote debugging as a possible workaround. ## Features - `#​855 <https://github.com/pytest-dev/pytest-xdist/issues/855>`\_: Users can now configure `load` scheduling precision using `--maxschedchunk` command line option. - `#​858 <https://github.com/pytest-dev/pytest-xdist/issues/858>`*: New `worksteal` scheduler, based on the idea of `work stealing <https://en.wikipedia.org/wiki/Work_stealing>`*. It's similar to `load` scheduler, but it should handle tests with significantly differing duration better, and, at the same time, it should provide similar or better reuse of fixtures. ## Trivial Changes - `#​870 <https://github.com/pytest-dev/pytest-xdist/issues/870>`\_: Make the tests pass even when `$PYTEST_XDIST_AUTO_NUM_WORKERS` is set. ### [`v3.1.0`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-310-2022-12-01) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v3.0.2...v3.1.0) \=============================== ## Features - `#​789 <https://github.com/pytest-dev/pytest-xdist/issues/789>`\_: Users can now set a default distribution mode in their configuration file: .. code-block:: ini [pytest] addopts = --dist loadscope - `#​842 <https://github.com/pytest-dev/pytest-xdist/issues/842>`\_: Python 3.11 is now officially supported. ## Removals - `#​842 <https://github.com/pytest-dev/pytest-xdist/issues/842>`\_: Python 3.6 is no longer supported. ### [`v3.0.2`](https://togithub.com/pytest-dev/pytest-xdist/blob/HEAD/CHANGELOG.rst#pytest-xdist-302-2022-10-25) [Compare Source](https://togithub.com/pytest-dev/pytest-xdist/compare/v2.5.0...v3.0.2) \=============================== ## Bug Fixes - `#​813 <https://github.com/pytest-dev/pytest-xdist/issues/813>`\_: Cancel shutdown when a crashed worker is restarted. ## Deprecations - `#​825 <https://github.com/pytest-dev/pytest-xdist/issues/825>`\_: The `--rsyncdir` command line argument and `rsyncdirs` config variable are deprecated. The rsync feature will be removed in pytest-xdist 4.0. - `#​826 <https://github.com/pytest-dev/pytest-xdist/issues/826>`\_: The `--looponfail` command line argument and `looponfailroots` config variable are deprecated. The loop-on-fail feature will be removed in pytest-xdist 4.0. ## Improved Documentation - `#​791 <https://github.com/pytest-dev/pytest-xdist/issues/791>`\_: Document the `pytest_xdist_auto_num_workers` hook. - `#​796 <https://github.com/pytest-dev/pytest-xdist/issues/796>`\_: Added known limitations section to documentation. - `#​829 <https://github.com/pytest-dev/pytest-xdist/issues/829>`\_: Document the `-n logical` option. ## Features - `#​792 <https://github.com/pytest-dev/pytest-xdist/issues/792>`\_: The environment variable `PYTEST_XDIST_AUTO_NUM_WORKERS` can now be used to specify the default for `-n auto` and `-n logical`. - `#​812 <https://github.com/pytest-dev/pytest-xdist/issues/812>`\_: Partially restore old initial batch distribution algorithm in `LoadScheduling`. pytest orders tests for optimal sequential execution - i. e. avoiding unnecessary setup and teardown of fixtures. So executing tests in consecutive chunks is important for optimal performance. In v1.14, initial test distribution in `LoadScheduling` was changed to round-robin, optimized for the corner case, when the number of tests is less than `2 * number of nodes`. At the same time, it became worse for all other cases. For example: if some tests use some "heavy" fixture, and these tests fit into the initial batch, with round-robin distribution the fixture will be created `min(n_tests, n_workers)` times, no matter how many other tests there are. With the old algorithm (before v1.14), if there are enough tests not using the fixture, the fixture was created only once. So restore the old behavior for typical cases where the number of tests is much greater than the number of workers (or, strictly speaking, when there are at least 2 tests for every node). ## Removals - `#​468 <https://github.com/pytest-dev/pytest-xdist/issues/468>`\_: The `--boxed` command-line option has been removed. If you still need this functionality, install `pytest-forked <https://pypi.org/project/pytest-forked>`\__ separately. ## Trivial Changes - `#​468 <https://github.com/pytest-dev/pytest-xdist/issues/468>`\_: The `py` dependency has been dropped. - `#​822 <https://github.com/pytest-dev/pytest-xdist/issues/822>`\_: Replace internal usage of `py.log` with a custom solution (but with the same interface). - `#​823 <https://github.com/pytest-dev/pytest-xdist/issues/823>`\_: Remove usage of `py._pydir` as an rsync candidate. - `#​824 <https://github.com/pytest-dev/pytest-xdist/issues/824>`\_: Replace internal usages of `py.path.local` by `pathlib.Path`. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/fluencelabs/spell). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS40MC4wIiwidXBkYXRlZEluVmVyIjoiMzUuOTUuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
afaict this shouldn't matter since this only supports
pytestnow (and that root is included)