Remove test_dask_layers_and_dependencies
#10242
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks! The test failures are associated with the new Zarr release. |
dcherian
added a commit
that referenced
this pull request
Apr 27, 2025
* main: (76 commits) Update how-to-add-new-backend.rst (#10240) Support extension array indexes (#9671) Switch documentation to pydata-sphinx-theme (#8708) Bump codecov/codecov-action from 5.4.0 to 5.4.2 in the actions group (#10239) Fix mypy, min-versions CI, xfail Zarr tests (#10255) Remove `test_dask_layers_and_dependencies` (#10242) Fix: Docs generation create temporary files that are not cleaned up. (#10238) opendap / dap4 support for pydap backend (#10182) Add RangeIndex (#10076) Fix mypy (#10232) Fix doctests (#10230) Fix broken Sphinx Roles (#10225) `DatasetView.map` fix `keep_attrs` (#10219) Add datatree repr asv (#10214) CI: Automatic PR labelling is back (#10201) Fixes dimension order in `xarray.Dataset.to_stacked_array` (#10205) Fix references to core classes in docs (#10207) Update pre-commit hooks (#10208) add `scipy-stubs` as extra `[types]` dependency (#10202) Fix sparse dask repr test (#10200) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This test is intended to break after dask/dask#11881
What is this test testing?
This test was implemented as part of #2603 at a time when HighLevelGraphs were just introduced. The test itself ensures pretty much that if a collection is passed to a Delayed objects, the underlying dask graph is a superset of the graph of the original collection. In different terms, the delayed is only adding additional tasks but is keeping the original graph as is.
Why is this now different?
One of the changes in dask/dask#11881 is that it now respects optimization of the collection when passed to the delayed function. In earlier dask versions, the collection would've been passed (and be computed) unoptimized, i.e. no culling, no fusion, no slicing, just the raw graph. That is not only inconsistent but it also created many (performance) problems in the past that should now be gone.
Instead of the test on dask internals, I replaced this test with a simple version that checks that we can indeed pass a
Datasetobject to a delayed object and get the expected result.