Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ Bug fixes
Documentation
~~~~~~~~~~~~~

- Clarify lazy behaviour and eager loading for ``chunks=None`` in :py:func:`~xarray.open_dataset`, :py:func:`~xarray.open_dataarray`, :py:func:`~xarray.open_datatree`, :py:func:`~xarray.open_groups` and :py:func:`~xarray.open_zarr` (:issue:`10612`, :pull:`10627`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.



Internal Changes
~~~~~~~~~~~~~~~~
Expand Down
24 changes: 16 additions & 8 deletions xarray/backends/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -578,8 +578,10 @@ def open_dataset(

- ``chunks="auto"`` will use dask ``auto`` chunking taking into account the
engine preferred chunks.
- ``chunks=None`` skips using dask, which is generally faster for
small arrays.
- ``chunks=None`` skips using dask. This uses xarray's internally private
:ref:`lazy indexing classes <internal design.lazy indexing>`,
but data is eagerly loaded into memory as numpy arrays when accessed.
This can be more efficient for smaller arrays or when large arrays are sliced before computation.
- ``chunks=-1`` loads the data with dask using a single chunk for all arrays.
- ``chunks={}`` loads the data with dask using the engine's preferred chunk
size, generally identical to the format's chunk size. If not available, a
Expand Down Expand Up @@ -819,8 +821,10 @@ def open_dataarray(

- ``chunks='auto'`` will use dask ``auto`` chunking taking into account the
engine preferred chunks.
- ``chunks=None`` skips using dask, which is generally faster for
small arrays.
- ``chunks=None`` skips using dask. This uses xarray's internally private
:ref:`lazy indexing classes <internal design.lazy indexing>`,
but data is eagerly loaded into memory as numpy arrays when accessed.
This can be more efficient for smaller arrays, though results may vary.
- ``chunks=-1`` loads the data with dask using a single chunk for all arrays.
- ``chunks={}`` loads the data with dask using engine preferred chunks if
exposed by the backend, otherwise with a single chunk for all arrays.
Expand Down Expand Up @@ -1044,8 +1048,10 @@ def open_datatree(

- ``chunks="auto"`` will use dask ``auto`` chunking taking into account the
engine preferred chunks.
- ``chunks=None`` skips using dask, which is generally faster for
small arrays.
- ``chunks=None`` skips using dask. This uses xarray's internally private
:ref:`lazy indexing classes <internal design.lazy indexing>`,
but data is eagerly loaded into memory as numpy arrays when accessed.
This can be more efficient for smaller arrays, though results may vary.
- ``chunks=-1`` loads the data with dask using a single chunk for all arrays.
- ``chunks={}`` loads the data with dask using the engine's preferred chunk
size, generally identical to the format's chunk size. If not available, a
Expand Down Expand Up @@ -1288,8 +1294,10 @@ def open_groups(

- ``chunks="auto"`` will use dask ``auto`` chunking taking into account the
engine preferred chunks.
- ``chunks=None`` skips using dask, which is generally faster for
small arrays.
- ``chunks=None`` skips using dask. This uses xarray's internally private
:ref:`lazy indexing classes <internal design.lazy indexing>`,
but data is eagerly loaded into memory as numpy arrays when accessed.
This can be more efficient for smaller arrays, though results may vary.
- ``chunks=-1`` loads the data with dask using a single chunk for all arrays.
- ``chunks={}`` loads the data with dask using the engine's preferred chunk
size, generally identical to the format's chunk size. If not available, a
Expand Down
6 changes: 4 additions & 2 deletions xarray/backends/zarr.py
Original file line number Diff line number Diff line change
Expand Up @@ -1370,8 +1370,10 @@ def open_zarr(

- ``chunks='auto'`` will use dask ``auto`` chunking taking into account the
engine preferred chunks.
- ``chunks=None`` skips using dask, which is generally faster for
small arrays.
- ``chunks=None`` skips using dask. This uses xarray's internally private
:ref:`lazy indexing classes <internal design.lazy indexing>`,
but data is eagerly loaded into memory as numpy arrays when accessed.
This can be more efficient for smaller arrays, though results may vary.
- ``chunks=-1`` loads the data with dask using a single chunk for all arrays.
- ``chunks={}`` loads the data with dask using engine preferred chunks if
exposed by the backend, otherwise with a single chunk for all arrays.
Expand Down
Loading