Skip to content

xarray.open_zarr to be deprecated  #70

Closed
@Mikejmnez

Description

@Mikejmnez

In the future, zarr files/stores will be opened with xarray.open_dataset as follows

ds = xarray.open_dataset(store, engine="zarr", ...)

Thus, (eventually) there needs to be a change on how intake-xarray will open zarr stores. Naively, this can be done by specifying open_dataset rather than open_zarr. This is, something like:

store = 'directoryA/subdirectory/zarr_store
self._mapper = fsspec.get_mapper(store, ...) 
_open = xr.open_dataset
self._ds = open(self._mapper, engine="zarr", ...)

However, xarray.open_dataset does not recognize output from fsspec.get_mapper. It works if store (as defined above) is passed.

On xarray.open_zarr, _mapper gets transformed into a ZarrStore and later decoded. This is, given the _mapper, the following will open the zarr store:

from xarray.backends.zarr import ZarrStore
from xarray import conventions
zarr_store = ZarrStore.open_group(_mapper, ...)
ds = conventions.decode_cf(zarr_store,...)

This brings two options IMHO:

  1. Drop using fsspec.get_mapper (not likely) and just pass the url/path as argument to xarray.open_dataset. (very unlikely), or
  2. Follow along the lines of the pseudo code above, and rather than import/use xarray.open_dataset directly, import ZarrStore.open_group and convenctions.decode_cf to open zarr stores.

It is my understanding, that zarr will potentially depend more on fsspec as it gets more developed, and thus No. 2 seems more likely.

Or, is there another secret option number 3 I fail to see?

pydata/xarray#4003
fsspec/filesystem_spec#286
zarr-developers/zarr-python#546

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions