Skip to content

Issue with xarray and zarr when dimensions start with "meta" #1105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zndr27 opened this issue Aug 2, 2022 · 5 comments · Fixed by #1123
Closed

Issue with xarray and zarr when dimensions start with "meta" #1105

zndr27 opened this issue Aug 2, 2022 · 5 comments · Fixed by #1123
Labels
bug Potential issues with the zarr-python library

Comments

@zndr27
Copy link

zndr27 commented Aug 2, 2022

Zarr version

2.12.0

Numcodecs version

0.10.1

Python Version

3.8.10, 3.10.5

Operating System

Linux

Installation

pip

Description

Currently I'm working with xarray for MRSI analysis. I have been using xarray datasets with one the dimensions labeled as "metabolite." Previously I have been able to save and load the data to zarr with no issues using xr.Dataset.to_zarr and xr.open_zarr.

Currently I'm getting an issue where I get an error raised by zarr that complains about this dimension starting with "meta." I think this may be due to a new version of zarr. I have copied the error below.

When I changed the dimension name (i.e. the zarr subfolder and .zmetadata file) from "metabolite" to something that doesn't start with "meta" then I can load the data properly.

Someone may need to modify the xr.Dataset.to_zarr and xr.open_zarr functions in case an xarray user decides to create a dimension that starts with "meta" and wants to save their dataset to zarr.

Please let me know if you need to send any additional info.

Steps to reproduce

See description. Here's a link to the bug report I posted on xarray's github page: pydata/xarray#6853

Additional output

No response

@zndr27 zndr27 added the bug Potential issues with the zarr-python library label Aug 2, 2022
@joshmoore
Copy link
Member

Thanks, @zndr27. I assume there needs to be a separator included in:

assert not path.startswith("meta")

like "meta/". cc: @grlee77

@tasansal
Copy link
Contributor

We also saw this issue because we had our internal "metadata" group.
Currently, we limited the upper bound for the zarr dependency to <2.12 until there is a solution.

What are the reserved prefixes for the new v3 spec that users should be aware of?
Maybe this also warrants a documentation piece as well?

@grlee77
Copy link
Contributor

grlee77 commented Aug 16, 2022

Yes, I can make a PR to allow names like metabolite. The intention was not to prevent these, so I think we just need to fix it in the next patch release.

@zndr27
Copy link
Author

zndr27 commented Aug 27, 2022

Just wanted to check if your pull request has been accepted. When is the next patch release supposed to happen?

@grlee77
Copy link
Contributor

grlee77 commented Sep 6, 2022

Hi @zndr27, thanks for checking in. I just made a PR with a fix today (#1123). I think a v2.13 release is planned to happen fairly soon, so hopefully it will make it in for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants