Skip to content

Default to a RemoteStore for (some) string StoreLike #2195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TomAugspurger opened this issue Sep 17, 2024 · 1 comment · Fixed by #2198
Closed

Default to a RemoteStore for (some) string StoreLike #2195

TomAugspurger opened this issue Sep 17, 2024 · 1 comment · Fixed by #2198
Labels
bug Potential issues with the zarr-python library

Comments

@TomAugspurger
Copy link
Contributor

Zarr version

v3

Numcodecs version

n/a

Python Version

n/a

Operating System

n/a

Installation

n/a

Description

Zarr v2 users (either direct, or indirect through libraries like xarray) probably relied on the ability to pass an fsspec URI to zarr.open and have it work with a remote file system. Is there any tolerance for trying to interpret some string store arguments as an argument to RemoteStore rather than LocalFile? For example:

zarr.open_group("s3://hrrrzarr")

would be interpreted as

zarr.open_group(zarr.store.RemoteStore("s3://hrrrzarr"))

We'd also need to expose storage_options in all the API endpoints that create a Store / StorePath from user input, to allow customizing the filesystem that's created.

Steps to reproduce

import zarr

g = zarr.open_group(store="s3://hrrrzarr", storage_options={"anon": False})

Additional output

No response

@TomAugspurger TomAugspurger added the bug Potential issues with the zarr-python library label Sep 17, 2024
@d-v-b
Copy link
Contributor

d-v-b commented Sep 17, 2024

Is there any tolerance for trying to interpret some string store arguments as an argument to RemoteStore rather than LocalFile? For example:

Definitely! In the short term, we should copy what v2 was doing, modulo the storage classes. Longer term, I would like to consider a few things in this direction:

  • expose a URI attribute on stores, groups and arrays.
  • set up a scheme: store class registry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants