How to specify ZarrSeparator

[stuarteberg]

By default, zarr chunk files use . as the axis separator in their names, e.g. 10.20.30. But in Python, one can use a NestedDirectoryStore to store in a directory tree, effectively using / as the separator, e.g. 10/20/30.

Unfortunately, the metadata in .zarray does not specify which format is used. (This is a known shortcoming of the zarr spec.)

Is there any way to tell neuroglancer which format a zarr array uses? I notice that the source code seems to support both, at least in theory. But I can't figure out how to tell neuroglancer that my zarr array uses the / separator in chunk names instead of the default . separator. Is there a way to do it?

[jbms]

You can use the key_encoding property:

https://google.github.io/tensorstore/tensorstore/driver/zarr/index.html

[stuarteberg]

Thanks very much for your reply. Sorry if I'm being dense, but I'm still confused. I'm trying to open the volume in neuroglancer. I can use either zarr or tensorstore to create the array on disk, but either way, the key_encoding setting is not stored in the metadata on disk, as far as I can tell.

Is there a way to somehow provide a tensorstore-style JSON volume configuration to neuroglancer? So far, I'm just providing a source in the form of zarr://http://localhost:9000/path/to/myarray.zarr. Are you saying neuroglancer also accepts tensorstore JSON data in that text control?

Again, sorry if that's a dumb question. I'm clearly missing something here.

[jbms]

Oh sorry, I didn't notice that you opened this issue on neuroglancer rather than tensorstore. Indeed, there is not at the moment support for specifying the separator. If you want to add support, I'd suggest making it a query string parameter, like

zarr://http://localhost:9000/path/to/myarray.zarr?separator=/

[joshmoore]

I will be looking at a PR for zarr-python at least which will use one of a small number of options for finding the key. Work on the Java side has taken place in bcdev/jzarr#17:

• heuristic that loops through chunks "0.0.0", "0/0/0", "0.0.1", "0/0/1" searching for existence... (with the downside being the overheaded of the search if no key_separator was chosen)
• uses optional metadata in the .zarray (with the downside that we need to agree on this metadata)
• ...