Cast fill value to array's dtype #2020
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
d-v-b
commented
Jul 9, 2024
dcherian
added a commit
to dcherian/zarr-python
that referenced
this pull request
Jul 25, 2024
* v3: (22 commits) chore: update pre-commit hooks (zarr-developers#2051) Apply ruff/flake8-bandit rule B006 (zarr-developers#2049) Move fixtures to `tests` (zarr-developers#1813) Multiple imports for an import name (zarr-developers#2047) Redundant list comprehension (zarr-developers#2048) chore: update pre-commit hooks (zarr-developers#2039) Cast fill value to array's dtype (zarr-developers#2020) chore: update pre-commit hooks (zarr-developers#2017) make shardingcodec pickleable (zarr-developers#2011) doc: copy 3.0.0.alpha changelog into release.rst (zarr-developers#2007) build(ci): enable python 3.12 in github actions (zarr-developers#2005) Bump NumPy to 2.0 (zarr-developers#1983) chore: update pre-commit hooks (zarr-developers#1989) Fix indexing with bools (zarr-developers#1968) Fix string interpolation (zarr-developers#1998) Unnecessary comprehension (zarr-developers#1997) Stop ignoring these ruff rules (zarr-developers#2001) Merge collapsible if statements (zarr-developers#1999) Unnecessary comprehension (zarr-developers#1996) Handle Path in `make_store_path` (zarr-developers#1992) ...
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In v3 right now, we don't apply any parsing or validation to the
fill_valueattribute of an array. This PR fixes that.In this PR, the
fill_valueis cast to an instance of the array's dtype, using normal numpy dtype casting semantics. I also added JSON serialization for complex dtypes, but this is not tested yet. I will add those tests as part of a later refactor of the metadata tests.For v2 arrays, I am importing the v2 fill value parsing function and wrapping it in a very light wrapper. This preserves all the v2 behavior.
I also added a
fill_valueattribute to theArrayclass. Happy to remove this if it's not supposed to be there.I put the metadata tests in a new directory structure:
tests/v3/metadata/test_v2.pyfor v2 stuff andtests/v3/metadata/test_v3.pyfor v3 stuff. This keeps the v2 and v3 logic separated, and removes the need to name functionstest_foo_v3and the like. We should ultimately use an analogous layout in the library code.Longer term we should support the raw bits datatypes defined in the zarr v3 spec, but I don't think we need that now.
TODO: