Some numcodecs.zarr3 codecs fail with 3.0.4+

[LDeakin]

Zarr version

3.0.4+

Numcodecs version

0.15.0

Python Version

3.12

Operating System

Linux

Installation

See reproducer

Description

The ZFPY and Pcodec codecs (and perhaps more) fail since 3.0.4 because they expect input arrays to be contiguous

Steps to reproduce

#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     # "zarr==3.0.2,<3.0.3", # WORKS
#     "zarr==3.0.4", # FAILS
#     "numcodecs==0.15.0",
#     "zfpy==1.0.1",
#     "pcodec==0.3.2",
# ]
# ///

import numpy as np
import zarr
from numcodecs.zarr3 import ZFPY, PCodec

for serializer in [
    ZFPY(mode = 4, tolerance=0.01),
    PCodec(level = 8, mode_spec="auto")
]:
    array = zarr.create_array(
        store=zarr.storage.MemoryStore(),
        shape=[2, 2],
        chunks=[2, 1],
        dtype=np.float32,
        serializer=serializer,
    )
    array[...] = np.array([[0, 1],[2, 3]])

Additional output

No response

[d-v-b]

thanks for this report. I see 2 problems here:

• the bug itself, which I cannot yet explain in terms of code changes that went in to the 3.04 release. Is it possible that this is somehow a side-effect of Fix a bug when setting complete chunks #2851?
• We are not testing enough. clearly we need to be testing against all codecs in numcodecs.

[mickyals]

Zarr Version

3.0.6

Numcodecs Version

0.15.1

Operating System

Linux

Description

The same issue as described previously - ValueError: an array with contiguous memory is required occurs due to PCodec expectation of contiguous arrays in memory. The result is corruption of the compressed arrays as shown below. Are there any updates regarding this bug?

[dstansby]

Yes, this issue bisects to 8b59a38. cc @dcherian who made that commit, do you have any insight into what a fix could be here?

[d-v-b]

@dcherian any guesses what could be going on here?

[dcherian]

Sorry for not responding. We are now better about not copying, so we are tripping up numcodecs

1. zfpy: https://github.com/zarr-developers/numcodecs/blob/725cf254aa42043dab5f4328342ef748765ca4f4/numcodecs/zfpy.py#L84C17-L87C18 . Here, I'm not sure if a copy meets the intention of the code, but it would make user experience smoother.
2. pcodec: https://github.com/zarr-developers/numcodecs/blob/725cf254aa42043dab5f4328342ef748765ca4f4/numcodecs/compat.py#L111 . We will need to add a arr.copy for that branch.