write_empty_chunks ignored in dataset.to_zarr()

[niklaspfeffer]

What happened?

Even though it was fixed in #10177, write_empty_chunks seems to be ignored. When I write to an by xarray created zarr (no matter if a zarr format 2 or 3) with a defined fill_value of 0, a completely 0 filled chunk is not written while a chunk with data is written. Setting write_empty_chunks to True does not change this behaviour.

What did you expect to happen?

An empty chunk should be created, but there is None.

Minimal Complete Verifiable Example

import dask.array
import xarray as xr
import numpy as np

path = "test.zarr"

# create empty zarr
dummies = dask.array.zeros(30, chunks=10)
ds = xr.Dataset({"foo": ("x", dummies)}, coords={"x": np.arange(30)})
ds.to_zarr(path, compute=False)

# create dataset containing only fill_value
random_data = np.zeros((10))
ds = xr.Dataset({"foo": ("x", random_data)}, coords={"x": np.arange(10)})

# should write empty chunks, but foo has no written chunks afterwards
ds.to_zarr(path, region={'x': slice(0,10)}, write_empty_chunks=True)

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• New issue — a search of GitHub Issues suggests this is not a duplicate.
• Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None
python: 3.12.3 | packaged by Anaconda, Inc. | (main, May 6 2024, 19:46:43) [GCC 11.2.0]
python-bits: 64
OS: Linux
OS-release: 6.8.0-65-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.4
libnetcdf: None

xarray: 2025.7.1
pandas: 2.2.3
numpy: 2.3.2
scipy: 1.15.1
netCDF4: None
pydap: None
h5netcdf: 1.5.0
h5py: 3.12.1
zarr: 3.1.1
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: 2025.1.0
distributed: None
matplotlib: 3.9.2
cartopy: None
seaborn: 0.13.2
numbagg: None
fsspec: 2025.3.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 80.9.0
pip: 25.0.1
conda: 24.4.0
pytest: 8.3.2
mypy: 1.10.1
IPython: 8.26.0
sphinx: None
None

[ianhi]

This might be a bug in zarr. I think this script using only zarr replicates essnetially the bug:

#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
#     "numpy",
# ]
# ///

import os

import numpy as np
import zarr

# Create array
store = zarr.open_group("test.zarr", mode="w")
arr = store.create(
    "data",
    shape=(20,),
    chunks=(10,),
    dtype="f8",
    fill_value=0.0,
    config={"write_empty_chunks": True},
)

# Reopen (like xarray does with regions)
store = zarr.open_group("test.zarr", mode="r+")
arr = zarr.open(store=store.store, path="data", config={"write_empty_chunks": True})

# Write zeros - should create chunk but doesn't
arr[0:10] = np.zeros(10)

# Check what was written
chunk_dir = "test.zarr/data/c"
chunks = (
    [f for f in os.listdir(chunk_dir) if f.isdigit()]
    if os.path.exists(chunk_dir)
    else []
)
print(f"Chunks written: {sorted(chunks)}")
print("Expected: ['0'] (chunk with zeros should be written)")

[ianhi]

Should be fixed by: zarr-developers/zarr-python#3378

thanks for the helpful report @niklaspfeffer ! you made it very easy to debug

[ianhi]

This is now fixed upstream:

#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
#     "xarray",
#     "dask",
# ]
# ///
import os

import dask.array
import numpy as np

import xarray as xr

path = "test.zarr"

# create empty zarr
dummies = dask.array.zeros(30, chunks=10)
ds = xr.Dataset({"foo": ("x", dummies)}, coords={"x": np.arange(30)})
ds.to_zarr(path, mode="w", compute=False)

# create dataset containing only fill_value
random_data = np.zeros(10)
ds = xr.Dataset({"foo": ("x", random_data)}, coords={"x": np.arange(10)})

ds.to_zarr(path, region={"x": slice(0, 10)}, write_empty_chunks=True)

# Check if chunks were written
chunk_dir = os.path.join(path, "foo", "c")
if os.path.exists(chunk_dir):
    chunks = [f for f in os.listdir(chunk_dir) if f.isdigit()]
    print(f"Chunks found in {chunk_dir}: {chunks}")
    if chunks:
        print("✅ fixed on Zarr Main")
    if not chunks:
        print("❌ no chunks written")
else:
    print(f"  -> No chunks directory found at {chunk_dir}")