Skip to content

Attributes are dropped after clip even if keep_attrs is True #3433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mraspaud opened this issue Oct 22, 2019 · 5 comments · Fixed by #4195
Closed

Attributes are dropped after clip even if keep_attrs is True #3433

mraspaud opened this issue Oct 22, 2019 · 5 comments · Fixed by #4195
Labels
topic-metadata Relating to the handling of metadata (i.e. attrs and encoding)

Comments

@mraspaud
Copy link
Contributor

MCVE Code Sample

import xarray as xr
import numpy as np

arr = xr.DataArray(np.ones((5, 5)), attrs={'units': 'K'})
xr.set_options(keep_attrs=True)
arr
# <xarray.DataArray (dim_0: 5, dim_1: 5)>
# array([[1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.]])
# Dimensions without coordinates: dim_0, dim_1
# Attributes:
#     units:    K

arr.clip(0, 1)
# <xarray.DataArray (dim_0: 5, dim_1: 5)>
# array([[1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.],
#        [1., 1., 1., 1., 1.]])
# Dimensions without coordinates: dim_0, dim_1

Expected Output

I would expect the attributes to be kept

Problem Description

keep_attrs set to True doesn't seem to be respected with the DataArray.clip method.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1062.1.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.6.2

xarray: 0.14.0
pandas: 0.25.1
numpy: 1.17.0
scipy: 1.3.0
netCDF4: 1.5.1.2
pydap: None
h5netcdf: 0.7.4
h5py: 2.10.0
Nio: None
zarr: 2.3.2
cftime: 1.0.3.4
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.0.28
cfgrib: None
iris: None
bottleneck: None
dask: 2.6.0
distributed: 2.6.0
matplotlib: 3.1.1
cartopy: 0.17.0
seaborn: None
numbagg: None
setuptools: 41.4.0
pip: 19.3
conda: None
pytest: 5.0.1
IPython: 7.8.0
sphinx: 2.2.0

@dcherian
Copy link
Contributor

Thanks @mraspaud

clip and other injected functions need to be updated to obey keep_attrs (also see #3348)

xarray/xarray/core/ops.py

Lines 321 to 349 in 72be873

def inject_all_ops_and_reduce_methods(cls, priority=50, array_only=True):
# prioritize our operations over those of numpy.ndarray (priority=1)
# and numpy.matrix (priority=10)
cls.__array_priority__ = priority
# patch in standard special operations
for name in UNARY_OPS:
setattr(cls, op_str(name), cls._unary_op(get_op(name)))
inject_binary_ops(cls, inplace=True)
# patch in numpy/pandas methods
for name in NUMPY_UNARY_METHODS:
setattr(cls, name, cls._unary_op(_method_wrapper(name)))
for name in PANDAS_UNARY_FUNCTIONS:
f = _func_slash_method_wrapper(getattr(duck_array_ops, name), name=name)
setattr(cls, name, cls._unary_op(f))
f = _func_slash_method_wrapper(duck_array_ops.around, name="round")
setattr(cls, "round", cls._unary_op(f))
if array_only:
# these methods don't return arrays of the same shape as the input, so
# don't try to patch these in for Dataset objects
for name in NUMPY_SAME_METHODS:
setattr(cls, name, _values_method_wrapper(name))
inject_reduce_methods(cls)
inject_cum_methods(cls)

Are you up for sending in a PR?

@mraspaud
Copy link
Contributor Author

Sure! What do you reckon the default should be ? False for backwards compatibility ?

@mraspaud
Copy link
Contributor Author

mraspaud commented Oct 23, 2019

ooh, maybe I answered too fast. I hadn't really looked at the code yet... but let's see:
if I understand correctly, in the function you cited, the functions get wrapped and include in the current cls. So I would need to fix the wrapper itself (_func_slash_method_wrapper) so that the attrs get copied and applied to the resulting array (if keep_attrs is True that is). However, do we have a guarantee that the wrapped functions are actually returning something that can have a .attrs ?

@mraspaud mraspaud changed the title Attributes are dropped after clip even is keep_attrs is True Attributes are dropped after clip even if keep_attrs is True Oct 23, 2019
@dcherian
Copy link
Contributor

I don't think so. You'll have to check which functions are used to inject methods on groupby, resample and rolling objects (at least)

@mraspaud
Copy link
Contributor Author

Ok, then I probably won't have the time to dig into this for now, sorry. If someone else with better knowledge of the code can work on this it would probably be for the best.

@dcherian dcherian added the topic-metadata Relating to the handling of metadata (i.e. attrs and encoding) label Jul 2, 2020
dcherian added a commit to dcherian/xarray that referenced this issue Jul 2, 2020
dcherian added a commit to dcherian/xarray that referenced this issue Aug 15, 2020
dcherian added a commit that referenced this issue Oct 14, 2020
* Propagate attrs with unary, binary functions

Closes #3490
Closes #4065
Closes #3433
Closes #3595

* Un xfail test

* bugfix

* Some progress. Still need keep_attrs in DataArray._unary_op

* Fix dataset attrs

* whats-new

* small fix

* Fix imag, real

* fix variable tests

* fix multiple return variables.

* review comments

* Update doc/whats-new.rst

* Propagate attrs with DataArray unary ops

* More tests

* Small cleanup

* Review comments.

* Fix duplication

Co-authored-by: Maximilian Roos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-metadata Relating to the handling of metadata (i.e. attrs and encoding)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants