Skip to content

reduce() by multiple dims on groupby object #3402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tlogan2000 opened this issue Oct 15, 2019 · 2 comments · Fixed by #3403
Closed

reduce() by multiple dims on groupby object #3402

tlogan2000 opened this issue Oct 15, 2019 · 2 comments · Fixed by #3403

Comments

@tlogan2000
Copy link

tlogan2000 commented Oct 15, 2019

MCVE Code Sample

# Your code here
import xarray as xr
import numpy as np
url = 'https://data.nodc.noaa.gov/thredds/dodsC/GCOS/monthly_five_degree/19810101-NODC-L3_GHRSST-SSTblend-GLOB_HadSST2-Monthly_FiveDeg_DayNitAvg_19810101_20071231-v01.7-fv01.0.nc'

ds = xr.open_dataset(url, chunks=dict(time=12))
# reduce() directly on dataArray - THIS IS OK
ds.analysed_sst.reduce(np.percentile, dim=('lat','lon'), q=0.5) # ok

# Group by example
rr = ds.analysed_sst.rolling(min_periods=1, center=True, time=5).construct("window")

g = rr.groupby("time.dayofyear")
print(g.dims)
test1d = g.reduce(np.percentile, dim=('time'), q=0.5) # ok
testall = g.reduce(np.percentile, dim=xr.ALL_DIMS, q=0.5) # ok

# .reduce() w/ 2dims  on grouby obj not working 
test2d = g.reduce(np.nanpercentile, dim=('time','window'), q=0.5)

Expected Output

reduced output performed over multiple dimensions (but not xr.ALL_DIMS) on a groupby object

Problem Description

Using .reduce() on a groupby object is only successful when given a single dimensions or by using xr.ALL_DIMS. I wish to apply a reduce on a subset of dims (last line of code above) but gives folowing error:

Traceback (most recent call last):
  File "/home/travis/.PyCharmCE2019.2/config/scratches/scratch_20.py", line 13, in <module>
    test = g.reduce(np.percentile, dim=('time','window'), q=0.5)
  File "/home/travis/.conda/envs/Xarray/lib/python3.7/site-packages/xarray/core/groupby.py", line 800, in reduce
    % (dim, self.dims)
ValueError: cannot reduce over dimension ('time', 'window'). expected either xarray.ALL_DIMS to reduce over all dimensions or one or more of ('time', 'lat', 'lon', 'window').

Note: Using reduce() on a subset of dims directly on a xr.DataArray seems fine (line 7).

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-65-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_CA.UTF-8 LOCALE: en_CA.UTF-8 libhdf5: 1.10.2 libnetcdf: 4.6.3 xarray: 0.14.0 pandas: 0.25.1 numpy: 1.17.2 scipy: 1.3.1 netCDF4: 1.5.2 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.3.4 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: 2.4.0 distributed: 2.4.0 matplotlib: 3.1.1 cartopy: None seaborn: None numbagg: None setuptools: 41.0.1 pip: 19.2.2 conda: None pytest: 5.0.1 IPython: None sphinx: None
@dcherian
Copy link
Contributor

🤦‍♂️ I can't believe we don't test for this. Fix is incoming.

dcherian added a commit to dcherian/xarray that referenced this issue Oct 15, 2019
dcherian added a commit to dcherian/xarray that referenced this issue Oct 15, 2019
dcherian added a commit to dcherian/xarray that referenced this issue Oct 15, 2019
@tlogan2000
Copy link
Author

Thx for the quick reply. For info this was working fine with v.0.13.0... it only is occuring since the 0.14.0 upgrade

dcherian added a commit that referenced this issue Oct 25, 2019
* Another groupby.reduce bugfix.

Fixes #3402

* Add whats-new.

* Use is_scalar instead

* bugfix

* fix whats-new

* Update xarray/core/groupby.py

Co-Authored-By: Maximilian Roos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants