Skip to content

xr.concat removes datetime information #6416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Illviljan opened this issue Mar 27, 2022 · 2 comments · Fixed by #6418
Closed

xr.concat removes datetime information #6416

Illviljan opened this issue Mar 27, 2022 · 2 comments · Fixed by #6418

Comments

@Illviljan
Copy link
Contributor

Illviljan commented Mar 27, 2022

What happened?

xr.concat removes datetime information and can't concatenate the arrays because they don't have compatible types anymore.

What did you expect to happen?

Succesful concatenation with the same type.

Minimal Complete Verifiable Example

import numpy as np
import xarray as xr
from datetime import datetime

month = np.arange(1, 13, 1)
data = np.sin(2 * np.pi * month / 12.0)

darray = xr.DataArray(data, dims=["time"])
darray.coords["time"] = np.array([datetime(2017, m, 1) for m in month])

darray_nan = np.nan * darray.isel(**{"time": -1})
darray = xr.concat([darray, darray_nan], dim="time")

Relevant log output

Traceback (most recent call last):

  File "<ipython-input-15-31040255a336>", line 2, in <module>
    darray = xr.concat([darray, darray_nan], dim="time")

  File "c:\users\j.w\documents\github\xarray\xarray\core\concat.py", line 244, in concat
    return f(

  File "c:\users\j.w\documents\github\xarray\xarray\core\concat.py", line 642, in _dataarray_concat
    ds = _dataset_concat(

  File "c:\users\j.w\documents\github\xarray\xarray\core\concat.py", line 555, in _dataset_concat
    combined_idx = indexes[0].concat(indexes, dim, positions)

  File "c:\users\j.w\documents\github\xarray\xarray\core\indexes.py", line 318, in concat
    coord_dtype = np.result_type(*[idx.coord_dtype for idx in indexes])

  File "<__array_function__ internals>", line 5, in result_type

TypeError: The DType <class 'numpy.dtype[datetime64]'> could not be promoted by <class 'numpy.dtype[int64]'>. This means that no common DType exists for the given inputs. For example they cannot be stored in a single array unless the dtype is `object`. The full list of DTypes is: (<class 'numpy.dtype[datetime64]'>, <class 'numpy.dtype[int64]'>)

Anything else we need to know?

Similar to #6384.

Happens around here:

yield PandasIndex([var.values.item()], dim)

Environment

INSTALLED VERSIONS

commit: None
python: 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:37:25) [MSC v.1916 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: ('Swedish_Sweden', '1252')
libhdf5: 1.10.6
libnetcdf: 4.7.4

xarray: 0.16.3.dev99+gc19467fb
pandas: 1.3.1
numpy: 1.21.5
scipy: 1.7.1
netCDF4: 1.5.6
pydap: installed
h5netcdf: 0.11.0
h5py: 2.10.0
Nio: None
zarr: 2.8.3
cftime: 1.5.0
nc_time_axis: 1.3.1
PseudoNetCDF: installed
rasterio: 1.2.6
cfgrib: None
iris: 3.0.4
bottleneck: 1.3.2
dask: 2021.10.0
distributed: 2021.10.0
matplotlib: 3.4.3
cartopy: 0.19.0.post1
seaborn: 0.11.1
numbagg: 0.2.1
fsspec: 2021.11.1
cupy: None
pint: 0.17
sparse: 0.12.0
setuptools: 49.6.0.post20210108
pip: 21.2.4
conda: None
pytest: 6.2.4
IPython: 7.31.0
sphinx: 4.3.2

@Illviljan Illviljan added bug needs triage Issue that has not been reviewed by xarray team member topic-indexing and removed needs triage Issue that has not been reviewed by xarray team member labels Mar 27, 2022
@Illviljan
Copy link
Contributor Author

cc @benbovy

@benbovy
Copy link
Member

benbovy commented Mar 28, 2022

Thanks @Illviljan for the report. Hopefully the last issue regarding this part of xr.concat internals.

-> #6418

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants