Skip to content

infer_freq not working (yet) with datetime64[s] #9998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
5 tasks done
kafitzgerald opened this issue Jan 29, 2025 · 2 comments · Fixed by #9999
Closed
5 tasks done

infer_freq not working (yet) with datetime64[s] #9998

kafitzgerald opened this issue Jan 29, 2025 · 2 comments · Fixed by #9999
Labels

Comments

@kafitzgerald
Copy link
Contributor

What happened?

We have upstream dev CI testing on geocat-comp that recently failed (NCAR/geocat-comp#681) because the dtype='datetime64[s]isn't handled byinfer_freq` yet in xarray. In particular this if statement looks like it needs to be generalized:

if dtype == "datetime64[ns]":

What did you expect to happen?

infer_freq to handle a broader range of dtypes

Minimal Complete Verifiable Example

import xarray as xr
import numpy as np
import pandas as pd

time = pd.to_datetime(['2020-01-01', '2020-01-02', '2020-01-04'])
non_uniform = xr.Dataset(data_vars={'data': (('time'), np.arange(3))}, coords={'time': time})
xr.infer_freq(non_uniform['time'])

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/katelynw/miniconda3/envs/geocat_comp_build/lib/python3.12/site-packages/xarray/coding/frequencies.py", line 96, in infer_freq
    index = CFTimeIndex(index.values)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/katelynw/miniconda3/envs/geocat_comp_build/lib/python3.12/site-packages/xarray/coding/cftimeindex.py", line 259, in __new__
    assert_all_valid_date_type(data)
  File "/Users/katelynw/miniconda3/envs/geocat_comp_build/lib/python3.12/site-packages/xarray/coding/cftimeindex.py", line 170, in assert_all_valid_date_type
    raise TypeError(
TypeError: CFTimeIndex requires cftime.datetime objects. Got object of <class 'numpy.datetime64'>.

Anything else we need to know?

Only happens with the dev versions of both pandas and xarray.

Environment

INSTALLED VERSIONS

commit: None
python: 3.12.8 | packaged by conda-forge | (main, Dec 5 2024, 14:19:53) [Clang 18.1.8 ]
python-bits: 64
OS: Darwin
OS-release: 21.6.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.4
libnetcdf: 4.9.2

xarray: 2025.1.2.dev19+g2bedca4f
pandas: 3.0.0.dev0+1871.gc0c778bdb7
numpy: 1.26.4
scipy: 1.14.1
netCDF4: 1.7.2
pydap: None
h5netcdf: None
h5py: None
zarr: None
cftime: 1.6.4
nc_time_axis: None
iris: None
bottleneck: 1.4.2
dask: 2025.1.0
distributed: 2025.1.0
matplotlib: 3.10.0
cartopy: 0.24.0
seaborn: None
numbagg: None
fsspec: 2024.12.0
cupy: None
pint: 0.24.4
sparse: None
flox: None
numpy_groupies: None
setuptools: 75.8.0
pip: 25.0
conda: None
pytest: 8.3.4
mypy: None
IPython: None
sphinx: None

@kafitzgerald kafitzgerald added bug needs triage Issue that has not been reviewed by xarray team member labels Jan 29, 2025
@TomNicholas TomNicholas removed the needs triage Issue that has not been reviewed by xarray team member label Jan 29, 2025
@kmuehlbauer
Copy link
Contributor

@kafitzgerald Thanks for the report. Indeed, we need to relax those checks.

Would you mind testing your workflow against #9999? Thanks!

@kafitzgerald
Copy link
Contributor Author

Thanks so much for the quick fix!

Working great now 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants