Skip to content

.sel method gives error with float32 values #3137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hdsingh opened this issue Jul 17, 2019 · 8 comments · Fixed by #3153
Closed

.sel method gives error with float32 values #3137

hdsingh opened this issue Jul 17, 2019 · 8 comments · Fixed by #3153

Comments

@hdsingh
Copy link

hdsingh commented Jul 17, 2019

.sel method gives error when it is used to select float32 values however it works fine for float64
Example:

import xarray as xr
import numpy as np

a = np.asarray([0.    , 0.111, 0.222, 0.333], dtype='float32')
ds = xr.Dataset(coords={'a': (['a'],a )})
ds.a.sel({'a': 0.111})

Error Traceback:

KeyError                                  Traceback (most recent call last)
~/anaconda3/envs/check3/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2656             try:
-> 2657                 return self._engine.get_loc(key)
   2658             except KeyError:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Float64HashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Float64HashTable.get_item()

KeyError: 0.111

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-7-b4190c1d9580> in <module>
      4 a = np.asarray([0.    , 0.111, 0.222, 0.333], dtype='float32')
      5 ds = xr.Dataset(coords={'a': (['a'],a )})
----> 6 ds.a.sel({'a': 0.111})
      7 ds.a.sel({'a': 0.111}, method= 'nearest')

~/anaconda3/envs/check3/lib/python3.7/site-packages/xarray/core/dataarray.py in sel(self, indexers, method, tolerance, drop, **indexers_kwargs)
    853         ds = self._to_temp_dataset().sel(
    854             indexers=indexers, drop=drop, method=method, tolerance=tolerance,
--> 855             **indexers_kwargs)
    856         return self._from_temp_dataset(ds)
    857 

~/anaconda3/envs/check3/lib/python3.7/site-packages/xarray/core/dataset.py in sel(self, indexers, method, tolerance, drop, **indexers_kwargs)
   1729         indexers = either_dict_or_kwargs(indexers, indexers_kwargs, 'sel')
   1730         pos_indexers, new_indexes = remap_label_indexers(
-> 1731             self, indexers=indexers, method=method, tolerance=tolerance)
   1732         result = self.isel(indexers=pos_indexers, drop=drop)
   1733         return result._overwrite_indexes(new_indexes)

~/anaconda3/envs/check3/lib/python3.7/site-packages/xarray/core/coordinates.py in remap_label_indexers(obj, indexers, method, tolerance, **indexers_kwargs)
    315 
    316     pos_indexers, new_indexes = indexing.remap_label_indexers(
--> 317         obj, v_indexers, method=method, tolerance=tolerance
    318     )
    319     # attach indexer's coordinate to pos_indexers

~/anaconda3/envs/check3/lib/python3.7/site-packages/xarray/core/indexing.py in remap_label_indexers(data_obj, indexers, method, tolerance)
    250         else:
    251             idxr, new_idx = convert_label_indexer(index, label,
--> 252                                                   dim, method, tolerance)
    253             pos_indexers[dim] = idxr
    254             if new_idx is not None:

~/anaconda3/envs/check3/lib/python3.7/site-packages/xarray/core/indexing.py in convert_label_indexer(index, label, index_name, method, tolerance)
    179                 indexer, new_index = index.get_loc_level(label.item(), level=0)
    180             else:
--> 181                 indexer = get_loc(index, label.item(), method, tolerance)
    182         elif label.dtype.kind == 'b':
    183             indexer = label

~/anaconda3/envs/check3/lib/python3.7/site-packages/xarray/core/indexing.py in get_loc(index, label, method, tolerance)
    106 def get_loc(index, label, method=None, tolerance=None):
    107     kwargs = _index_method_kwargs(method, tolerance)
--> 108     return index.get_loc(label, **kwargs)
    109 
    110 

~/anaconda3/envs/check3/lib/python3.7/site-packages/pandas/core/indexes/numeric.py in get_loc(self, key, method, tolerance)
    434             pass
    435         return super(Float64Index, self).get_loc(key, method=method,
--> 436                                                  tolerance=tolerance)
    437 
    438     @cache_readonly

~/anaconda3/envs/check3/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2657                 return self._engine.get_loc(key)
   2658             except KeyError:
-> 2659                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2660         indexer = self.get_indexer([key], method=method, tolerance=tolerance)
   2661         if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Float64HashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Float64HashTable.get_item()

KeyError: 0.111

xarray version: 0.12.1
numpy version: 1.16.3

@dcherian
Copy link
Contributor

da.sel(a=np.float32(0.111)) works so maybe we should cast to the dtype of the coordinate?

@shoyer
Copy link
Member

shoyer commented Jul 17, 2019

da.sel(a=np.float32(0.111)) works so maybe we should cast to the dtype of the coordinate?

Sounds like a good idea to me!

@dcherian
Copy link
Contributor

@hdsingh Any interest in sending in a PR?

@hdsingh
Copy link
Author

hdsingh commented Jul 18, 2019

As of now I am a bit short on time due to GSOC project. I may not be able to look into this for next 2-3 weeks.

@HasanAhmadQ7
Copy link
Contributor

@dcherian I would like to send a PR and work on this issue if there is no problem in this.

@dcherian
Copy link
Contributor

@HasanAhmadQ7 go for it!

HasanAhmadQ7 pushed a commit to HasanAhmadQ7/xarray that referenced this issue Jul 20, 2019
.sel method gives error when it is used to select float32 values

Resolves: pydata#3137
HasanAhmadQ7 pushed a commit to HasanAhmadQ7/xarray that referenced this issue Jul 20, 2019
.sel method gives error when it is used to select float32 values

Resolves: pydata#3137
HasanAhmadQ7 pushed a commit to HasanAhmadQ7/xarray that referenced this issue Jul 20, 2019
.sel method gives error when it is used to select float32 values

Resolves: pydata#3137
HasanAhmadQ7 pushed a commit to HasanAhmadQ7/xarray that referenced this issue Jul 21, 2019
    .sel method gives error when it is used to select float32 values

    Resolves: pydata#3137
@HasanAhmadQ7
Copy link
Contributor

HasanAhmadQ7 commented Jul 26, 2019

@dcherian I sent a PR last week. Any feedback would be appreciated

@HasanAhmadQ7
Copy link
Contributor

The case also arises if coords are float16. e.g.

        data_values = np.arange(4)
        coord_values_16 = np.asarray(float_values, dtype='float16')
        array_16 = DataArray(data_values, [('float16_coord', coord_values_16)])
        array_16.sel(float16_coord=float_values[1:3])

shoyer pushed a commit that referenced this issue Aug 10, 2019
* BUG fix +test .sel method gives error with float32 values

    .sel method gives error when it is used to select float32 values

    Resolves: #3137

* BUG  .sel method gives error with float32 values

avoid copying indexers for efficiency

* BUG: .sel method fails when label is float differnt from coords float type

casting at a lower level closer to pandas.Index methods

* BUG: .sel method fails when label is float differnt from coords float type

casting at a lower level closer to pandas.Index methods

* Update xarray/core/indexing.py

Co-Authored-By: Stephan Hoyer <[email protected]>

* BUG: .sel method gives error when label is float different from coords float type

generalizing to coerce  numpy arrays in addtion to lists/floats

* Black
@dcherian dcherian unpinned this issue Aug 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants