int64 overflow/wrap around with nansum()

[xflr6]

See pandas-dev/pandas#15453

In [1]: import bottleneck as bn

In [2]: bn.__version__
Out[2]: '1.2.0'

In [3]: import numpy as np

In [4]: bn.nansum(np.array([2**31],dtype='int64'))
Out[4]: -2147483648

[kwgoodman]

That's odd. On linux with py2.7:

In [1]: bn.nansum(np.array([2**31],dtype='int64'))
Out[1]: 2147483648
In [2]: bn.nansum(np.array([2**131],dtype='int64'))
OverflowError: Python int too large to convert to C long

Python, not bottleneck, raises on the second example.

[cgohlke]

Using PyLong_FromLongLong instead of PyInt_FromLong in reduce_template.c fixes this issue as well as pandas-dev/pandas#15453 for me on Windows.

PyLong_FromLongLong is probably too much for the 32-bit functions, but it works here.

Would be nice to have this issue fixed in the 1.2.1 release #168.

[cgohlke]

I was using this:

diff --git a/bottleneck/src/reduce_template.c b/bottleneck/src/reduce_template.c
index d9fa955..031e1eb 100644
--- a/bottleneck/src/reduce_template.c
+++ b/bottleneck/src/reduce_template.c
@@ -129,7 +129,7 @@ REDUCE_ALL(nansum, DTYPE0)
         NEXT
     }
     BN_END_ALLOW_THREADS
-    return PyInt_FromLong(asum);
+    return PyLong_FromLongLong(asum);
 }
 
 REDUCE_ONE(nansum, DTYPE0)
@@ -506,7 +506,7 @@ REDUCE_ALL(NAME, DTYPE0)
         NEXT
     }
     BN_END_ALLOW_THREADS
-    return PyInt_FromLong(extreme);
+    return PyLong_FromLongLong(extreme);
 }
 
 REDUCE_ONE(NAME, DTYPE0)
@@ -571,7 +571,7 @@ REDUCE_ALL(NAME, DTYPE0)
         VALUE_ERR("All-NaN slice encountered");
         return NULL;
     } else {
-        return PyInt_FromLong(idx);
+        return PyLong_FromLongLong(idx);
     }
 }
 
@@ -636,7 +636,7 @@ REDUCE_ALL(NAME, DTYPE0)
     }
     BN_END_ALLOW_THREADS
     DECREF_INIT_ALL_RAVEL
-    return PyInt_FromLong(idx);
+    return PyLong_FromLongLong(idx);
 }
 
 REDUCE_ONE(NAME, DTYPE0)
@@ -728,7 +728,7 @@ REDUCE_ALL(ss, DTYPE0)
         NEXT
     }
     BN_END_ALLOW_THREADS
-    return PyInt_FromLong(asum);
+    return PyLong_FromLongLong(asum);
 }
 
 REDUCE_ONE(ss, DTYPE0)

[kwgoodman]

@cgohlke Your bug fix is a huge help. Thank you!

[kwgoodman]

@xflr6 thanks for the bug report
@cgohlke thanks for the fix

merged to master