Skip to content

Commit 0d58446

Browse files
has2k1jreback
authored andcommitted
Fix pandas.Timedelta range
closes #12727 *Problem* Pandas Timedelta derives from `datetime.timedelta` and increase the resolution of the timedeltas. As such the Pandas.Timedelta object can only have a smaller range of values. *Solution* This change modifies the properties that report the range and resolution to reflect Pandas capabilities. **Reference** https ://github.com/python/cpython/blob/8d1d7e6816753248768e4cc1c0370221814e 9cf1/Lib/datetime.py#L651-L654 Author: Hassan Kibirige <[email protected]> Closes #12728 from has2k1/fix-timedelta-limits and squashes the following commits: 2b78e5a [Hassan Kibirige] DOC: Timedelta & timestamp limitations f1a93d1 [Hassan Kibirige] BUG: pandas.Timedelta min and max limits
1 parent b6959c8 commit 0d58446

File tree

6 files changed

+65
-22
lines changed

6 files changed

+65
-22
lines changed

doc/source/gotchas.rst

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -356,27 +356,6 @@ such as ``numpy.logical_and``.
356356
See the `this old issue <https://github.com/pydata/pandas/issues/2388>`__ for a more
357357
detailed discussion.
358358

359-
.. _gotchas.timestamp-limits:
360-
361-
Timestamp limitations
362-
---------------------
363-
364-
Minimum and maximum timestamps
365-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
366-
367-
Since pandas represents timestamps in nanosecond resolution, the timespan that
368-
can be represented using a 64-bit integer is limited to approximately 584 years:
369-
370-
.. ipython:: python
371-
372-
begin = pd.Timestamp.min
373-
begin
374-
375-
end = pd.Timestamp.max
376-
end
377-
378-
See :ref:`here <timeseries.oob>` for ways to represent data outside these bound.
379-
380359
Parsing Dates from Text Files
381360
-----------------------------
382361

doc/source/timedeltas.rst

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,20 @@ The ``unit`` keyword argument specifies the unit of the Timedelta:
109109
to_timedelta(np.arange(5), unit='s')
110110
to_timedelta(np.arange(5), unit='d')
111111
112+
.. _timedeltas.limitations:
113+
114+
Timedelta limitations
115+
~~~~~~~~~~~~~~~~~~~~~
116+
117+
Pandas represents ``Timedeltas`` in nanosecond resolution using
118+
64 bit integers. As such, the 64 bit integer limits determine
119+
the ``Timedelta`` limits.
120+
121+
.. ipython:: python
122+
123+
pd.Timedelta.min
124+
pd.Timedelta.max
125+
112126
.. _timedeltas.operations:
113127

114128
Operations

doc/source/timeseries.rst

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -307,6 +307,21 @@ using various combinations of parameters like ``start``, ``end``,
307307
The start and end dates are strictly inclusive. So it will not generate any
308308
dates outside of those dates if specified.
309309

310+
.. _timeseries.timestamp-limits:
311+
312+
Timestamp limitations
313+
---------------------
314+
315+
Since pandas represents timestamps in nanosecond resolution, the timespan that
316+
can be represented using a 64-bit integer is limited to approximately 584 years:
317+
318+
.. ipython:: python
319+
320+
pd.Timestamp.min
321+
pd.Timestamp.max
322+
323+
See :ref:`here <timeseries.oob>` for ways to represent data outside these bound.
324+
310325
.. _timeseries.datetimeindex:
311326

312327
DatetimeIndex
@@ -1691,7 +1706,7 @@ the quarter end:
16911706
Representing out-of-bounds spans
16921707
--------------------------------
16931708

1694-
If you have data that is outside of the ``Timestamp`` bounds, see :ref:`Timestamp limitations <gotchas.timestamp-limits>`,
1709+
If you have data that is outside of the ``Timestamp`` bounds, see :ref:`Timestamp limitations <timeseries.timestamp-limits>`,
16951710
then you can use a ``PeriodIndex`` and/or ``Series`` of ``Periods`` to do computations.
16961711

16971712
.. ipython:: python

doc/source/whatsnew/v0.18.1.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,7 @@ Bug Fixes
129129

130130

131131
- Bug in ``Timestamp.__repr__`` that caused ``pprint`` to fail in nested structures (:issue:`12622`)
132+
- Bug in ``Timedelta.min`` and ``Timedelta.max``, the properties now report the true minimum/maximum ``timedeltas`` as recognized by Pandas. See :ref:`documentation <timedeltas.limitations>`. (:issue:`12727`)
132133

133134

134135

pandas/tseries/tests/test_timedeltas.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1083,6 +1083,35 @@ def test_timedelta_hash_equality(self):
10831083
ns_td = Timedelta(1, 'ns')
10841084
self.assertNotEqual(hash(ns_td), hash(ns_td.to_pytimedelta()))
10851085

1086+
def test_implementation_limits(self):
1087+
min_td = Timedelta(Timedelta.min)
1088+
max_td = Timedelta(Timedelta.max)
1089+
1090+
# GH 12727
1091+
# timedelta limits correspond to int64 boundaries
1092+
self.assertTrue(min_td.value == np.iinfo(np.int64).min + 1)
1093+
self.assertTrue(max_td.value == np.iinfo(np.int64).max)
1094+
1095+
# Beyond lower limit, a NAT before the Overflow
1096+
self.assertIsInstance(min_td - Timedelta(1, 'ns'),
1097+
pd.tslib.NaTType)
1098+
1099+
with tm.assertRaises(OverflowError):
1100+
min_td - Timedelta(2, 'ns')
1101+
1102+
with tm.assertRaises(OverflowError):
1103+
max_td + Timedelta(1, 'ns')
1104+
1105+
# Same tests using the internal nanosecond values
1106+
td = Timedelta(min_td.value - 1, 'ns')
1107+
self.assertIsInstance(td, pd.tslib.NaTType)
1108+
1109+
with tm.assertRaises(OverflowError):
1110+
Timedelta(min_td.value - 2, 'ns')
1111+
1112+
with tm.assertRaises(OverflowError):
1113+
Timedelta(max_td.value + 1, 'ns')
1114+
10861115

10871116
class TestTimedeltaIndex(tm.TestCase):
10881117
_multiprocess_can_split_ = True

pandas/tslib.pyx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2722,6 +2722,11 @@ class Timedelta(_Timedelta):
27222722
__pos__ = _op_unary_method(lambda x: x, '__pos__')
27232723
__abs__ = _op_unary_method(lambda x: abs(x), '__abs__')
27242724

2725+
2726+
# Resolution is in nanoseconds
2727+
Timedelta.min = Timedelta(np.iinfo(np.int64).min+1, 'ns')
2728+
Timedelta.max = Timedelta(np.iinfo(np.int64).max, 'ns')
2729+
27252730
cdef PyTypeObject* td_type = <PyTypeObject*> Timedelta
27262731

27272732
cdef inline bint is_timedelta(object o):

0 commit comments

Comments
 (0)