DateTimeIndex.to_frame() drops timezone in pandas 0.24

[jakevdp]

Behavior in pandas 0.23:

>>> import pandas as pd
>>> pd.__version__
0.23.0
>>> x = pd.date_range(start='2019-01-01', end='2019-01-30', freq='D')
>>> x = x.tz_localize('UTC')
>>> x.to_frame().dtypes
0    datetime64[ns, UTC]
dtype: object

Behavior in pandas 0.24

>>> import pandas as pd
>>> pd.__version__
0.24.2
>>> x = pd.date_range(start='2019-01-01', end='2019-01-30', freq='D')
>>> x = x.tz_localize('UTC')
>>> x.to_frame().dtypes
0    datetime64[ns]
dtype: object

[jakevdp]

Issue is here:

pandas/pandas/core/indexes/base.py

Line 1211 in 05780f8

result = DataFrame({name: self.values.copy()})

index.values converts data to a numpy array with no timezone information in the dtype:

x = pd.date_range(start='2019-01-01', end='2019-01-30', freq='D')
x = x.tz_localize('UTC')
x.values.dtype
# dtype('<M8[ns]')

[jakevdp]

Some initial experiments suggest that replacing self.values with self.to_numpy() will fix this; I'll give this a try and prepare a PR.

[mroeschke]

Thanks for the report @jakevdp! I already have a PR up to fix this issue #25811.

The semantics with .values changed recently along with our migration to use ExtensionArrays more internally (in this case DatetimeArray). Passing the internal extension array (._values) into the constructor solves this issue.