Skip to content

df.applymap() produces unwanted type conversion of datetime64[ns] to timedelta #19256

Closed
@Apeder

Description

@Apeder

Code Sample, a copy-pastable example if possible

time_data.head()

print time_data.dtypes

time_data[nt] = time_data[nt].applymap(lambda x: x.strftime('%I:%M:%S %p') if pd.notnull(x) else pd.NaT)

print time_data.dtypes

Problem description

When executing applymap on a masked subset of df datetime columns, two of the four columns are converted to timedelta. Can't figure out what might be happening, perhaps an error similar to #18493? But why only two of the four?!

Here's what time_data.head() looks like:

         Date           Name               In AM              Out AM  \
0  2017-11-06   AUSTIN LEWIS 1900-01-01 06:10:24 1900-01-01 12:03:23   
1  2017-11-06     FRED MOORE 1900-01-01 06:58:37 1900-01-01 12:12:11   
2  2017-11-06  KERRIE PAUSSA 1900-01-01 11:58:48 1900-01-01 19:39:49   
3  2017-11-06   OMAR CUELLAR                 NaT                 NaT   
4  2017-11-07   AUSTIN LEWIS 1900-01-01 07:07:27 1900-01-01 12:06:43   

                In PM              Out PM Sick Time  
0 1900-01-01 12:32:03 1900-01-01 17:31:50       NaT  
1 1900-01-01 12:42:53 1900-01-01 17:31:50       NaT  
2                 NaT                 NaT       NaT  
3 1900-01-01 20:00:19 1900-01-01 23:59:41       NaT  
4 1900-01-01 12:35:26 1900-01-01 17:33:20       NaT              

Expected Output

Date                        object
Name                        object
In AM               datetime64[ns]
Out AM              datetime64[ns]
In PM               datetime64[ns]
Out PM              datetime64[ns]
Sick Time           datetime64[ns]
Total Clock Time            object
dtype: object

Date                        object
Name                        object
In AM               datetime64[ns]
Out AM              datetime64[ns]
In PM               datetime64[ns]
Out PM              datetime64[ns]
Sick Time           datetime64[ns]
Total Clock Time            object
dtype: object

Output of pd.show_versions()

Name                        object
In AM               datetime64[ns]
Out AM              datetime64[ns]
In PM               datetime64[ns]
Out PM              datetime64[ns]
Sick Time           datetime64[ns]
Total Clock Time            object
dtype: object

Date                         object
Name                         object
In AM                        object
Out AM                       object
In PM               timedelta64[ns]
Out PM              timedelta64[ns]
Sick Time            datetime64[ns]
Total Clock Time            float64
dtype: object 
INSTALLED VERSIONS ------------------ commit: None python: 2.7.10.final.0 python-bits: 64 OS: Darwin OS-release: 17.3.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None

pandas: 0.22.0
pytest: None
pip: 9.0.1
setuptools: 38.2.4
Cython: None
numpy: 1.14.0
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: 1.1.15
pymysql: 0.8.0
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions