Skip to content

BUG: concat() with datetime variable produces unclear/misleading error #55132

@MichalRIcar

Description

@MichalRIcar

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
from pandas import DataFrame as pDF
from datetime import datetime

pd.concat([
    pDF({'A': datetime.strptime('23/09/14', "%y/%m/%d")}, index=[0]),
    pDF({'A': datetime.strptime('23/09/14', "%y/%m/%d")}, index=[0]).astype('datetime64[ns]')
])

Issue Description

The example above raises a 'ValueError: Shape of passed values is (1, 1), indices imply (2, 1).'

However, this error occurs because the datetime variables have different precisions; one is datetime64[us], and the other is datetime64[ns].

I assume that the concatenation operation intends to raise an error in this case, even though similar cases with concatenating float and int values do not raise an error and join datasets by casting precision from int to float. Therefore, I am considering whether to handle datetime concatenation in the same way and adjust it accordingly..?

Expected Behavior

Either don't raise and concat while casting precision, similarly as int goes to float, or raise an error but with proper reasoning.

Installed Versions

python : 3.11.4.final.0 pandas : 2.1.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    Non-Nanodatetime64/timedelta64 with non-nanosecond resolutionRegressionFunctionality that used to work in a prior pandas versionReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions