Skip to content

Concat fails with tz-aware and empty series (axis=1) #16684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
linar-jether opened this issue Jun 12, 2017 · 3 comments
Closed

Concat fails with tz-aware and empty series (axis=1) #16684

linar-jether opened this issue Jun 12, 2017 · 3 comments
Labels
Bug Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode Timezones Timezone data dtype

Comments

@linar-jether
Copy link

Code Sample, a copy-pastable example if possible

# This works as expected, and as it worked in the past (0.17.1)
pd.concat([pd.util.testing.makeTimeSeries(), pd.Series()], axis=1)

# This fails in 0.20.1
pd.concat([pd.util.testing.makeTimeSeries().tz_localize('America/Chicago'), pd.Series()], axis=1)

Problem description

Only happens with axis=1
Shouldn't be a problem to concat an empty series to a dataframe...

Expected Output

Should filter out empty objects passed to concat.

pd.concat([pd.Series(pd.util.testing.makeTimeSeries().tz_localize('America/Chicago'), name='with_tz'), pd.Series(name='empty')], axis=1)
Out[2]: 
                            with_tz
2000-01-03 00:00:00-06:00 -0.999300
2000-01-04 00:00:00-06:00 -0.800759
2000-01-05 00:00:00-06:00  0.166496
2000-01-06 00:00:00-06:00 -0.496464
2000-01-07 00:00:00-06:00 -0.237731
2000-01-10 00:00:00-06:00 -0.242352
2000-01-11 00:00:00-06:00  1.779163
2000-01-12 00:00:00-06:00  0.193893

Output of pd.show_versions()

``` INSTALLED VERSIONS ------------------ commit: None python: 2.7.12.final.0 python-bits: 64 OS: Windows OS-release: 8.1 machine: AMD64 processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None pandas: 0.20.2 pytest: 3.0.7 pip: 9.0.1 setuptools: 34.3.1 Cython: None numpy: 1.12.1 scipy: 0.18.1 xarray: None IPython: 5.3.0 sphinx: None patsy: 0.4.1 dateutil: 2.6.0 pytz: 2017.2 blosc: None bottleneck: None tables: None numexpr: 2.6.2 feather: None matplotlib: 2.0.2 openpyxl: None xlrd: 1.0.0 xlwt: None xlsxwriter: None lxml: None bs4: 4.5.1 html5lib: 0.9999999 sqlalchemy: None pymysql: None psycopg2: None jinja2: 2.9.6 s3fs: None pandas_gbq: None pandas_datareader: None
@chris-b1
Copy link
Contributor

On your expected output - shouldn't there be an 'empty' column filled with NaNs, like the non tz case?

For consistency, I agree this should work, although in practice it's sort of odd behavior to rely on, I'd be more inclined to explicitly filter, like this. Still, thanks for the report and PRs welcome!

In [15]: pd.concat([x for x in series if not x.empty], axis=1)

@chris-b1 chris-b1 added Bug Timezones Timezone data dtype labels Jun 12, 2017
@chris-b1 chris-b1 added this to the Next Major Release milestone Jun 12, 2017
@chris-b1
Copy link
Contributor

#16230 might be related

@jreback
Copy link
Contributor

jreback commented Jun 13, 2017

this is a duplicate of #16230. PR's to fix that would be great!

@jreback jreback closed this as completed Jun 13, 2017
@jreback jreback added Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Jun 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode Timezones Timezone data dtype
Projects
None yet
Development

No branches or pull requests

3 participants