Skip to content

Inconsistent behavior of rolling with missing value #32724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
CharlesAuguste opened this issue Mar 15, 2020 · 0 comments · Fixed by #36789
Closed

Inconsistent behavior of rolling with missing value #32724

CharlesAuguste opened this issue Mar 15, 2020 · 0 comments · Fixed by #36789
Labels
Needs Tests Unit test(s) needed to prevent regressions Window rolling, ewma, expanding

Comments

@CharlesAuguste
Copy link

Code Sample, a copy-pastable example if possible

>>> import pandas as pd
>>> pd.__version__
'1.0.2'
>>> df = pd.DataFrame({"A": [1, None], "B":[4, 5], "C": [7, 8]})
>>> df.rolling(min_periods=1, window=2, axis=1).sum()
     A    B     C
0  1.0  4.0  11.0
1  NaN  5.0  13.0
>>> df.T.rolling(min_periods=1, window=2).sum().T
     A    B     C
0  1.0  5.0  11.0
1  NaN  5.0  13.0

Problem description

In the above example, I would expect that rolling from one axis or transposing and rolling from the other axis give the same result. This is true when there is no missing value in the dataframe. However, here there is a missing value the value at position .loc[0, "B"] ends up being different.

In the first case the value is not summed with the value at .loc[0, "A"], but in the second case it is. I think this may be an issue because I would expect to get the same output in both cases.

I observed this both in pandas 1.0.1 and 1.0.2.

Expected Output

>>> df.rolling(min_periods=1, window=2, axis=1).sum()
     A    B     C
0  1.0  5.0  11.0
1  NaN  5.0  13.0
>>> df.T.rolling(min_periods=1, window=2).sum().T
     A    B     C
0  1.0  5.0  11.0
1  NaN  5.0  13.0

or

>>> df.rolling(min_periods=1, window=2, axis=1).sum()
     A    B     C
0  1.0  4.0  11.0
1  NaN  5.0  13.0
>>> df.T.rolling(min_periods=1, window=2).sum().T
     A    B     C
0  1.0  4.0  11.0
1  NaN  5.0  13.0

Output of pd.show_versions()

>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.6.2.final.0
python-bits      : 64
OS               : Linux
OS-release       : 4.15.0-76-generic
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.0.2
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 46.0.0
Cython           : 0.29.15
pytest           : 3.2.1
hypothesis       : None
sphinx           : 1.5.6
blosc            : None
feather          : None
xlsxwriter       : 0.9.8
lxml.etree       : 3.8.0
html5lib         : 0.9999999
pymysql          : None
psycopg2         : None
jinja2           : 2.10
IPython          : 7.0.1
pandas_datareader: None
bs4              : 4.6.0
bottleneck       : 1.2.1
fastparquet      : None
gcsfs            : None
lxml.etree       : 3.8.0
matplotlib       : 3.0.0
numexpr          : 2.6.2
odfpy            : None
openpyxl         : 2.4.8
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : 3.2.1
pyxlsb           : None
s3fs             : None
scipy            : 1.2.1
sqlalchemy       : 1.1.13
tables           : 3.4.2
tabulate         : None
xarray           : None
xlrd             : 1.1.0
xlwt             : 1.3.0
xlsxwriter       : 0.9.8
numba            : 0.42.1
@jbrockmendel jbrockmendel added the Window rolling, ewma, expanding label Jun 5, 2020
@phofl phofl added the Needs Tests Unit test(s) needed to prevent regressions label Oct 1, 2020
@phofl phofl mentioned this issue Oct 1, 2020
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Tests Unit test(s) needed to prevent regressions Window rolling, ewma, expanding
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants