Skip to content

Inconsistent behavior of rolling with missing value #32724

Closed
@CharlesAuguste

Description

@CharlesAuguste

Code Sample, a copy-pastable example if possible

>>> import pandas as pd
>>> pd.__version__
'1.0.2'
>>> df = pd.DataFrame({"A": [1, None], "B":[4, 5], "C": [7, 8]})
>>> df.rolling(min_periods=1, window=2, axis=1).sum()
     A    B     C
0  1.0  4.0  11.0
1  NaN  5.0  13.0
>>> df.T.rolling(min_periods=1, window=2).sum().T
     A    B     C
0  1.0  5.0  11.0
1  NaN  5.0  13.0

Problem description

In the above example, I would expect that rolling from one axis or transposing and rolling from the other axis give the same result. This is true when there is no missing value in the dataframe. However, here there is a missing value the value at position .loc[0, "B"] ends up being different.

In the first case the value is not summed with the value at .loc[0, "A"], but in the second case it is. I think this may be an issue because I would expect to get the same output in both cases.

I observed this both in pandas 1.0.1 and 1.0.2.

Expected Output

>>> df.rolling(min_periods=1, window=2, axis=1).sum()
     A    B     C
0  1.0  5.0  11.0
1  NaN  5.0  13.0
>>> df.T.rolling(min_periods=1, window=2).sum().T
     A    B     C
0  1.0  5.0  11.0
1  NaN  5.0  13.0

or

>>> df.rolling(min_periods=1, window=2, axis=1).sum()
     A    B     C
0  1.0  4.0  11.0
1  NaN  5.0  13.0
>>> df.T.rolling(min_periods=1, window=2).sum().T
     A    B     C
0  1.0  4.0  11.0
1  NaN  5.0  13.0

Output of pd.show_versions()

>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.6.2.final.0
python-bits      : 64
OS               : Linux
OS-release       : 4.15.0-76-generic
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.0.2
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 46.0.0
Cython           : 0.29.15
pytest           : 3.2.1
hypothesis       : None
sphinx           : 1.5.6
blosc            : None
feather          : None
xlsxwriter       : 0.9.8
lxml.etree       : 3.8.0
html5lib         : 0.9999999
pymysql          : None
psycopg2         : None
jinja2           : 2.10
IPython          : 7.0.1
pandas_datareader: None
bs4              : 4.6.0
bottleneck       : 1.2.1
fastparquet      : None
gcsfs            : None
lxml.etree       : 3.8.0
matplotlib       : 3.0.0
numexpr          : 2.6.2
odfpy            : None
openpyxl         : 2.4.8
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : 3.2.1
pyxlsb           : None
s3fs             : None
scipy            : 1.2.1
sqlalchemy       : 1.1.13
tables           : 3.4.2
tabulate         : None
xarray           : None
xlrd             : 1.1.0
xlwt             : 1.3.0
xlsxwriter       : 0.9.8
numba            : 0.42.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs TestsUnit test(s) needed to prevent regressionsWindowrolling, ewma, expanding

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions