BUG: Grouped rank incorrect behaviour with nullable types

[NMAC427]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

# numpy dtype (None -> np.NaN)
# This will print the following:
# 
# 0    1.0
# Name: x, dtype: float64
#
# This is the expected behaviour

df_np = pd.DataFrame({ "x": [None] }, dtype="float64")
print(df_np.groupby("x", dropna=False)["x"].rank(method="min", na_option="bottom"))


# pandas nullable dtype (None -> pd.NA)
# This will print the following:
#
# 0    <NA>
# Name: x, dtype: Float64
#
# This is NOT the expected behaviour, because the rank is NA instead of 1

df_ext = pd.DataFrame({ "x": [None] }, dtype="Float64")
print(df_ext.groupby("x", dropna=False)["x"].rank(method="min", na_option="bottom"))

Issue Description

When using the rank function on grouped data frames, the result is different when using nullable datatypes compared to numpy datatypes.
This is only an issue when passing na_option="bottom" or na_option="top".

Expected Behavior

The result shouldn't depend on the datatype that got used.

Installed Versions

INSTALLED VERSIONS

commit : 0f43794
python : 3.9.6.final.0
python-bits : 64
OS : Darwin
OS-release : 22.5.0
Version : Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.0.3
numpy : 1.25.1
pytz : 2023.3
dateutil : 2.8.2
setuptools : 68.0.0
pip : 23.1.2
Cython : None
pytest : 7.4.0
hypothesis : None
sphinx : 7.0.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.9.6
jinja2 : 3.1.2
IPython : 8.14.0
pandas_datareader: None
bs4 : 4.12.2
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : 2.0.19
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

[rhshadrach]

Thanks for the report! In pandas.core.arrays.masked.py::BaseMaskedArray._groupby_op, just before the call to _cython_op_ndim_compat we are setting up result_mask. In the case of rank with na_option specified as "top" or "bottom", we are currently copying the current mask; instead we should be setting it all to False.

A test can be added to pandas.tests.groupby.test_rank.

[prasad-yashdeep]

take

[rsm-23]

Is this issue open for taking? @rhshadrach @Sai-Suraj-27

[rhshadrach]

The PR for closing this is linked and just about ready (I think only a small issue with the test remains).