multi-dimensional-indexing deprecation warning has inconsistent stack level.

[tacaswell]

Code Sample, a copy-pastable example if possible

# Your code here
import warnings
import pandas as pd

warnings.simplefilter("always")


x = pd.Series([], dtype="float64")

df = pd.DataFrame(
    {
        "year": [2018, 2018, 2018],
        "month": [1, 1, 1],
        "day": [1, 2, 3],
        "value": [1, 2, 3],
    }
)
df["date"] = pd.to_datetime(df[["year", "month", "day"]])
monthly = df[["date", "value"]].groupby(["date"]).sum()
dates = monthly.index

dates[:, None]
x.index[:, None]

Problem description

Running the above script gives:

sharon@22:14  ➤  python /tmp/pd_test.py 
/tmp/pd_test.py:21: DeprecationWarning: Support for multi-dimensional indexing (e.g. `index[:, None]`) on an Index is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.
  dates[:, None]
/home/tcaswell/.virtualenvs/sys37/lib/python3.7/site-packages/pandas/core/indexes/range.py:708: DeprecationWarning: Support for multi-dimensional indexing (e.g. `index[:, None]`) on an Index is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.
  return super().__getitem__(key)

I think the issue is that

pandas/pandas/core/indexers.py

Lines 263 to 270 in 7d616b9

	if np.ndim(result) > 1:
	warnings.warn(
	"Support for multi-dimensional indexing (e.g. `index[:, None]`) "
	"on an Index is deprecated and will be removed in a future "
	"version. Convert to a numpy array before indexing instead.",
	DeprecationWarning,
	stacklevel=3,
	)

is pushing the warning up a fixed number of stack layers, but due to implementation details it maybe deeper / less deep than expected.

Expected Output

pandas should be the module reported in the warning about slicing in both cases or neither case. You may want to use something like https://github.com/matplotlib/matplotlib/blob/e4dbf4e7b5241e7d74f79adc854f84a954b45e8f/lib/matplotlib/cbook/__init__.py#L2013-L2032 to find the first stack not in pandas to put the warning at.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Linux
OS-release : 5.0.0-38-generic
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.0.0rc0
numpy : 1.16.5
pytz : 2019.2
dateutil : 2.8.0
pip : 19.2.2
setuptools : 41.0.1
Cython : 0.29.13
pytest : 3.10.1
hypothesis : None
sphinx : 2.2.0
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.12.0.dev
pandas_datareader: None
bs4 : 4.8.2
bottleneck : 1.2.1
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.2.0rc2.post1219+g3d8a6cb83
numexpr : 2.7.0
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : 3.10.1
s3fs : None
scipy : 1.3.1
sqlalchemy : None
tables : 3.6.1
tabulate : None
xarray : 0.14.0
xlrd : 1.2.0
xlwt : None
xlsxwriter : None
numba : None

[jorisvandenbossche]

Yes, the problem here is that the place where the warning is raised can be reached in different ways (and so potentially needing a different stacklevel).
(more specifically, I think for all types the stacklevel is fine, except for RangeIndex, because its __getitem__ calls super().__getiitem__(key) leading to an extra stack)

Such a "dynamic stack-level inferring" sounds interesting. Your experience with it in matplotlib is good? It has proven robust?

[jbrockmendel]

We now have pandas.util._exceptions.find_stack_level doing pretty much exactly what matplotlib's _warn_external does. I've been trying to use the "cheat" sparingly, but if its needed, this is now an easy fix.

[tacaswell]

Your experience with it in matplotlib is good? It has proven robust?

answering 17month later, we have not had any reported issues with it!