Skip to content

Improve MultiIndex label rename checks #61769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -909,6 +909,7 @@ Other
- Bug in :meth:`DataFrame.sort_values` where sorting by a column explicitly named ``None`` raised a ``KeyError`` instead of sorting by the column as expected. (:issue:`61512`)
- Bug in :meth:`DataFrame.transform` that was returning the wrong order unless the index was monotonically increasing. (:issue:`57069`)
- Bug in :meth:`DataFrame.where` where using a non-bool type array in the function would return a ``ValueError`` instead of a ``TypeError`` (:issue:`56330`)
- Bug in :meth:`Index._transform_index` where transformations are applied across all levels one by one even when a level is not specified (:issue:`55169`)
- Bug in :meth:`Index.sort_values` when passing a key function that turns values into tuples, e.g. ``key=natsort.natsort_key``, would raise ``TypeError`` (:issue:`56081`)
- Bug in :meth:`MultiIndex.fillna` error message was referring to ``isna`` instead of ``fillna`` (:issue:`60974`)
- Bug in :meth:`Series.describe` where median percentile was always included when the ``percentiles`` argument was passed (:issue:`60550`).
Expand Down
15 changes: 9 additions & 6 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -6429,12 +6429,15 @@ def _transform_index(self, func, *, level=None) -> Index:
Only apply function to one level of the MultiIndex if level is specified.
"""
if isinstance(self, ABCMultiIndex):
values = [
self.get_level_values(i).map(func)
if i == level or level is None
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mroeschke - I've managed to trace this issue to this line here. The Index._transform_index function is only used in one other place: merge.py>_items_overlap_with_suffix

Checking if this has had an adverse impact on other tests...

Copy link
Author

@TabLand TabLand Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like certain existing transformations have to be applied to each individual level component of a MultiIndex, whereas others make more sense to apply to the entire thing.

I'm guessing a new parameter to DataFrame.rename() is needed which could then be cascaded down to Index._transform_index()? Something like scope: {"full", "partial"}?

else self.get_level_values(i)
for i in range(self.nlevels)
]
if level is None:
return self.map(func)
else:
values = [
self.get_level_values(i).map(func)
if i == level
else self.get_level_values(i)
for i in range(self.nlevels)
]
return type(self).from_arrays(values)
else:
items = [func(x) for x in self]
Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/frame/methods/test_rename.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,14 @@ def test_rename_multiindex(self):
renamed = df.rename(index={"foo1": "foo3", "bar2": "bar3"}, level=0)
tm.assert_index_equal(renamed.index, new_index)

def test_rename_multiindex_tuples_with_checks(self):
df = DataFrame({("a", "count"): [1, 2], ("a", "sum"): [3, 4]})
renamed = df.rename(
columns={("a", "count"): ("b", "number_of"), ("a", "sum"): ("b", "total")}, errors="raise"
)
new_columns = MultiIndex.from_tuples([("b", "number_of"), ("b", "total")])
tm.assert_index_equal(renamed.columns, new_columns)

def test_rename_nocopy(self, float_frame):
renamed = float_frame.rename(columns={"C": "foo"})

Expand Down
Loading