Backport PR #38148: ENH: Improve performance for df.__setitem__ with list-like indexers

[simonjayhawkins]

Backport PR #38148

[simonjayhawkins]

@phofl #38148 (comment) was this necessary? do we know what commit caused the performance regression. the existing code on 1.1.x looks much simpler to what was replaced on master.

[phofl]

The underlying performance problems existed with this solution too. The loop is the actual problem, not the if else logic.

[phofl]

That is tricky. The tests are failing, because Index.union is different on master.

idx = pd.RangeIndex(0, 2)
result = idx.union([1])

This returns a RangeIndex on master and an Index with dtype="object" on 1.1.4.

Suggestions about how to proceed here?

cc @simonjayhawkins

[jorisvandenbossche]

Since this was not a performance regression (at least, the issue #37954 does not mention anything about a regression, AFAIK), I don't think we should be backporting this.

[simonjayhawkins]

Since this was not a performance regression (at least, the issue #37954 does not mention anything about a regression, AFAIK), I don't think we should be backporting this.

I agree.

closing this an can maybe revisit pending outcome of discussions in #38148

if not reverted the release note needs to be moved, but this is tracked by https://github.com/simonjayhawkins/pandas-release/actions/runs/391543194 so won't be missed.