BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

Xbar · 2017-11-23T02:43:00Z

MultiIndex.from_tuples accept zipped tuples in python 3. Ensures compatibility
between 2 and 3.

closes BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18434
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

codecov · 2017-11-23T08:25:47Z

Codecov Report

Merging #18440 into master will decrease coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #18440      +/-   ##
==========================================
- Coverage   91.36%   91.32%   -0.04%     
==========================================
  Files         163      163              
  Lines       49700    49764      +64     
==========================================
+ Hits        45407    45446      +39     
- Misses       4293     4318      +25

Flag	Coverage Δ
#multiple	`89.12% <100%> (-0.03%)`	⬇️
#single	`40.71% <50%> (+0.99%)`	⬆️

Impacted Files	Coverage Δ
pandas/core/indexes/multi.py	`96.44% <100%> (+0.03%)`	⬆️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/io/clipboard/__init__.py	`56.86% <0%> (-19.33%)`	⬇️
pandas/core/dtypes/missing.py	`90.24% <0%> (-0.5%)`	⬇️
pandas/core/indexes/datetimelike.py	`96.91% <0%> (-0.2%)`	⬇️
pandas/core/frame.py	`97.8% <0%> (-0.1%)`	⬇️
pandas/core/series.py	`94.76% <0%> (-0.08%)`	⬇️
pandas/core/dtypes/cast.py	`88.52% <0%> (-0.08%)`	⬇️
pandas/core/indexes/base.py	`96.4% <0%> (-0.04%)`	⬇️
pandas/core/base.py	`96.54% <0%> (-0.03%)`	⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aaee541...7a220aa. Read the comment docs.

jreback · 2017-11-23T16:00:55Z

pandas/core/indexes/multi.py

@@ -1206,6 +1206,12 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not hasattr(tuples, '__len__'):


would just do

if not is_list_like(tuples): raise TypeError... elif is_iterator(tuples): tuples = list(tuples)

jreback · 2017-11-23T16:01:41Z

pandas/core/indexes/multi.py

@@ -1206,6 +1206,12 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not hasattr(tuples, '__len__'):
+            try:
+                iter(tuples)  # if iterable, then it's iterator or generator


needs checking in from_product and from_arrays as well.

needs tests that hit all 3 cases

Xbar · 2017-11-24T03:19:52Z

Added the test cases and fix from_tuples/from_arrays/from_product

jreback

some error context comments. otherwise lgtm. ping on green.

jreback · 2017-11-24T17:00:49Z

pandas/core/indexes/multi.py

@@ -1162,6 +1162,11 @@ def from_arrays(cls, arrays, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not is_list_like(arrays):
+            raise TypeError("Inappropriate input for list of arrays.")


make a more user-friendly message, maybe

input must be a list-like of arrays

jreback · 2017-11-24T17:00:59Z

pandas/core/indexes/multi.py

@@ -1206,6 +1211,11 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not is_list_like(tuples):
+            raise TypeError('Inappropriate input for tuples.')


same (except say tuples)

jreback · 2017-11-24T17:01:04Z

pandas/core/indexes/multi.py

@@ -1260,6 +1270,11 @@ def from_product(cls, iterables, sortorder=None, names=None):
        from pandas.core.categorical import _factorize_from_iterables
        from pandas.core.reshape.util import cartesian_product

+        if not is_list_like(iterables):


jreback · 2017-11-24T17:01:16Z

pandas/tests/indexes/test_multi.py

        result = MultiIndex.from_arrays(arrays)
        assert list(result) == list(self.index)
+        # iterator as input


blank line here

jreback · 2017-11-24T17:01:20Z

pandas/tests/indexes/test_multi.py

+        # iterator as input
+        result2 = MultiIndex.from_arrays(iter(arrays))
+        assert list(result2) == list(self.index)
+        # invlide iterator input


jreback · 2017-11-24T17:01:29Z

pandas/tests/indexes/test_multi.py

+        expected = MultiIndex(levels=[[1, 3], [2, 4]],
+                              labels=[[0, 1], [0, 1]],
+                              names=['a', 'b'])
+        # input tuples


jreback

pls rebase as well. lgtm. ping on green.

jreback · 2017-11-24T19:48:51Z

pandas/tests/indexes/test_multi.py

@@ -827,6 +837,16 @@ def test_from_product(self):
        tm.assert_index_equal(result, expected)
        assert result.names == names

+        # iterator as input


can you make this a separate test (the added code), and do so for from_product, from_tuples, from_arrays as well, call it something like

test_multiindex_from_product_iterable and add the issue number to it.

jreback · 2017-11-24T19:49:07Z

pandas/core/indexes/multi.py

@@ -1206,6 +1211,11 @@ def from_tuples(cls, tuples, sortorder=None, names=None):
        MultiIndex.from_product : Make a MultiIndex from cartesian product
                                  of iterables
        """
+        if not is_list_like(tuples):
+            raise TypeError('Input must be a list /sequence of tuple-likes.')


need a space after the /

jreback · 2017-11-24T19:49:27Z

pandas/tests/indexes/test_multi.py

+                              labels=[[0, 1], [0, 1]],
+                              names=['a', 'b'])
+
+        # input tuples


same (new test)

jreback · 2017-11-24T19:49:51Z

pandas/tests/indexes/test_multi.py

+                              names=['a', 'b'])
+
+        # input tuples
+        res1 = MultiIndex.from_tuples(((1, 2), (3, 4)), names=['a', 'b'])


use result =

jreback · 2017-11-24T19:49:56Z

pandas/tests/indexes/test_multi.py

+        assert res1.equals(expected)
+
+        # input iterator for tuples
+        res2 = MultiIndex.from_tuples(zip([1, 3], [2, 4]), names=['a', 'b'])


jreback · 2017-11-24T19:50:28Z

pandas/tests/indexes/test_multi.py

+        # input iterator for tuples
+        res2 = MultiIndex.from_tuples(zip([1, 3], [2, 4]), names=['a', 'b'])
+        assert expected.names == res2.names
+        assert res2.equals(expected)


use tm.assert_index_equal(result, expected) for the checking

jreback · 2017-11-24T19:51:02Z

pandas/tests/indexes/test_multi.py

        result = MultiIndex.from_arrays(arrays)
        assert list(result) == list(self.index)

+        # iterator as input


separate test

jreback · 2017-11-24T19:51:13Z

pandas/core/indexes/multi.py

-        for lev, lab in zip(self.levels, self.labels):
+        for idx, (lev, lab) in enumerate(zip(self.levels, self.labels)):
+            na_idxs = np.where(lab == -1)[0]
+


looks like you picked up another commit.

Xbar · 2017-11-25T01:30:00Z

All build passed.

jreback · 2017-11-25T15:08:14Z

pandas/tests/indexes/test_multi.py

+
+        # iterator as input
+        result = MultiIndex.from_arrays(iter(arrays))
+        assert list(result) == list(self.index)


use tm.assert_index_equal, IOW construct the expected MI

jreback · 2017-11-25T15:08:24Z

pandas/tests/indexes/test_multi.py

+
+        # iterator as input
+        result = MultiIndex.from_product(iter([first, second]), names=names)
+        assert result.equals(expected)


use assert_index_equal

jreback · 2017-11-25T15:08:36Z

pandas/tests/indexes/test_multi.py

+
+        # input tuples
+        result = MultiIndex.from_tuples(((1, 2), (3, 4)), names=['a', 'b'])
+        assert expected.names == result.names


jreback · 2017-11-25T15:08:40Z

pandas/tests/indexes/test_multi.py

+        # input tuples
+        result = MultiIndex.from_tuples(((1, 2), (3, 4)), names=['a', 'b'])
+        assert expected.names == result.names
+        assert result.equals(expected)


jreback · 2017-11-25T15:08:45Z

pandas/tests/indexes/test_multi.py

+                              names=['a', 'b'])
+
+        result = MultiIndex.from_tuples(zip([1, 3], [2, 4]), names=['a', 'b'])
+        assert expected.names == result.names


jreback · 2017-11-25T20:57:02Z

thanks @Xbar nice patch and very responsive interactions!

Xbar · 2017-11-25T22:13:05Z

Thanks! 👍

jreback requested changes Nov 23, 2017

View reviewed changes

jreback changed the title ~~Fix issue 18434~~ BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples Nov 23, 2017

jreback added 2/3 Compat Reshaping Concat, Merge/Join, Stack/Unstack, Explode Error Reporting Incorrect or improved errors from pandas labels Nov 23, 2017

Xbar force-pushed the master branch from a60467d to c2bbe5c Compare November 24, 2017 03:17

jreback requested changes Nov 24, 2017

View reviewed changes

Xbar force-pushed the master branch from b084cd4 to 7f38fd5 Compare November 24, 2017 17:24

jreback requested changes Nov 24, 2017

View reviewed changes

Xbar force-pushed the master branch from 7f38fd5 to 9ceadbe Compare November 24, 2017 21:05

Make independent test cases for issue 18434

65fbc91

Xbar force-pushed the master branch from 9ceadbe to 65fbc91 Compare November 24, 2017 21:16

jreback requested changes Nov 25, 2017

View reviewed changes

use assert_index_equal for assert

7a220aa

jreback added this to the 0.22.0 milestone Nov 25, 2017

jreback approved these changes Nov 25, 2017

View reviewed changes

jreback merged commit 0bcd77e into pandas-dev:master Nov 25, 2017

simonjayhawkins mentioned this pull request Feb 8, 2019

API: more consistent error message for MultiIndex.from_arrays #25189

Merged

Uh oh!

BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

BUG: in Python3 MultiIndex.from_tuples cannot take "zipped" tuples #18440

Uh oh!

Conversation

Xbar commented Nov 23, 2017

Uh oh!

codecov bot commented Nov 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Xbar commented Nov 24, 2017

Uh oh!

jreback left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jreback left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Xbar commented Nov 25, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jreback commented Nov 25, 2017

Uh oh!

Xbar commented Nov 25, 2017

Uh oh!

Uh oh!

codecov bot commented Nov 23, 2017 •

edited

Loading