Skip to content

Ensure xarray imports (and docs build) even without pandas.Panel #3058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 30, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 27 additions & 13 deletions doc/pandas.rst
Original file line number Diff line number Diff line change
Expand Up @@ -151,10 +151,11 @@ However, you will need to set dimension names explicitly, either with the
Transitioning from pandas.Panel to xarray
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:py:class:`~pandas.Panel`, pandas' data structure for 3D arrays, has always
``Panel``, pandas' data structure for 3D arrays, has always
been a second class data structure compared to the Series and DataFrame. To
allow pandas developers to focus more on its core functionality built around
the DataFrame, pandas plans to eventually deprecate Panel.
the DataFrame, panads has deprecated ``Panel``. It will be removed in pandas
0.25.

xarray has most of ``Panel``'s features, a more explicit API (particularly around
indexing), and the ability to scale to >3 dimensions with the same interface.
Expand All @@ -172,28 +173,41 @@ So you can represent a Panel, in two ways:
Let's take a look:

.. ipython:: python
:okwarning:

panel = pd.Panel(np.random.rand(2, 3, 4), items=list('ab'), major_axis=list('mno'),
minor_axis=pd.date_range(start='2000', periods=4, name='date'))
panel
data = np.random.RandomState(0).rand(2, 3, 4)
items = list('ab')
major_axis = list('mno')
minor_axis = pd.date_range(start='2000', periods=4, name='date')

As a DataArray:
With old versions of pandas (prior to 0.25), this could stored in a ``Panel``:

.. ipython::
:verbatim:

In [1]: pd.Panel(data, items, major_axis, minor_axis)
Out[1]:
<class 'pandas.core.panel.Panel'>
Dimensions: 2 (items) x 3 (major_axis) x 4 (minor_axis)
Items axis: a to b
Major_axis axis: m to o
Minor_axis axis: 2000-01-01 00:00:00 to 2000-01-04 00:00:00

To put this data in a ``DataArray``, write:

.. ipython:: python

# or equivalently, with Panel.to_xarray()
xr.DataArray(panel)
array = xr.DataArray(data, [items, major_axis, minor_axis])
array

As you can see, there are three dimensions (each is also a coordinate). Two of
the axes of the panel were unnamed, so have been assigned ``dim_0`` and
``dim_1`` respectively, while the third retains its name ``date``.
the axes of were unnamed, so have been assigned ``dim_0`` and ``dim_1``
respectively, while the third retains its name ``date``.

As a Dataset:
You can also easily convert this data into ``Dataset``:

.. ipython:: python

xr.Dataset(panel)
array.to_dataset(dim='dim_0')

Here, there are two data variables, each representing a DataFrame on panel's
``items`` axis, and labelled as such. Each variable is a 2D array of the
Expand Down
10 changes: 10 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ Enhancements
Bug fixes
~~~~~~~~~

- Fix to ensure that xarray can still be imported with pandas 0.25, which will
remove `pd.Panel`.
By `Stephan Hoyer <https://github.com/shoyer>`_.

.. _whats-new.0.12.2:

v0.12.2 (29 June 2019)
Expand Down Expand Up @@ -164,6 +168,12 @@ Bug fixes
By `Maximilian Roos <https://github.com/max-sixty>`_.
- Fixed performance issues with cftime installed (:issue:`3000`)
By `0x0L <https://github.com/0x0L>`_.
- Replace incorrect usages of `message` in pytest assertions
with `match` (:issue:`3011`)
By `Maximilian Roos <https://github.com/max-sixty>`_.
- Add explicit pytest markers, now required by pytest
(:issue:`3032`).
By `Maximilian Roos <https://github.com/max-sixty>`_.
- Test suite fixes for newer versions of pytest (:issue:`3011`, :issue:`3032`).
By `Maximilian Roos <https://github.com/max-sixty>`_
and `Stephan Hoyer <https://github.com/shoyer>`_.
Expand Down
10 changes: 5 additions & 5 deletions xarray/core/dataarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ def __init__(
coords = [data.index, data.columns]
elif isinstance(data, (pd.Index, IndexVariable)):
coords = [data]
elif isinstance(data, pd.Panel):
elif hasattr(pd, 'Panel') and isinstance(data, pd.Panel):
coords = [data.items, data.major_axis, data.minor_axis]
if dims is None:
dims = getattr(data, 'dims', getattr(coords, 'dims', None))
Expand Down Expand Up @@ -1825,8 +1825,7 @@ def reduce(self, func: Callable[..., Any],
**kwargs)
return self._replace_maybe_drop_dims(var)

def to_pandas(self) -> Union[
'DataArray', pd.Series, pd.DataFrame, pd.Panel]:
def to_pandas(self) -> Union['DataArray', pd.Series, pd.DataFrame]:
"""Convert this array into a pandas object with the same shape.

The type of the returned object depends on the number of DataArray
Expand All @@ -1845,8 +1844,9 @@ def to_pandas(self) -> Union[
# attributes that correspond to their indexes into a separate module?
constructors = {0: lambda x: x,
1: pd.Series,
2: pd.DataFrame,
3: pd.Panel}
2: pd.DataFrame}
if hasattr(pd, 'Panel'):
constructors[3] = pd.Panel
try:
constructor = constructors[self.ndim]
except KeyError:
Expand Down
4 changes: 3 additions & 1 deletion xarray/core/merge.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@
from .dataset import Dataset


PANDAS_TYPES = (pd.Series, pd.DataFrame, pd.Panel)
PANDAS_TYPES = (
(pd.Series, pd.DataFrame) + (pd.Panel,) if hasattr(pd, 'Panel') else ()
)

_VALID_COMPAT = Frozen({'identical': 0,
'equals': 1,
Expand Down