implement Frame

[gdementen]

which would store different fields/columns with (potentially) different types.

Each field from an LFrame should also be accessible via __getattr__.

We might want to use a ColumnArray instead of a numpy structured array like in LIAM2 as the storage backend so that we can add fields efficiently.

See the https://github.com/liam2/larray/tree/structured_array branch.

[gdementen]

There are several big issues to resolve:

• Q: what does read_csv et al return: LArray or LFrame?
  A: I think they should return an LFrame by default. API-wise, I think it would work, but it would be a performance hit, relative to what we have now, because the LFrame would at best need to be converted to LArray at the first sign of an aggregate.
• Q: do we implement aggregates over the column/fields axis, if so how?
  A: the easiest would be to raise an Exception in that case, but then we cannot return an LFrame by default for read_csv. One option would be to transparently convert to LArray when doing aggregates. This screams pandas.NDFrame all over again but I think it is sane.
• Q: if we store as a ColumnArray, we will probably need to re-implement many operations, is there a way to avoid this?
  A: I do not think we can avoid this, but we do shouldn't need to re-implement user-facing methods, but rather make ColumnArray implement the subset of the numpy API that we use internally. This is some significant work, but should not be too hard.

We should see what xarray does here.

[alixdamman]

• Q: do we implement aggregates over the column/fields axis, if so how?
  A: the easiest would be to raise an Exception in that case, but then we cannot return an LFrame by default for read_csv.

@gdementen why?

[gdementen]

because currently read_csv returns LArrays where aggregates on the column axis work. This would be a massive backward incompatibility/pain in the * for our users if suddenly all their data loaded from .csv files failed to aggregate over the column axis.