Implement ExtensionArray interface #701
Conversation
jorisvandenbossche
force-pushed
the
extension-array
branch
from
9214cff to
9af7f8d
Compare
jorisvandenbossche
force-pushed
the
extension-array
branch
from
9af7f8d to
56e6019
Compare
geopandas/array.py
Outdated
| return self | ||
| from pandas import factorize | ||
| _, uniques = factorize(self) | ||
| return uniques |
There was a problem hiding this comment.
Note that two geometries can have the same value but different pointers. Do we have have a policy on what to do here?
There was a problem hiding this comment.
See the discussion for factorize
geopandas/array.py
Outdated
| """Return my 'dense' representation | ||
| @property | ||
| def nbytes(self): | ||
| return self.data.nbytes |
There was a problem hiding this comment.
This is probably an under-estimate. Geometries are generally far larger than the pointers. It's possible that GEOS has something for this, although I doubt it. If it's cheap we might consider converting to WKB and taking the length of that as an approximation.
| pandas.factorize | ||
| ExtensionArray.factorize | ||
| """ | ||
| return from_wkb(values) |
There was a problem hiding this comment.
This implementation seems surprising given the method title.
There was a problem hiding this comment.
Ah I see, we represent geometries with the appropriate bytestring
There was a problem hiding this comment.
Yes, for now I went for using WKB as an intermediate representation for unique / factorize (and possibly also for sorting). With the idea that WKB is the cheapest representation to generate that can be used for algorithms like unique and factorize, without implementing them ourselves (in principle we could also code them in cython using the geos interface for equality and populating a table)
jorisvandenbossche
force-pushed
the
extension-array
branch
from
10c58b7 to
05162c7
Compare
|
Remaining error for pandas master is a bug in extension arrays in pandas: pandas-dev/pandas#20743 |
Codecov Report
@@ Coverage Diff @@
## geopandas-cython #701 +/- ##
====================================================
- Coverage 91.32% 88.55% -2.77%
====================================================
Files 16 16
Lines 1694 1783 +89
====================================================
+ Hits 1547 1579 +32
- Misses 147 204 +57
Continue to review full report at Codecov.
|
|
The ExtensionArray interface how been implemented in the mean time. |
First pass to implement the extension array interface. All interface tests are passing except the ones that rely on sorting.
Still need to better integrate with block code to support released version of pandas as well.
Some other tests are still failing: pandas-dev/pandas#20576, pandas-dev/pandas#20578