Closed
Description
Feature Request / Improvement
Pre-req: Arrow 17 (#929)
Details
pyarrow.dataset.Scanner.from_fragment
's filter
parameter now takes an Expression
type in Arrow 17.
https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Scanner.html#pyarrow.dataset.Scanner.from_fragment
We can try to push down both the user specified filters and the positional deletes.
iceberg-python/pyiceberg/io/pyarrow.py
Lines 1232 to 1234 in f05b1ae
and
iceberg-python/pyiceberg/io/pyarrow.py
Lines 1243 to 1253 in f05b1ae
Requires Arrow 17
Metadata
Metadata
Assignees
Labels
No labels