GH-39220: [Python] Let RecordBatch.filter accept a boolean expression in addition to mask array

[jorisvandenbossche]

Rationale for this change

Table.filter() already accepted either a boolean mask array or a boolean expression. But the equivalent method on RecordBatch only accepted the array. This makes both methods consistent in accepting both types of mask.

What changes are included in this PR?

Consolidate the Table.filter and RecordBatch.fitler methods into a single shared method on the base class, and expanded the _filter_table Acero helper to also work with RecordBatch in addition to Table (and ensure to return a batch if the input was a batch)

Are these changes tested?

Yes

• GitHub Issue: [Python] Extend RecordBatch.filter to take an Expression in addition to a boolean mask Array #39220

[github-actions]

⚠️ GitHub issue #39220 has been automatically assigned in GitHub to PR creator.

[wjones127]

Does this mean Table.filter() will no longer work if PyArrow was not built with Acero? I forget how optional that is considered.

[jorisvandenbossche]

Does this mean Table.filter() will no longer work if PyArrow was not built with Acero? I forget how optional that is considered.

It will still work without Acero for a plain boolean mask (in that case we only require the compute kernel), only for filtering with an expression it needs Acero.

(and note this was already the case for Table.filter, this PR is just adding the same capability to RecordBatch.filter)

[wjones127]

That makes sense. Thanks for explaining!

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 577cafb.

There was 1 benchmark result indicating a performance regression:

• Commit Run on ec2-m5-4xlarge-us-east-2 at 2024-06-26 17:32:21Z
  • partitioned-dataset-filter (R) with dataset=dataset-taxi-parquet, language=R, query=vignette

The full Conbench report has more details. It also includes information about 12 possible false positives for unstable benchmarks that are known to sometimes produce them.