Skip to content

Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque<usize> #15573

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

Both @rluvaton and I have seen

Image

Image

I saw this when running the following query

./datafusion-cli-filter-pushdown -c "SELECT \"WatchID\", \"ClientIP\", COUNT(*) AS c, SUM(\"IsRefresh\"), AVG(\"ResolutionWidth\") FROM hits WHERE \"SearchPhrase\" <> '' GROUP BY \"WatchID\", \"ClientIP\" ORDER BY c DESC LIMIT 10;"

Using hits.parquet: https://datasets.clickhouse.com/hits_compatible/hits.parquet

I am not sure what @rluvaton was doing when he saw it

Describe the solution you'd like

Review and see if this is a performance bottleneck we can remove

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions