Release DataFusion `51.0.0` (Nov 2025)

[alamb]

Is your feature request related to a problem or challenge?

Tracking ticket for next release, also a place to track desired inclusions

Previous release will be #16799 (Sep, 2025) so next major release would be approximately late Nov 2025

Steps:

• Create a branch-51 branch:
• Create blog post: Blog post for the DataFusion 50.0.0 release #16931
• Update version and changelog:
• Test with DataFusion Python
• Test with DataFusion Comet
• Test with delta.rs:
• Test vortex:
• Test with iceberg-rust:
• Test with LakeSail:
• Test with parquet viewer
• Test with datafusion-materialized-views
• Voting Thread:
• Publish to crates.io
• Create ticket for next release:

TODOs

Prior release tickets:

• Release DataFusion 50.0.0 (Aug/Sep 2025) #16799

Related

• Boston USA Nov 2025 meetup: DISCUSSION: DataFusion Meetup in Boston, USA - Nov 12, 2025 #16703

Features to mention in the blog (if they make it)

• Support Decimal32/64 types #17501 from @AdamGS
• Support WHERE, ORDER BY, LIMIT, SELECT, EXTEND pipe operators #17278 (comment)
• feat: change Expr OuterReferenceColumn and Alias to Box type for reducing expr struct size #16771
• Examples of extending SQL syntax #17824 from the @theirix

Bugs that need to be fixed

• Restore window sort optimizations without exponential planning time #17624
• Exponential planning time when window function is partitioned by multiple columns #17401
• Optimizer 'common_sub_expression_eliminate' fails in a window function (SQLStorm) #17770 / fix: optimizer common_sub_expression_eliminate fails in a window function #17852
• Regression: sql_planner benchmark panic'ing on main #17801
• to_timestamp(double) gives different results depending on scalar/vectorized call context #16678
• Streaming Aggregate operator not being used in deduplication of pre-sorted Parquet files #16919

Community Wishlist

• Regression: DataFrameWriteOptions::with_single_file_output produces a directory #13323

[mjgarton]

Should the title of this issue say 51.0.0 instead of 50.0.0?

[xudong963]

Should the title of this issue say 51.0.0 instead of 50.0.0?

yes, thank you, i changed it.

[findepi]

I removed #16842 from "Changes to add to upgrade guide". I think it was completed in 50.0.0. If not, let's add it back.