Skip to content

Conversation

findepi
Copy link
Member

@findepi findepi commented Aug 14, 2025

See individual commits

@github-actions github-actions bot added logical-expr Logical plan and expressions physical-plan Changes to the physical-plan crate labels Aug 14, 2025
It became redundant when `input_expr_types: &[DataType]` was replaced
with `input_expr_fields: &[Field]`, i.e. in
577c424 commit. The benefit of passing
in a "field" is that it combines data type and nullable attribute.
@findepi findepi force-pushed the findepi/drop-redundant-param-from-windowfunctiondefinition-return-field-bdd5f7 branch from 7c383fc to dbf832a Compare August 14, 2025 09:49
@findepi findepi requested review from alamb and timsaucer August 14, 2025 10:40
@github-actions github-actions bot added sql SQL Planner optimizer Optimizer rules core Core DataFusion crate catalog Related to the catalog crate datasource Changes to the datasource crate labels Aug 14, 2025
In `UserDefinedLogicalNode::check_invariants`, the actual plan to check
for invariants is `self`. The `plan` is always `LogicalPlan::Extension`
and provides no further information.  It's confusing.
@findepi findepi added the api change Changes the API exposed to users of the crate label Aug 14, 2025
@findepi findepi changed the title Drop redundant param from WindowFunctionDefinition::return_field Miscellaneous cleanups Aug 14, 2025
@github-actions github-actions bot added substrait Changes to the substrait crate common Related to common crate labels Aug 14, 2025
Copy link
Contributor

@timsaucer timsaucer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These all look like very nice cleanups. Thank you for the PR!

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi -- the only thing I am concerned about with this PR is the change to check_invariants. Otherwise this entire PR looks like a very nice improvement

field_qualifiers: vec![None; field_count],
functional_dependencies: FunctionalDependencies::empty(),
};
dfschema.check_names()?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context is that checking names here makes this consistent with the other (fallible) ways to create a DFSchema from a Schema

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW it seems this has caused a regression for some users in 50.0.0 (there are some code paths that violate an invariant). See #17706

@github-actions github-actions bot removed the substrait Changes to the substrait crate label Aug 14, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi and @timsaucer

@alamb alamb merged commit 0a024a2 into apache:main Aug 15, 2025
27 checks passed
@alamb alamb removed the api change Changes the API exposed to users of the crate label Aug 15, 2025
@findepi findepi deleted the findepi/drop-redundant-param-from-windowfunctiondefinition-return-field-bdd5f7 branch August 17, 2025 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
catalog Related to the catalog crate common Related to common crate core Core DataFusion crate datasource Changes to the datasource crate logical-expr Logical plan and expressions optimizer Optimizer rules physical-plan Changes to the physical-plan crate sql SQL Planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants