Skip to content

DataFrame select_exprs #12519

Closed
Closed
@milenkovicm

Description

@milenkovicm

Is your feature request related to a problem or challenge?

DataFrame exposes select_columns and select methods. The first of which expects list of columns to be selected, later one expects list of expressions created using logical expression API , which is straight forward but still needs some effort.

Describe the solution you'd like

With parse_sql_expr available, it may make sense to provide select_exprs which would accept list of expressions as strings, and convert them to actual logical expressions.

Something similar to:

pub fn select_exprs(self, exprs: &[&str]) -> Result<DataFrame> {
        let expr_list = exprs
            .iter()
            .map(|e| self.parse_sql_expr(e))
            .collect::<Result<Vec<_>>>()?;

        self.select(expr_list)
}

This would be equivalent to spark dataframe selectExp https://spark.apache.org/docs/3.4.2/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.selectExpr.html

Describe alternatives you've considered

No response

Additional context

#12518 looks like there is issue with parse_sql_expr and alias handling

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions