Closed
Description
Is your feature request related to a problem or challenge?
#14677 introduces basic tree rendering for SQL EXPLAIN output but requires follow-up work to improve completeness, test coverage, and code quality. Below are the key tasks and enhancements needed:
Here is an example of running the new explains:
set datafusion.explain.format = 'tree';
create table foo(x int, y int) as values (1,2), (3,4);
explain select * from foo where x = 4;
The output looks like
+---------------+------------------------------------+
| plan_type | plan |
+---------------+------------------------------------+
| logical_plan | Filter: foo.x = Int32(4) |
| | TableScan: foo projection=[x, y] |
| physical_plan | ┌───────────────────────────┐ |
| | │ CoalesceBatchesExec │ |
| | └─────────────┬─────────────┘ |
| | ┌─────────────┴─────────────┐ |
| | │ FilterExec │ |
| | └─────────────┬─────────────┘ |
| | ┌─────────────┴─────────────┐ |
| | │ DataSourceExec │ |
| | │ -------------------- │ |
| | │ partition_sizes: [1] │ |
| | │ partitions: 1 │ |
| | └───────────────────────────┘ |
| | |
+---------------+------------------------------------+
Items to complete
Information Completion for all ExecutionPlans
The current implementation lacks detailed information for many physical plans.
Add rendering logic for missing operators (In every physical plan's fmt_as
function).
- Implement
tree
explain forFilterExec
#15000 - Implement tree explain for
DataSourceExec
#15029 - Implement
tree
explain forProjectionExec
#15023 - Implement
tree
explain forAggregateExec
#15024 - Implement
tree
explain forLocalLimitExec
#15025 - Implement
tree
explain forGlobalLimitExec
#15026 - Implement tree explain for
PartialSortExec
#15064 - Implement
tree
explain forSortExec
#15076 - feat: Implement tree explain for
RepartitionExec
andWorkTableExec
#15097 - Implement
tree
explain forBoundedWindowAggExec
andWindowAggExec
#15083 - Implement tree explain for
SortPreservingMergeExec
#15139 - Implement
tree
explain forHashJoinExec
#15078 - Implement
tree
explain forNestedLoopJoinExec
,CrossJoinExec
,SortMergeJoinExec
andSymmetricHashJoinExec
#15080 - Implement tree rendering for
StreamingTableExec
#15086 - Implement
tree
explain forValuesExec
#15093 - Implement tree explain for InterleaveExec #15196
- Implement tree explain for LazyMemoryExec #15171
- Implement
tree
explain forForeignExecutionPlan
- Implement
tree
explain forJsonSink
#15089 - Implement tree explain for CsvSink #15186
- Implement tree explain for
ArrowFileSink
#15112 - Implement tree explain for
PlaceholderRowExec
#15138 - Implement tree explain for
CoalesceBatchesExec
#15141 - Implement tree explain for
CoalescePartitionsExec
#15195 - Implement tree explain for UnionExec #15277
- (TODO file more)
New Features:
- Support different
EXPLAIN
formats via SQL #15021 - Do not print logical plans in
tree
explain format #15020 - Simpler / easier to see expressions in
tree
explain mode #15107 - Simplify the printing of all plans containing
expr
intree
mode. #15238 - [tree explain] Simplify display format of
AggregateFunctionExpr
#15252
Process / Documentation
- Document the
tree
explain in the user documentation - Add upgrade guide in 47 for
TreeRender
in DisplayFormatType` - Use
tree
explain by default
Test Coverage Expansion
- Add tests for plans with long metrics (e.g., partition_sizes: [20, 20, 20, 30, ...]) to validate line-wrapping and truncation logic.
- Tests for TPCH queries (e.g., Q5, Q9) to test rendering of deeply nested plans.
- Validate alignment and indentation for multi-level operators.
Code Cleanup & Improvements
TODOs
- Improve
explain tree
formatting for longer lines / word wrap #15019 - Improve Parsing for KV Format in
tree
explain. #15098
Make variables (e.g., indentation size, truncation thresholds) configurable:
// TODO: Make these variables configurable.
Dead Code Removal
Remove or utilize the unused Coordinate struct:
#[allow(dead_code)]
pub struct Coordinate { ... }
// TODO: It's never used.