Move basic SQL query examples to user guide #11217
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
core
alamb
commented
Jul 2, 2024
| @@ -592,3 +592,9 @@ doc_comment::doctest!( | |||
| "../../../docs/source/user-guide/example-usage.md", | |||
| user_guid_example_tests | |||
| ); | |||
|
|
|||
| #[cfg(doctest)] | |||
There was a problem hiding this comment.
This means these tests are run as part of the doc tests
$ cargo test --doc --features avro,json -- library_user_guide
...
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.16s
Doc-tests datafusion
running 5 tests
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 764) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 718) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 673) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 775) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 641) ... ok
...
alamb
force-pushed
the
alamb/sql_user_guide
branch
from
07bd95b to
f21b576
Compare
alamb
commented
Jul 2, 2024
| - [`parquet_sql_multiple_files.rs`](examples/parquet_sql_multiple_files.rs): Build and run a query plan from a SQL statement against multiple local Parquet files | ||
| - [`parquet_exec_visitor.rs`](examples/parquet_exec_visitor.rs): Extract statistics by visiting an ExecutionPlan after execution | ||
| - [`parse_sql_expr.rs`](examples/parse_sql_expr.rs): Parse SQL text into Datafusion `Expr`. | ||
| - [`plan_to_sql.rs`](examples/plan_to_sql.rs): Generate SQL from Datafusion `Expr` and `LogicalPlan` | ||
| - [`pruning.rs`](examples/parquet_sql.rs): Use pruning to rule out files based on statistics | ||
| - [`pruning.rs`](examples/pruning.rs): Use pruning to rule out files based on statistics |
alamb
commented
Jul 2, 2024
| @@ -47,10 +47,8 @@ cargo run --example csv_sql | |||
| - [`advanced_udf.rs`](examples/advanced_udf.rs): Define and invoke a more complicated User Defined Scalar Function (UDF) | |||
There was a problem hiding this comment.
The removed examples were just inlined into the guide
alamb
commented
Jul 2, 2024
| It is also possible to read multiple files as a single table. This is done | ||
| with the ListingTableProvider which takes a list of file paths and reads them | ||
| as a single table, matching schemas as appropriate | ||
|
|
||
| Coming Soon |
There was a problem hiding this comment.
This would be cool to do but this PR is already large enought
This was referenced Jul 2, 2024
|
Thank you for the review @jonahgao |
comphead
pushed a commit
to comphead/arrow-datafusion
that referenced
this pull request
Jul 8, 2024
findepi
pushed a commit
to findepi/datafusion
that referenced
this pull request
Jul 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Fixes #11210
Part of #11172 and #1813
Rationale for this change
Hopefully this fixes the CI failure we were seeing in #11210 while also improving the documentation
There are several places in the existing documentation for running SQL via SessionContext
Thus we don't need another one in the examples directory
What changes are included in this PR?
cargo doc ...)Are these changes tested?
Yes, they are run via doctests
Are there any user-facing changes?
New docs