-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Move basic SQL query examples to user guide #11217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -36,9 +36,9 @@ cd datafusion | |
# Download test data | ||
git submodule update --init | ||
|
||
# Run the `csv_sql` example: | ||
# Run the `dataframe` example: | ||
# ... use the equivalent for other examples | ||
cargo run --example csv_sql | ||
cargo run --example dataframe | ||
``` | ||
|
||
## Single Process | ||
|
@@ -47,10 +47,8 @@ cargo run --example csv_sql | |
- [`advanced_udf.rs`](examples/advanced_udf.rs): Define and invoke a more complicated User Defined Scalar Function (UDF) | ||
- [`advanced_udwf.rs`](examples/advanced_udwf.rs): Define and invoke a more complicated User Defined Window Function (UDWF) | ||
- [`advanced_parquet_index.rs`](examples/advanced_parquet_index.rs): Creates a detailed secondary index that covers the contents of several parquet files | ||
- [`avro_sql.rs`](examples/avro_sql.rs): Build and run a query plan from a SQL statement against a local AVRO file | ||
- [`catalog.rs`](examples/catalog.rs): Register the table into a custom catalog | ||
- [`composed_extension_codec`](examples/composed_extension_codec.rs): Example of using multiple extension codecs for serialization / deserialization | ||
- [`csv_sql.rs`](examples/csv_sql.rs): Build and run a query plan from a SQL statement against a local CSV file | ||
- [`csv_sql_streaming.rs`](examples/csv_sql_streaming.rs): Build and run a streaming query plan from a SQL statement against a local CSV file | ||
- [`custom_datasource.rs`](examples/custom_datasource.rs): Run queries against a custom datasource (TableProvider) | ||
- [`dataframe-to-s3.rs`](examples/external_dependency/dataframe-to-s3.rs): Run a query using a DataFrame against a parquet file from s3 and writing back to s3 | ||
|
@@ -66,12 +64,11 @@ cargo run --example csv_sql | |
- [`memtable.rs`](examples/memtable.rs): Create an query data in memory using SQL and `RecordBatch`es | ||
- [`optimizer_rule.rs`](examples/optimizer_rule.rs): Use a custom OptimizerRule to replace certain predicates | ||
- [`parquet_index.rs`](examples/parquet_index.rs): Create an secondary index over several parquet files and use it to speed up queries | ||
- [`parquet_sql.rs`](examples/parquet_sql.rs): Build and run a query plan from a SQL statement against a local Parquet file | ||
- [`parquet_sql_multiple_files.rs`](examples/parquet_sql_multiple_files.rs): Build and run a query plan from a SQL statement against multiple local Parquet files | ||
- [`parquet_exec_visitor.rs`](examples/parquet_exec_visitor.rs): Extract statistics by visiting an ExecutionPlan after execution | ||
- [`parse_sql_expr.rs`](examples/parse_sql_expr.rs): Parse SQL text into Datafusion `Expr`. | ||
- [`plan_to_sql.rs`](examples/plan_to_sql.rs): Generate SQL from Datafusion `Expr` and `LogicalPlan` | ||
- [`pruning.rs`](examples/parquet_sql.rs): Use pruning to rule out files based on statistics | ||
- [`pruning.rs`](examples/pruning.rs): Use pruning to rule out files based on statistics | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. drive by cleanup |
||
- [`query-aws-s3.rs`](examples/external_dependency/query-aws-s3.rs): Configure `object_store` and run a query against files stored in AWS S3 | ||
- [`query-http-csv.rs`](examples/query-http-csv.rs): Configure `object_store` and run a query against files vi HTTP | ||
- [`regexp.rs`](examples/regexp.rs): Examples of using regular expression functions | ||
|
This file was deleted.
This file was deleted.
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -592,3 +592,9 @@ doc_comment::doctest!( | |
"../../../docs/source/user-guide/example-usage.md", | ||
user_guid_example_tests | ||
); | ||
|
||
#[cfg(doctest)] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This means these tests are run as part of the doc tests $ cargo test --doc --features avro,json -- library_user_guide
...
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.16s
Doc-tests datafusion
running 5 tests
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 764) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 718) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 673) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 775) ... ok
test datafusion/core/src/lib.rs - library_user_guide_example_usage (line 641) ... ok
... |
||
doc_comment::doctest!( | ||
"../../../docs/source/library-user-guide/using-the-sql-api.md", | ||
library_user_guide_example_usage | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The removed examples were just inlined into the guide