Description
Is your feature request related to a problem or challenge?
As DataFusion matures and becomes the foundation for more and more systems, ensuring it works bug free with many SQL queries is even more important. We have our own substantial test suites, but we still find issues from time to time, and more systematic testing would be even better
This ticket attempts to
- summarize [DISCUSSION] More SqlLogicTest test coverage for queries, including join queries #13470
- organize the follow on items that @Omega359 discovered during initial implementation
Describe the solution you'd like
As described on #13470, there is a well known, very large corpus of sql test files originally from sqlite in sqllogictest format. It is common for other sql engines to run these scripts
The main description page is: https://sqlite.org/sqllogictest/file?name=about.wiki&ci=tip
@Omega359 has undertaken a substantial effort to create a DataFusion test runner to run these tests
I believe the code is here
Describe alternatives you've considered
Infrastructure Tasks
- prototype test runner: https://github.com/Omega359/arrow-datafusion/tree/feature/sqllogictest_add_sqlite
- Triage initial results (thanks @Omega359 ❤️)
- Complete test runner integration: Complete / integrate sqlite sqllogictest test scripts integrattion #13812
- Refactor sqllogictest to extract postgres functionality into a separate file #13948
- Validate all remaining failures in sqlite tests #13972
- Automatically run sqlitetests regularly (but not with all PRs) to DataFusion #13967
- sql result discrepency with sqlite, postgres and duckdb bug #3 #13784
- [EPIC] Run full sqllogic / sqlite test suite against DataFusion #13811
- Automate updating sqllogictest updates #14158
Feature Gaps
Fix bugs discovered by the initial testing
- sql result discrepency with sqlite, postgres and duckdb bug #3 #13784
- sql result discrepency with sqlite, postgres and duckdb bug #2 #13782
- sql odd case of rounding compared to duckdb and postgresql #13781
- sql result discrepency with sqlite, postgres and duckdb #13780
- sql result discrepency with sqlite and postgres #13779
- multiply overflow in stats.rs #13775
- sqlite test query results in Internal error: predicate did not evaluate to an array #14099
Additional context
No response