Skip to content

Support file pruning through JsonPredicates and predicates queryRequest parameters #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

duhizjame
Copy link
Collaborator

No description provided.

Copy link

@tmnd1991 tmnd1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still some work to do (I know, this PR is huge!) but very good job until here

@duhizjame duhizjame force-pushed the feature/93-support-parameters-predicatehints-and-jsonpredicatehints-in-querytable-api branch from ca5bb6c to 9553187 Compare December 8, 2023 19:58
@duhizjame duhizjame force-pushed the feature/93-support-parameters-predicatehints-and-jsonpredicatehints-in-querytable-api branch from dadbfdc to fa40ab3 Compare December 16, 2023 14:22
@duhizjame duhizjame force-pushed the feature/93-support-parameters-predicatehints-and-jsonpredicatehints-in-querytable-api branch from eba1ae8 to 429e8a9 Compare December 28, 2023 01:59
@duhizjame duhizjame force-pushed the feature/93-support-parameters-predicatehints-and-jsonpredicatehints-in-querytable-api branch 2 times, most recently from 9fc7980 to 9948574 Compare January 19, 2024 02:50
@duhizjame duhizjame force-pushed the feature/93-support-parameters-predicatehints-and-jsonpredicatehints-in-querytable-api branch from 26e9791 to 8f18c09 Compare January 30, 2024 00:06
@duhizjame
Copy link
Collaborator Author

refactored a lot of stuff based on the comments;
notable change: INTEGER type is now formed from string "int" instead of "integer" to match the types from the JsonPredicateUtils
all other changes are mostly refactoring for readability and deduplication of code

@agilelab-tmnd1991
Copy link
Collaborator

agilelab-tmnd1991 commented Jan 30, 2024

refactored a lot of stuff based on the comments; notable change: INTEGER type is now formed from string "int" instead of "integer" to match the types from the JsonPredicateUtils all other changes are mostly refactoring for readability and deduplication of code

Is this compliant with the spec? https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#primitive-types

@duhizjame
Copy link
Collaborator Author

duhizjame commented Jan 30, 2024

it should be; based on the actual protocol update in #203(line 585 in the delta protocol) + even delta standalone refers to this type by its short name
io/delta/standalone/types/IntegerType.java

all other usages of IntegerType(the io.whitefox.types one) is our internal usage

@agilelab-tmnd1991 agilelab-tmnd1991 dismissed tmnd1991’s stale review January 31, 2024 10:33

because it's me from another account

@agilelab-tmnd1991 agilelab-tmnd1991 changed the title wip-first draft of JsonPredicates and a test partitioned table Support file pruning through JsonPredicates and predicates queryRequest parameters Jan 31, 2024
@agilelab-tmnd1991 agilelab-tmnd1991 merged commit 94aefa6 into main Jan 31, 2024
@agilelab-tmnd1991 agilelab-tmnd1991 deleted the feature/93-support-parameters-predicatehints-and-jsonpredicatehints-in-querytable-api branch January 31, 2024 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support parameters predicateHints and jsonPredicateHints in QueryTable API
3 participants