Skip to content

Support adding parquet files to partitioned table #1035

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jonathanc-n opened this issue Mar 3, 2025 · 4 comments
Open

Support adding parquet files to partitioned table #1035

jonathanc-n opened this issue Mar 3, 2025 · 4 comments
Assignees
@jonathanc-n
Labels
enhancement New feature or request
Milestone
0.6.0 Release

Comments

@jonathanc-n
Copy link
Contributor

jonathanc-n commented Mar 3, 2025 •
edited
Loading

Looking at the type_to_literal function, I think this is correct. We don't want to set default values in the partition spec. I think there are two options:

Originally posted by @Fokko in #960 (comment)

Support adding parquet files to partitioned tables as is mentioned by @Fokko here
@jonathanc-n
Copy link
Contributor Author

I can also work on this

@liurenjie1024
Copy link
Contributor

I can also work on this

Thanks !

@liurenjie1024 liurenjie1024 added the enhancement New feature or request label Mar 3, 2025
@liurenjie1024 liurenjie1024 added this to the 0.5.0 Release milestone Mar 3, 2025
@jonathanc-n jonathanc-n mentioned this issue Mar 13, 2025
Merged
liurenjie1024 pushed a commit that referenced this issue Apr 8, 2025
@jonathanc-n
feat: Infer partition values from bounds (#1079)
e3ef617 
## Which issue does this PR close?



- Part of #1035.

## What changes are included in this PR?
Added API for creating partition struct from statistics


## Are these changes tested?
Will add tests after follow up pr for integrating it with the
add_parquet_file api
@liurenjie1024
Copy link
Contributor

cc @jonathanc-n Do you think it's better to move it to 0.6.0 milestone?

@jonathanc-n
Copy link
Contributor Author

Yes, I think #1116 is quite a large blocker. I can put a note with the function and we can move onto putting out the next release.

@jonathanc-n jonathanc-n mentioned this issue Apr 9, 2025
Merged
liurenjie1024 added a commit that referenced this issue Apr 11, 2025
@jonathanc-n @liurenjie1024
doc: add_parquet_files is not fully supported for version 0.5.0 (#1187
b2408cc 
)

## Which issue does this PR close?

- Mentioned in #1035

Co-authored-by: Renjie Liu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jonathanc-n jonathanc-n

Labels
enhancement New feature or request
Projects
iceberg-rust
Status: No status
Milestone
0.6.0 Release
Development

No branches or pull requests
2 participants
@liurenjie1024 @jonathanc-n