Skip to content

feat: Add conversion from FileMetaData to ParquetMetadata #1074

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jonathanc-n
Copy link
Contributor

Which issue does this PR close?

What changes are included in this PR?

Add conversion from filemetadat to parquet metadata using thrift decode_metadata

Are these changes tested?

liurenjie1024
liurenjie1024 previously approved these changes Mar 13, 2025
Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n , LGTM! Just a minor concern about thrift depdenency, let's wait for @Xuanwo to take a look.

Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most changes looks good to me. It's better if we can remove some unwarp().

@liurenjie1024 liurenjie1024 dismissed their stale review March 14, 2025 01:38

We should remove the unwrap here.

@jonathanc-n
Copy link
Contributor Author

@Xuanwo @liurenjie1024 Should be fine now. Thanks for the review!

Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @jonathanc-n !

@liurenjie1024 liurenjie1024 merged commit 3cc1a68 into apache:main Mar 14, 2025
18 checks passed
@jonathanc-n jonathanc-n deleted the change-parquet-metadata-version branch March 28, 2025 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consolidate methods of converting parquet file to data file builder.
3 participants