Skip to content

Conversation

rambleraptor
Copy link
Contributor

Closes #14265

This adds promotion logic for date->timestamp, as well as proper plumbing to ensure that these promotions only occur for Table Format v3

Copy link
Contributor

@talatuyarer talatuyarer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see few things are missing:

  • Add partition field validation logic
  • Add test for partition field validation
  • Add negative test: v2 tables rejecting date→timestamp
  • Add negative test: v3 tables rejecting date→timestamptz
  • update formatVersion parameter in JavaDoc
  • Please update adding integration test
  • Add promotion test on TestSchemaUnionByFieldName.java
  • add tests for Parquet another data formats that verifies that data written with old types can be read correctly after promotion.

@talatuyarer
Copy link
Contributor

One more thing Did you have a chance to run revapi ? @rambleraptor

https://iceberg.apache.org/contribute/#checking-for-api-breakages

@rambleraptor
Copy link
Contributor Author

@talatuyarer revapi is passing

@rambleraptor
Copy link
Contributor Author

@talatuyarer changes made! I'm unsure what a Parquet test would look like in practice. Doesn't look like we have an existing test that I can work off of.

Copy link
Contributor

@nandorKollar nandorKollar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I took a similar approach in my abandoned PR.

@rambleraptor
Copy link
Contributor Author

@nandorKollar thanks so much for the review!

@RussellSpitzer
Copy link
Member

Do we account for metrics written as the pre promotion type? I think this code is already part of metrics evaluation but we should test to make sure a promoted field still filters properly

@rambleraptor
Copy link
Contributor Author

@RussellSpitzer @talatuyarer I just added a metrics test for Parquet, which I think covers the use case you're describing. Let me know if it doesn't.

rambleraptor and others added 6 commits October 12, 2025 17:01
This adds promotion logic for date->timestamp, as well as proper
plumbing to ensure that these promotions only occur for Table Format v3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow for v3 date -> time promotion

5 participants