Skip to content

Implement incremental updating of table metadata #454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Fokko opened this issue Feb 20, 2024 · 0 comments · Fixed by #471
Closed

Implement incremental updating of table metadata #454

Fokko opened this issue Feb 20, 2024 · 0 comments · Fixed by #471
Assignees

Comments

@Fokko
Copy link
Contributor

Fokko commented Feb 20, 2024

Feature Request / Improvement

When we accumulate changes when doing updates to the metadata, we want to accumulate the changes in the metadata.

For example, if you first add a new column to a table, and then directly write some data, it should take the updated schema. The game goes for when you update the table properties and do a write after that, it should take the table properties into account.

This is an important concept that's currently not working of PyIceberg. For example, an upsert operation in PyIceberg is modeled as first a delete, and then an append operation. This way we want to stack the operations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant