Skip to content

partial overwrite operation #1840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

kevinjqliu
Copy link
Contributor

Rationale for this change

Are these changes tested?

Are there any user-facing changes?

@kevinjqliu kevinjqliu force-pushed the kevinjqliu/overwrite-summary branch from 0b65090 to 366c2df Compare March 24, 2025 23:25
@kevinjqliu kevinjqliu force-pushed the kevinjqliu/overwrite-summary branch from 366c2df to 5bbcb11 Compare March 24, 2025 23:27
@kevinjqliu kevinjqliu changed the title produce overwrite operation partial overwrite operation Mar 24, 2025
@Fokko Fokko added this to the PyIceberg 0.9.1 milestone Mar 26, 2025
@Fokko
Copy link
Contributor

Fokko commented Apr 3, 2025

Ahh thanks @kevinjqliu I don't think we've really implemented this properly.

image

Java produces:

{
    "added-data-files": "1",
    "added-files-size": "707",
    "added-records": "2",
    "app-id": "local-1743678304626",
    "changed-partition-count": "1",
    "deleted-data-files": "1",
    "deleted-records": "3",
    "engine-name": "spark",
    "engine-version": "3.5.5",
    "iceberg-version": "Apache Iceberg 1.8.1 (commit 9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d)",
    "removed-files-size": "693",
    "spark.app.id": "local-1743678304626",
    "total-data-files": "3",
    "total-delete-files": "0",
    "total-equality-deletes": "0",
    "total-files-size": "1993",
    "total-position-deletes": "0",
    "total-records": "4"
}

@kevinjqliu
Copy link
Contributor Author

Closing in favor of #1879

@kevinjqliu kevinjqliu closed this Apr 11, 2025
@kevinjqliu kevinjqliu deleted the kevinjqliu/overwrite-summary branch April 17, 2025 18:36
@kevinjqliu kevinjqliu removed this from the PyIceberg 0.9.1 milestone Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants