Skip to content

SettingWithCopyWarning in job manager #641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
soxofaan opened this issue Oct 4, 2024 · 2 comments
Closed

SettingWithCopyWarning in job manager #641

soxofaan opened this issue Oct 4, 2024 · 2 comments
Assignees

Comments

@soxofaan
Copy link
Member

soxofaan commented Oct 4, 2024

reported by Joris C:

Not sure whether it's due to a change in the job manager, or whether it was there already, but I'm getting a warning on the line here:
https://github.com/Open-EO/openeo-python-client/blob/master/openeo/extra/job_management.py#L650
SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
active.loc[i, "running_start_time"] = rfc3339.utcnow()

@soxofaan
Copy link
Member Author

soxofaan commented Oct 9, 2024

I already looked into this issue, but I could not yet reproduce this warning from using the job manager.

I can reproduce it with some simple pandas usage like

df = pandas.DataFrame({"foo": range(10)})
active = df[df.foo % 3 == 0]
active.loc[3, "foo"] = 666

but I would need a more practical usage example of the job manager that triggers this

soxofaan added a commit that referenced this issue Oct 11, 2024
seems like pandas is handy to build foot guns

maybe related to #641
soxofaan added a commit that referenced this issue Nov 13, 2024
by making the `copy()` explicit
@soxofaan
Copy link
Member Author

I could somehow reproduce the problem and 803fa2b seems to fix it. Unfortunately I didn't find a way to capture it in a unit test (because I could only reproduce it by hacks I don't want to keep in the code, and I couldn't wrap my head around the exact conditions that trigger the warning), so I'll just merge this and close this ticket. Will be part of next release (0.35.0). Feel free to reopen when problem persists

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant