Skip to content

FileNotFoundError when accessing same file from multiple processes #8411

Closed as not planned
@prateekjjw001

Description

@prateekjjw001

What is your issue?

I am trying to access the same file using xr.load_dataset() from multiple processes. They are all trying to read it at the same time (or within 0.1s of each other). But only the 1st process is able to access it, while the other processes are not able to read it. It gives a generic "FileNotFoundError" even though the file is there. The file is written about 2-3s before it is read by different processes. Is this expencted? Earlier I suspected xr.open_dataset() to be the culprit but removing that with load_dataset() also did not solve the issue.

The issue is sporadic and cannot be reproduced easily but it happens in our production process. Any suggestions please?

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs mcvehttps://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions