Skip to content

make modo add remote-compatible #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 tasks done
cmdoret opened this issue Mar 11, 2024 · 1 comment · Fixed by #30
Closed
2 tasks done

make modo add remote-compatible #22

cmdoret opened this issue Mar 11, 2024 · 1 comment · Fixed by #30
Assignees

Comments

@cmdoret
Copy link
Member

cmdoret commented Mar 11, 2024

Modo add creates a folder locally. To support remote write operations we can directly "mount" the s3 folder

Requirements:

  • rely on zarr's built-in s3 support to update metadata transparently
  • use s3fs (?) to manually upload / remove / delete files on s3

note for now we ignore access control and use anon buckets

@almutlue
Copy link
Contributor

almutlue commented Mar 13, 2024

Re metadata update:

TODO

  • find a fix for consolidate_metadata with remote objects
  • fix remove for remote storage
  • adapt cli to work with remote objects

@almutlue almutlue linked a pull request Mar 21, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants