Support Open Science Foundation?

[haberdashPI]

Hi there; I love this project! What a great tool.

I'm wondering if you could add support for an open science framework project as a remote. It is for hosting data as part of a published research report. They have a python api for sending and retrieving files, so I'm guessing it would be pretty easy to add this feature. If it's not too difficult for someone new to this project, I would be happy to help out with adding this feature myself, if you could provide me some pointers on where to get started.

[efiop]

Hi @haberdashPI !

That looks very promising!

Just wanted to make sure you are aware that our remotes are not really human-readable and they are more like a content-based storage, so you usually need to use dvc to access your data by human readable name. 🙁

As to implementation, it would indeed pretty simple on dvc side. Here is an example of the whole code needed for azure https://github.com/iterative/dvc/blob/master/dvc/remote/azure.py . For osf the key points would be:

1. create dvc/remote/osf.py
2. add osf to

   dvc/dvc/config.py

   Line 179 in 253a0ea

   "azure": {"connection_string": str, **REMOTE_COMMON},

there might be some other tiny adjustments along the way.

Regarding the url format for osf, seems like osf://myproject/path should do the trick. Plus username(email) for the config. Note that I'm not too familiar with osf, so might be wrong here, so please let me know if I'm missing something here. 🙂

If you decide to take a shot implementing it, please don't hesitate to submit a WIP PR, we'll try to help with everything we can 🙂 Also, feel free to join our discord chat dvc.org/chat , we have #dev-talk channel there.

[haberdashPI]

Great thank you for the pointers!

Yes, I understand the data I stored in OSF using dvc would not be very human readable. However, you've done a good job documenting the format the data is stored in, so I think as long as I copy the documentation to my OSF repository that it would work pretty well for my purposes.

I am a bit busy at the moment, but hope to get to working on and submitting a PR in the next week or so. (But we'll see...)

[aerubanov]

I would like to try to implement this function

[haberdashPI]

Go for it! I started an attempt a while back but it looks like a bit has changed with the internals, and I won't have a chance to take a crack at it for at least another few months.

[efiop]

For the record: the only difference is that the example used to be in dvc/remote/ but now it is in dvc/tree 🙂

[efiop]

Doesn't seem like there is a lot of demand for this. If someone would be interested in having support for this, first step these days is to implemented fsspec fs for it (e.g. see https://github.com/fsspec/s3fs). Dvc core team is likely not going to support this officialy at all, but only as a third-party plugin. Closing for now.