Add simple archives.ipfs.io website

[ghost]

I've put this together from an older existing archives website that @davidar pointed me to, and put it up at https://archives.ipfs.io, so we can point to it at 33c3.

[ghost]

cc @flyingzumwalt @RichardLitt

[victorb]

Looks good and simple to me. One question though, how can we make sure the hashes actually resolves and are shared by someone? Should have the hashes in some list somewhere and also something checking once a week or something that the hashes resolves.

[Kubuxu]

LGTM, what do you think of referencing hashes in fs:/ipfs/Qm...AAA form?

[eminence]

I think I was the one who wrote the initial version of this page (in #26). I think I have a newer version around, let me poke around my ipfs scratch space

[eminence]

Here's an updated version:

https://ipfs.io/ipfs/QmNMRuvBxoiBrnECYUFAaXEKVimjZpXfWUbk5J7r4NuDRj/

[eminence]

@victorbjelkholm

One question though, how can we make sure the hashes actually resolves and are shared by someone?

A great question. I'm positive that some of these hashes are no longer fully hosted.

I'm not sure what the best way to ensure they all are hosted, but I think it would be cool to have some kind of volunteer network to pin all of the archived data. I have a few TB to give to this purpose, but that won't be enough for everything.

[ghost]

I'm gonna drop ipfs-cluster here (cc @hsanjuan) -- let's continue this in another issue

[flyingzumwalt]

Could we also include a way for people to tell us about new archives? I have a hunch there are a lot more datasets out there, with many more to come in the next few months.

Keep in mind that this should just be a stop-gap. I want to encourage people to publish IPLD metadata about their datasets so that anyone can build (and display) registries of the datasets they care about. There's a lot of work and experimentation in this area. I don't want to distract people from participating in that experimentation, but I do agree that in the meantime we want to have some listing of datasets that are available via IPFS. (Related: this call to form a community around decentralied web in Libraries Archvies & Museums )

[flyingzumwalt]

(if you want to merge this PR, I can raise these longer-term ideas/concerns in another issue)

[ghost]

Updated version here: https://ipfs.io/ipfs/QmNhjxBrtYoZyWqJuuGMo57yzCBv4WD22d9ktP1TtM2qbz/

• @eminence: I added/updated the hashes from your version, also removed all assets apart from bootstrap.min.css
• @Kubuxu: I changed the hashes to paths at least. I think the firefox addon is the only place so far where we support fs: URIs. We should up that in 2017 :)
• @flyingzumwalt yes completely agreed 👍 this can only be a first step to making all this more visible and discoverable.

[Kubuxu]

I would change the hrefs to be relative /ipfs/Qm from including the https://ipfs.io/.

[jbenet]

Very cool!

I echo @flyingzumwalt's thoughts. To push closer on this, can we decide on a set of recommendations for datasets on ipfs, that we can make a requirement for our listing? (could be use the ipfs-pack tool to ensure we have a complete manifest, have a data-packages.json (or whatever similar), have a per-dataset page that is generated from the metadata file, ensure we can replicate the data (open access), etc. Ideally, i'd like this in a tool (avail in both command-line and web) that guides people through publishing a dataset.

We should figure out what the index should look like, too--

• whether to assign a unique short name (like a package name in other registries), or a number (sequential or random), etc.
• how to replicate the index, and all it contains, etc.

Replicating these datasets, and the entire archive, would be a great test case for ipfs-cluster.

[jbenet]

cc @davidar @hsanjuan o/

[jbenet]

(Btw, "the index" here is just a single log (hashchain/blockchain/whatever you wanna call it), a simple data structure that tracks all the datasets, and their versions.)

[eminence]

What's the process to to get this page updated in the future? PR against this repo and then ping @lgierth to get it push to https://archives.ipfs.io/ ?

[hsanjuan]

I'm not sure what the best way to ensure they all are hosted, but I think it would be cool to have some kind of volunteer network to pin all of the archived data. I have a few TB to give to this purpose, but that won't be enough for everything.

Yes! I have added this as an ipfs-cluster user-story with some notes from the top of my head: ipfs-cluster/ipfs-cluster#7 . If anyone has further thoughts please add them there.

I find this is a going to be a widely requested use-case which we should strive to cover in the near future.