🌟 Parity to IPFS

[daviddias]

Update: I rewrote this first post to make it more useful for tracking the developments of this.

Goal: Add a 'Storage Backend' for js-ipfs that backed by Parity, the Rust implementation of Ethereum. This will enable js-ipfs nodes to fetch content from the Ethereum Network as if it was just a big bucket with all the blocks from the chain.

TASKS

• rust-cid
• Implement an http endpoint in Parity that lets us retrieve the blocks
• Implement the datastore backend that mounts on top of parity
• Try it out, a lot :)

[kumavis]

from parity mh-ipfs branch

cargo build --release
./target/release/parity

while thats running, you can make requests by cid

curl 'http://localhost:5001/api/v0/block/get?arg=z45oqTRzb9a5xyvx5RbfSXH1K5jibyZ4AxnXyYReuLw7KU5veYw'

[kumavis]

i made this util to help with this https://github.com/kumavis/eth-ipld

[daviddias]

@kumavis let's pick up on this again, now that we have datastore bubbling up all the way :)

[kumavis]

whee

[kumavis]

seems simple enough

https://github.com/ipfs/js-datastore-level
https://github.com/ipfs/interface-datastore

how do you mount multiple stores? and can you mount one as read-only?

[kumavis]

also - do you have a preferred isomorphic xhr module? otherwise ill use what i usually do
EDIT: I guess the http module works fine for browserify, not sure about webpack

[daviddias]

We shim stream-http in js-ipfs-api https://github.com/ipfs/js-ipfs-api/blob/master/package.json#L10

The code for reaching to the block store, given that it exposes the same API as go-ipfs does for blocks, is exactly what js-ipfs-api does https://github.com/ipfs/js-ipfs-api/blob/master/src/api/block.js

On read-only, it means that essentially when it calls write, you say 'All good folks', but in reality does nothing.

Right now, ipfs-repo is mounting a single datastore for the blockstore (blocks folder) https://github.com/ipfs/js-ipfs-repo/blob/master/src/index.js#L136-L139, which is them used throughout blockstore code: https://github.com/ipfs/js-ipfs-repo/blob/master/src/blockstore.js#L34-L137

Tasks:

• Ability to mount more than one datastore for blockstore (make it an array essentially)
• Make ./blockstore check if there is an array of this.store and issue the get/put to all of them.

Note: blockstore will pass a base32 encoded CID to your parity-datastore, which you then have to decode to send through the js-ipfs-api block.get.

[kumavis]

would be nice if js-ipfs-repo's data store could be specified externally somehow
seems hardcoded in with a lot of other config https://github.com/ipfs/js-ipfs-repo/blob/master/src/index.js#L136-L139

[daviddias]

Agreed, right now the decision is being made inside the ipfs-repo module by signalling through the browser property in the package.json, here:

https://github.com/ipfs/js-ipfs-repo/blob/master/package.json#L8

I would rather have those options passed in, so that we can hot swap it per application basis, just like libp2p bundles.

[kumavis]

would be nice if we could boot the http api by passing in an instance of an ipfs node, instead of it needing to boot its own

[kumavis]

for now, i think this will work for ipfs-repo's datastore but NotAGoodIdea™

node.on('ready', () => {
  console.log('node ready!')
  // hacky late add to MountStore
  console.log('Adding mount to Parity')
  repo.store.mounts.push({
    prefix: new Key('/'),
    datastore: new RemoteIpfsDataStore({host: 'localhost', port: '5002', protocol: 'http'})
  })
  setupHttpApi()
})