IPNS notifications

[ajres]

Is there a mechanism for a Node to receive events when an IPNS of interest is updated, other than polling?

Andy

[harlantwood]

+1 very interested to know about this if it exists, or upvote the feature if not. @john-light I believe this would form a foundation for realtime chat, as well as a host of other applications, as we were just discussing...

[jbenet]

• yeah we really want this.
• we will be implementing a pubsub mechanism that uses ipns + the routing system
• pubsub gives you push notifications
• not clear how it will work exactly yet, but several solutions have been sketched out.

just one example, but by no means the best:

• store a "pubsub group" record under an ipns name.
• people subscribe to it by storing "pubsub subscribe" routing records at the group's key. (DSHT so can store many records per key)
• publishers would periodically check for subscriptions, keep a local list.
• (optionally) publishers gossip (like PEX) subscriber records to ensure they have a complete view
• publishers send notifications directly to subscribers.

other ways of doing this can perform much better (rely less on high perf of routing) if we have the subscribers or the publishers do a bit more work (maintaining the table or even relaying messages)

[gjvnq]

I think that sending notifications directly to the nodes is a bad idea because the node may be powered off when the message is sent.

One possible way around this is for nodes to publish requests like they would do with files but with backlinks to the channel IPNS address.

When the channel owner wants to send messages, they look for all requests that backlinks to the channel IPNS address and publish new messages (in a simlar way blocks are published) linking to the recipients.

A while later, when a node wants to checks its messages, it search for all messages that link to it.

[fps]

Hi, while it is not true pub/sub in the sense that a node gets a message about a change, but rather it involves polling, morphis has dpush, which is the technology underlying morphis' dmail. It basically consists of the publication of a target key prefix which the node will regularly scan using the prefix search inherent in kademlia.

it also depends on morphis' notion of updateable keys (that is documents can be updated while retaining their key). so i don't know how applicable it is to ipfs, but maybe there is enough overlap. here's some links for further details:

https://morph.is/
https://morph.is/v0.8/faq.html

and especially:

https://morph.is/v0.8/dpush-whitepaper.odt

there's also a subreddit:

http://reddit.com/morphis

and an irc channel on freenode: #morphis

looking forward to productive exchange between the projects :)

[davidar]

it also depends on morphis' notion of updateable keys

IPNS would be the equivalent here.

[DaneBettis]

I am also interested in this feature: a pub/sub system, even if not really real-time, would enable a robust VFX asset usage pipeline.

I'd be glad to further discuss options and possibly provide some real-world testing of ipfs in the VFX content development production pipeline.

[doublethefish]

I was thinking the same thing.

On 8 Oct 2015, at 19:51, Dane Bettis [email protected] wrote:

I am also interested in this feature: a pub/sub system, even if not really real-time, would enable a robust VFX asset usage pipeline.

I'd be glad to further discuss options and possibly provide some real-world testing of ipfs in the VFX content development production pipeline.

—
Reply to this email directly or view it on GitHub.

[1l0]

👍

[jbenet]

Indeed. We're a bit overloaded with things atm. if you want to speed it up, coordinate here ipfs/notes#64

[magik6k]

There has been some related work done recently:

• There is now an experimental feature called namesys-pubsub which allows nodes to listen for ipns updates - https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipns-pubsub
• ipfs name resolve --stream allows to get updates in real time, but currently it returns as soon as ipfs name resolve would normally return, so it isn't really suitable for notifications

What still needs to be done is a flag for resolve --stream (and SearchValue in the namesys/routing layers) which would tell it to keep looking for new records even after a 'good' record is discovered.

As a work-around, when publisher is running with namesys-pubsub enabled, it's possible to listen for updates, but it's a bit tricky to construct the topic:

ipfs pubsub sub /record/$((printf '/ipns/%s'; printf 'QmPeerID' | bases base58 raw) | bases bin b64u)

(You need to build the bases program with this PR applied)

Messages on this topic will be libp2p-record protobufs - https://github.com/libp2p/go-libp2p-record/blob/master/pb/record.proto which you'll need to validate manually

[Stebalien]

Latest work is here: #4462

[zicklag]

I'd like to bring up that I have a use-case for this. I'm trying to workaround right now with something similar to what @magik6k posted above, but I'm a little confused by it and it doesn't seem to be a properly formatted bash command.

Is there any more "modern" way to convert to the IPNS key to the pubsub topic? Does the IPFS command have any tools to do the base-conversion in it now? I see there's a multibase encoder, but I can't figure out if that helps.