Additional services

[Kagami]

Hi.

I'd like to introduce several new node types compatible with the Bitmessage protocol: NODE_MOBILE and NODE_GATEWAY. They are almost identical to the nodes described here: https://bitmessage.org/wiki/Mobile_Protocol_specification

But several PyBitmessage quirks stop me:

• PyBitmessage doesn't check services bitfield at all. This means that if I advertise node which uses non-TCP transport with services = 4 (BE value), PyBitmessage will constantly try to connect to it despite the fact the node hasn't NODE_NETWORK bit.
• I can't advertise NODE_NETWORK and NODE_GATEWAY node as two different addr entries with the same IP because knownNodes structure is a host-ip/host-details map. Oops, I missed the fact knownNodes is a host+port/host-details map now.
• Even if I advertise NODE_GATEWAY as a separate addr entry, PyBitmessage won't be able to connect to it anyway so NODE_GATEWAY addresses will propogate badly through the network because of the node alive checkings: https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L666
• PyBitmessage currently doesn't take into account services field when it packs the addr message: https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L672, https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L275

So far I know one rather awkward solution: advertise NODE_NETWORK and NODE_GATEWAY fields (i.e. services = 5) as single addr entry and use hardcoded port for non-TCP transports (e.g. 18444) but this is rather tricky.

[Kagami]

Sorry, I've updated the issue with several details.

[Kagami]

Also, node services data isn't saved to the disk. It was saved before inside the knownnodes[1] table records but not anymore with knownnodes.dat.

  CREATE TABLE knownnodes (
    timelastseen int,
    stream int,
    services blob,
    host blob,
    port blob,
    UNIQUE(host, stream, port) ON CONFLICT REPLACE
)

[Kagami]

So, I have two variants how it can be fixed:

• PyBitmessage should store net_addr even if it hasn't NODE_NETWORK bit. And it shouldn't connect to the nodes without NODE_NETWORK bit.
• PyBitmessage shouldn't store net_addrs without NODE_NETWORK bit.

Each variant has it's own disadvantages:

• In the second case, we need to hardcode NODE_GATEWAY port because there is no other place where it can be stored (except the higher bits of the services bitfield itself but it's too hackish).
• But in the first case it's very easy to spam knownnodes because common node cannot verify availability of the node without NODE_NETWORK bit.

So this is the plan how the second variant can be implemented:

• PyBitmessage should store services field of the net_addr (it would require to change knownnodes format, but it's not too hard)
• PyBitmessage should also properly advertise services field loaded from knownnodes.dat
• PyBitmessage should discard net_addrs without NODE_NETWORK bit

@Atheros1, do you have any objections/suggestions? I think I can make PR for this in 1-2 months.

[Kagami]

Though, the sad part is: nodes with special services still won't be propogated fast because PyBitmessage won't give advantage to them when sending big addr message. So the alternative approaches are:

• Use own set of bootstrap nodes which would propogate only nodes with NODE_GATEWAY services (equivalent to forking the Bitmessage network)
• Broadcast objects encrypted with well-known keys which contain NODE_GATEWAY net_addrs (even more hackish)
• Somehow say to the PyBitmessage via version exchange that we are only interested in NODE_GATEWAY nodes

[Kagami]

Also, the interesting question: what if some node will lie about net_addr's services and constantly rebroadcast wrong values? How are we going to detect and ban such node?