proposal: net: please make it easy to get the destination address and port of a datagram received on an unconnected UDP socket

[jech]

Hi,

In a UDP server running on a host with multiple addresses, it is necessary to use the address on which a request was received in order to send the reply. This can be done either by binding a socket to each address of the local host, and monitoring the coming and going of addresses (which I find tedious and errorprone), or by using IP(V6)_PKTINFO in both recvmsg and sendmsg (which I prefer).

It looks like net.UDPConn does not expose the PKTINFO information. Please?

[ianlancetaylor]

What would the API for this look like?

Is there any reason this could not go into golang.org/x/net/{ipv4,ipv6}? That would definitely be preferable if possible.

[bradfitz]

The net package aims to be a portable interface across many operating systems. What Go API would you propose and would it work on Windows, Solaris, Mac, Plan 9, etc?

[jech]

would it work on Windows, Solaris, Mac, Plan 9, etc?

IPV6_PKTINFO is documented in RFC 3542, so it should in principle be supported by any system that has IPv6 sockets. IP_PKTINFO is a traditional BSD interface, and should therefore be supported by any implementation of sockets. (I've just checked that MSDN documents both.) I have no idea about Plan9 (which I'm told uses a Unixy interface instead of BSD sockets).

What Go API would you propose

I'm sure you have better taste and more experience than I do. One possibility would be to mimick the treatment of the oob parameter, and have a *net.IP parameter that's filled in by Recvmsg, and used by Sendmsg. That's simple and type-safe, but not extensible. Another possibility would be to have a new API that selects desired cmsgs. Then have the Recvmsg function return a map[string]interface{} with the received cmsgs, and Sendmsg take a similar map. (In which case the oob parameter could be omitted, they're redundant with cmsg.)

[mikioh]

Are ipv4.ControlMessage and ipv6.ControlMessage in https://godoc.org/golang.org/x/net/ipv4 and https://godoc.org/golang.org/x/net/ipv6 insufficient to manipulate ancillary information on each IP datagram?

[jech]

@mikioh, I haven't tried, but I think I'd prefer something that's better integrated with net.UDPConn. First, this is basic functionality, that is required by every single UDP server that wishes to support multihomed hosts, or even hosts with multiple addresses. Second, using the ipv6 and ipv4 addresses for dual-protocol sockets feels clumsy to me. (I'd love to be proved wrong — can you show me how to write a double-stack multihomed UDP server without excessive per-protocol special-casing?)

[mikioh]

First, ...

I don't understand. Those are IP-level ancillary information and not specific to UDP. It's application's responsibility to choose and manipulate appropriate options.

Second, ...

I'd recommend that you take a look at each operating system's online manual. A transport protocol connection endpoint that can accommodate both IPv4 and IPv6 addresses by using a wildcard address with AF_INET6 is just for convenience of IPv4-IPv6 transition and the behavior is different between platform implementations. One accepts both IPv4 and IPv6-level options, some accepts only IPv6 options, and the others prohibit from creating such transport protocol over dual IP stack connection endpoints. In general, it's better to have two transport protocol connection endpoints that have two different address families if you want deterministic operations.

For more questions, please have a look at https://github.com/golang/go/wiki/Questions.

[jech]

I don't understand. Those are IP-level ancillary information and not specific to UDP.

Easy access to the received packet's destination address is a required feature in order to write a UDP server. It's not usually required for writing a TCP server, where the transport layer sets the outgoing packet's source address with no help from the application.

It's application's responsibility to choose and manipulate appropriate options.

Yes. And the networking library should make the common cases easy for the application. (Which, admittedly, the sockets API fails to do.)

I'd recommend that you take a look at each operating system's online manual.

[...]

For more questions, please have a look at https://github.com/golang/go/wiki/Questions.

I cannot help but feel that I'm being patronised here.

[mikioh]

feel that I'm being patronised here.

I may have given a misleading impression. That's just an answer to your question; can you show me how to write .... Sorry to cause you trouble due to my poor language ability.

[tmm1]

Easy access to the received packet's destination address is a required feature in order to write a UDP server.

Accessing the source address of UDP datagram is already both possible and easy, using either net.UDPConn.ReadFrom or net.UDPConn.ReadFromUDP (both of which return the source address).

You can also get the source interface for advanced use-cases with ipv[46].ControlMessage from x/net:

udpConn, err := net.ListenUDP("udp4", addr)
pkConn := ipv4.NewPacketConn(udpConn)
pkConn.SetControlMessage(ipv4.FlagInterface, true)

_, cm, from, _ := pkConn.ReadFrom(nil)
log.Printf("got udp message from %v via interface %d", from, cm.IfIndex)

[jech]

Easy access to the received packet's destination address is a required feature in order to write a UDP server. Accessing the source address of UDP datagram is already both possible and easy, using either net.UDPConn.ReadFrom or net.UDPConn.ReadFromUDP (both of which return the source address).

I know that. The report is about the *destination* address of the received packet.

You can also get the source interface for advanced use-cases with ipv [46].ControlMessage from x/net:

I know that. However, since this is required functionality for even a basic UDP server, this is not an "advanced" used case -- it's required functionality. The (C-based) sockets API requires parsing a control message in order to get the destination address. This is a flaw of the sockets API, and Go should not be repeating that particular mistake. Please make it easy to get at the destination address and port of a datagram received on an unconnected UDP socket.

…

-- Juliusz

[bradfitz]

Please make it easy to get at the destination address and port of
a datagram received on an unconnected UDP socket.

I'm not a networking expert, so that's the first line in this bug report that I fully understood.

Could we retitle the bug to something like that?

Looking at

https://golang.org/pkg/net/#UDPConn.ReadFromUDP
and
https://golang.org/pkg/net/#UDPConn.ReadMsgUDP

... both are kinda gross. I'd hate to add another gross one. I wish we would've used some extensible type instead where we could add a field or method to some existing type rather than adding new UDPConn methods with yet more complicated signatures. But it seems that's what we'd need to do here.

Brainstorming:

// UDPMessage is the input & output for a UDPConn.ReadUDP call etc etc write real docs.
type UDPMessage struct {
	Data     []byte // if non-nil, data to read
	OOB      []byte // if non-nil, OOB to read
	DataRead int // populated by UDPConn.ReadURP
	OOBRead  int // populated by UDPConn.ReadURP
	Flags    int // populated by UDPConn.ReadURP
	SrcAddr  *UDPAddr // if non-nil, UDPConn.ReadURP sets this
	DstAddr  *UDPAddr // if non-nil, UDPConn.ReadURP sets this
}

// ReadUDP reads a UDP message and populates m.
func (c *UDPConn) ReadUDP(m *UDPMessage) error { /* ... */ }

[jech]

type UDPMessage struct {

Makes sense to me.

SrcAddr *UDPAddr
DstAddr *UDPAddr

Is there any reason why these are pointers rather than inline values?

func (c *UDPConn) ReadUDP(m UDPMessage) error { / ... */ }

The buffer pre-allocated by the caller, right?

What's the symmetric sending function, that picks the source address?

[bradfitz]

Is there any reason why these are pointers rather than inline values?

Yes, so the caller gets to decide whether getting either/both of those values is worth the cost to them. If they leave it nil, they can get more throughput if they don't care about, say, the destination address.

The buffer pre-allocated by the caller, right?

Yes, the []byte fields would be pre-allocated by the caller like the current two+ methods.

What's the symmetric sending function, that picks the source address?

https://golang.org/pkg/net/#DialUDP already lets you set the source address. Then you can reuse that conn to call https://golang.org/pkg/net/#UDPConn.WriteMsgUDP with different dst addresses.

[jech]

> What's the symmetric sending function, that picks the source address? https://golang.org/pkg/net/#DialUDP already lets you set the source address. Then you can reuse that conn to call https://golang.org/pkg/net/#UDPConn.WriteMsgUDP with different dst addresses.

That makes my socket into a connected socket, right? Connected UDP sockets have different semantics for a few things (notably receive filtering and error handling), so I'm not sure that would work. Even if DialUDP doesn't connect the socket (I'd have to look at the source, I don't think it's documented), then using it makes things much more messy. If you have a symmetric SendUDP function, then a UDP server looks something like (omitting error handling for the sake of brevity): sock := ListenUDP() for { r := ReadUDP(sock) data := process(sock.Data) s := UDPMessage{Data = data, SrcAddr = r.DstAddr, DstAddr = r.SrcAddr} SendUDP(sock, s) } On the other hand, if you need to call DialUDP, there's a jarring asymmetry between the receive and the send side.

…

-- Juliusz

[bradfitz]

That makes my socket into a connected socket, right?

I have no clue what a connected vs unconnected UDP socket even means. It's just bytes on a wire, right?

I'll bow out of this bug and defer to @mikioh who speaks networking.

[jech]

I have no clue what a connected vs unconnected UDP socket even means.

Two almost, but not quite, completely incompatible APIs to speak UDP using the sockets API. The connected API is easier to use, but the disconnected API is more general.

It's just bytes on a wire, right?

The distinction is at the API level, the on-the-wire protocol is the same.

[bradfitz]

You realize we're not built on C, right? We're not constrained by libc's API choices.

[jech]

You realize we're not built on C, right? We're not constrained by libc's API choices.

I'm reasonably sure that the connected/unconnected distinction is kernel-side state.

…

-- Juliusz

[bradfitz]

Again, I defer to @mikioh or others who know networking details in depth.

[quentinmit]

On Linux we also have the IP_RECVORIGDSTADDR control message. I think that should probably be preferred over IP_PKTINFO when available, though I haven't exhaustively tested them. The former is not affected by e.g. local iptables rules, while the latter is. You need the former for certain types of proxying applications.

[jech]

On Linux we also have the IP_RECVORIGDSTADDR control message. I think that should probably be preferred over IP_PKTINFO when available,

I believe that they are different things, and that IP_PKTINFO is what is needed here.

[mikioh]

The OP still does not address @ianlancetaylor's question, which comes from https://golang.org/doc/faq#x_in_std.

I personally think it's fine to have a UDP-specific package in golang.org/x/net repository like #8328 because we now see https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options. But for adding something to the net package of standard library, still not sure whether it's worth vendoring control message and option parsers from a few packages in golang.org/x/net repository.

[rsc]

It seems like there's a definite lack here as far as API, but also a lack of consensus about what the new API should look like. Should we experiment with something new in x/net and maybe elevate to standard library once people are happy?

[jech]

@mikioh @rsc Agreed, x/ seems like the right place to experiment with a better API.

[rsc]

Declining proposal; per discussion above, work in x/net for now and please submit a new proposal once the API is clear.