net: File method of {TCP,UDP,IP,Unix}Conn and {TCP,Unix}Listener should leave the socket in nonblocking mode

[rothskeller]

What version of Go are you using (go version)?

1.10.1

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

linux/amd64 and darwin/amd64

What did you do?

listener, _ := net.Listen("tcp", ":8000")
listenerFH, _ := listener.(*net.TCPListener).File()
server.Serve(listener)
// later, in a different goroutine
server.Shutdown(context.Background())

What did you expect to see?

Server graceful shutdown.

What did you see instead?

Shutdown() returned immediately but Serve() never returned.

Diagnosis

The call to File() on the listener puts the socket in blocking mode. As a result, when the listener is closed, the Accept() called by Serve() does not return. Note that, if the call to File() is moved so that it occurs after the call to Accept(), the problem does not occur.

Response from iant@golang.org on golang-nuts, April 18, 2018:

Before about Go 1.9 or so the os package could not handle a non-blocking descriptor, so File had to change the connection to blocking mode in order to make it usable. Now, however, the os package does use nonblocking descriptors where possible. We could probably change the File method to use a nonblocking *os.File. Please do open an issue for that.

[rothskeller]

Probably irrelevant detail: the reason I am calling File() on the listener at all is so that I can add it to (exec.Command).ExtraFiles to pass it to a subprocess. Definitely no desire to change the socket behavior in so doing.

[fraenkel]

@ianlancetaylor The change is trivial but do we care about the behavior change? Unlike what was done with os.File, we cannot easily enhance an existing method.

[ianlancetaylor]

The package net methods return a *os.File, which will continue to work as expected if we make it nonblocking. I think it would be hard to write a program to detect this change. Let me know if you think otherwise.

[fraenkel]

The only nuance I am aware is that Deadline calls now work.

[fraenkel]

Shall this also apply to IPConn and TCPConn or just TCPListener? Seems weird to only have one behave differently, plus its easier to affect all 3.

[ianlancetaylor]

I assume it should affect all three.

[gopherbot]

Change https://golang.org/cl/108297 mentions this issue: net: calling File leaves the socket in nonblocking mode

[twmb]

I would guess that fd's were set to nonblocking to begin with (a144e3e) was for a roundabout way of deregistering from epoll (the logic is a bit to follow for the repo at that time). dup would've been problematic because a Close did not deregister from epoll, meaning a registered fd would never be unregistered.

netFD's now unconditionally fall into EPOLL_CTL_DEL on close, so this is no longer an issue. Removing SetNonblocking from netFD narrows #24481 down to the os package specifically. This might fix #24455 (but not the real underlying issue, if any).

[twmb]

Didn't realize this issue had closed (was looking into commits for too long with this page open) -- I'll move portions of that last comment to the relevant open issues.