Skip to content

net/http: No error with http2 client on empty reply body and content-length != 0 #46071

Closed
@MichaelEischer

Description

@MichaelEischer

What version of Go are you using (go version)?

$ go version
go version go1.16.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/michael/Library/Caches/go-build"
GOENV="/Users/michael/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/michael/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/michael/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/michael/go1.16.4"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/michael/go1.16.4/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/michael/Projekte/restic/test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/vs/zdx8nt1n4xlc2fgvtw589_100000gn/T/go-build1480369913=/tmp/go-build -gno-record-gcc-switches -fno-common"
uname -v: Darwin Kernel Version 19.6.0: Mon Apr 12 20:57:45 PDT 2021; root:xnu-6153.141.28.1~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.15.7
BuildVersion:	19H1030
lldb --version: lldb-1200.0.44.2
Apple Swift version 5.3.2 (swiftlang-1200.0.45 clang-1200.0.32.28)

What did you do?

Issue a GET request to a HTTP/2 server which then replies with the Content-Length header but afterwards fails to send data to the client for some reason. That is the reply has an empty body. See the Go playground link for a minimal example.

https://play.golang.org/p/jbhV10xhWKI

What did you expect to see?

Now, the client should receive an "Unexpected EOF" error when reading the reply:

2021/05/09 20:19:26 unexpected EOF

What did you see instead?

Header: map[Content-Length:[42] Date:[Sun, 09 May 2021 18:17:42 GMT]]
Body: {%!q(*bytes.Reader=&{[] 0 -1})}
Content-Length: 0
result ""

Note that res.ContentLength is set to zero despite the Content-Length header in the reply. This lets clients erroneously assume that they successfully received a reply to their request.

Executing the example on the Go playground currently deadlocks.

Go playground error

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [select]:
net/http.(*Transport).getConn(0xc0000e83c0, 0xc0000bc2c0, 0x0, 0xc0000c80c0, 0x5, 0xc0000b6c50, 0xf, 0x0, 0x0, 0x0, ...)
	/usr/local/go-faketime/src/net/http/transport.go:1368 +0x589
net/http.(*Transport).roundTrip(0xc0000e83c0, 0xc00012a000, 0x30, 0x6f8940, 0x7fc92cc09500)
	/usr/local/go-faketime/src/net/http/transport.go:579 +0x7eb
net/http.(*Transport).RoundTrip(0xc0000e83c0, 0xc00012a000, 0xc0000e83c0, 0x0, 0x0)
	/usr/local/go-faketime/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc00012a000, 0x76c920, 0xc0000e83c0, 0x0, 0x0, 0x0, 0xc0000b8058, 0x417ed7, 0x1, 0x0)
	/usr/local/go-faketime/src/net/http/client.go:251 +0x454
net/http.(*Client).send(0xc0000a0d20, 0xc00012a000, 0x0, 0x0, 0x0, 0xc0000b8058, 0x0, 0x1, 0xc00012a000)
	/usr/local/go-faketime/src/net/http/client.go:175 +0xff
net/http.(*Client).do(0xc0000a0d20, 0xc00012a000, 0x0, 0x0, 0x0)
	/usr/local/go-faketime/src/net/http/client.go:717 +0x45f
net/http.(*Client).Do(...)
	/usr/local/go-faketime/src/net/http/client.go:585
net/http.(*Client).Get(0xc0000a0d20, 0xc0000c80c0, 0x17, 0x7fc953853888, 0x4, 0x406885)
	/usr/local/go-faketime/src/net/http/client.go:474 +0xbe
main.main()
	/tmp/sandbox703608843/prog.go:24 +0x136

goroutine 18 [IO wait]:
internal/poll.runtime_pollWait(0x7fc95385beb8, 0x72, 0x0)
	/usr/local/go-faketime/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0000fa018, 0x72, 0x0, 0x0, 0x70e0ec)
	/usr/local/go-faketime/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go-faketime/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc0000fa000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go-faketime/src/internal/poll/fd_unix.go:401 +0x212
net.(*netFD).accept(0xc0000fa000, 0x50, 0x6e0440, 0xc000096df0)
	/usr/local/go-faketime/src/net/fd_unix.go:172 +0x45
net.(*TCPListener).accept(0xc0000ae0a8, 0x50, 0xc00018c000, 0x7fc953855c20)
	/usr/local/go-faketime/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc0000ae0a8, 0x6e0440, 0xc000080000, 0x0, 0xc000096e20)
	/usr/local/go-faketime/src/net/tcpsock.go:261 +0x65
crypto/tls.(*listener).Accept(0xc0000ae5b8, 0x6ebd00, 0xc0001820c0, 0x6b9aa0, 0x8ec100)
	/usr/local/go-faketime/src/crypto/tls/tls.go:67 +0x37
net/http.(*Server).Serve(0xc000110000, 0x770ad0, 0xc0000ae5b8, 0x0, 0x0)
	/usr/local/go-faketime/src/net/http/server.go:2981 +0x285
net/http/httptest.(*Server).goServe.func1(0xc0000fe000)
	/usr/local/go-faketime/src/net/http/httptest/server.go:308 +0x6e
created by net/http/httptest.(*Server).goServe
	/usr/local/go-faketime/src/net/http/httptest/server.go:306 +0x5c

goroutine 19 [IO wait]:
internal/poll.runtime_pollWait(0x7fc95385bdd0, 0x77, 0x0)
	/usr/local/go-faketime/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0000fa218, 0x77, 0xc0000b6000, 0x1, 0x0)
	/usr/local/go-faketime/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitWrite(...)
	/usr/local/go-faketime/src/internal/poll/fd_poll_runtime.go:96
internal/poll.(*FD).WaitWrite(...)
	/usr/local/go-faketime/src/internal/poll/fd_unix.go:528
net.(*netFD).connect(0xc0000fa200, 0x771290, 0xc0000b6020, 0x0, 0x0, 0x76cca0, 0xc0000fc2e0, 0x0, 0x0, 0x0, ...)
	/usr/local/go-faketime/src/net/fd_unix.go:141 +0x27b
net.(*netFD).dial(0xc0000fa200, 0x771290, 0xc0000b6020, 0x772058, 0x0, 0x772058, 0xc0000a1380, 0x0, 0x7fc953853778, 0x10)
	/usr/local/go-faketime/src/net/sock_posix.go:149 +0x10b
net.socket(0x771290, 0xc0000b6020, 0x70d741, 0x3, 0x2, 0x1, 0x0, 0x0, 0x772058, 0x0, ...)
	/usr/local/go-faketime/src/net/sock_posix.go:70 +0x1c5
net.internetSocket(0x771290, 0xc0000b6020, 0x70d741, 0x3, 0x772058, 0x0, 0x772058, 0xc0000a1380, 0x1, 0x0, ...)
	/usr/local/go-faketime/src/net/ipsock_posix.go:141 +0x145
net.(*sysDialer).doDialTCP(0xc0000fa180, 0x771290, 0xc0000b6020, 0x0, 0xc0000a1380, 0xc000093601, 0x6e6b00, 0x1)
	/usr/local/go-faketime/src/net/tcpsock_posix.go:65 +0xc5
net.(*sysDialer).dialTCP(0xc0000fa180, 0x771290, 0xc0000b6020, 0x0, 0xc0000a1380, 0xc0000b6c50, 0xc000093768, 0x40d77b)
	/usr/local/go-faketime/src/net/tcpsock_posix.go:61 +0xd7
net.(*sysDialer).dialSingle(0xc0000fa180, 0x771290, 0xc0000b6020, 0x76e9f8, 0xc0000a1380, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go-faketime/src/net/dial.go:580 +0x5e8
net.(*sysDialer).dialSerial(0xc0000fa180, 0x771290, 0xc0000b6020, 0xc00009eea0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go-faketime/src/net/dial.go:548 +0x15e
net.(*Dialer).DialContext(0x8f8fa0, 0x771290, 0xc0000b6020, 0x70d741, 0x3, 0xc0000b6c50, 0xf, 0x0, 0x0, 0x0, ...)
	/usr/local/go-faketime/src/net/dial.go:425 +0x6e5
net/http.(*Transport).dial(0xc0000e83c0, 0x771290, 0xc0000b6020, 0x70d741, 0x3, 0xc0000b6c50, 0xf, 0x0, 0xc000093ad0, 0x58d33d, ...)
	/usr/local/go-faketime/src/net/http/transport.go:1171 +0x16f
net/http.(*Transport).dialConn(0xc0000e83c0, 0x771290, 0xc0000b6020, 0x0, 0xc0000c80c0, 0x5, 0xc0000b6c50, 0xf, 0x0, 0xc0000ca7e0, ...)
	/usr/local/go-faketime/src/net/http/transport.go:1600 +0x1b85
net/http.(*Transport).dialConnFor(0xc0000e83c0, 0xc0000dc2c0)
	/usr/local/go-faketime/src/net/http/transport.go:1442 +0xc6
created by net/http.(*Transport).queueForDial
	/usr/local/go-faketime/src/net/http/transport.go:1411 +0x40f

Disabling the use of HTTP2 (comment out the ts.EnableHTTP2 = true line) or sending at least one byte (see w.Write([]byte("a"))) leads to the expected error.

Activity

added
NeedsFixThe path to resolution is known, but the work has not been done.
on May 10, 2021
modified the milestones: Go1.18, Go1.17 on May 10, 2021
heschi

heschi commented on May 10, 2021

@heschi
Contributor
modified the milestones: Go1.17, Backlog on May 10, 2021
heschi

heschi commented on May 10, 2021

@heschi
Contributor
networkimprov

networkimprov commented on May 16, 2021

@networkimprov
fraenkel

fraenkel commented on May 16, 2021

@fraenkel
Contributor

https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.6

A response that is defined to have no payload,
can have a non-zero content-length header field, even
   though no content is included in DATA frames.
bradfitz

bradfitz commented on May 17, 2021

@bradfitz
Contributor

Looks like a bug. And I'm surprised, as I thought it handled this.

@fraenkel, that's about HEAD responses primarily, no? Doesn't apply to this case where it's just a GET?

fraenkel

fraenkel commented on May 17, 2021

@fraenkel
Contributor

Unfortunately it's under malformed requests and responses, not tied to a method.

MichaelEischer

MichaelEischer commented on May 17, 2021

@MichaelEischer
Author

@fraenkel The quote is missing the reference to RFC7230:

A response that is defined to have no payload, as described in [RFC7230],
Section 3.3.2, can have a non-zero content-length header field, even
though no content is included in DATA frames.

In RFC 7230 section 3.3 the last paragraph seems to describe all cases when a response has a message body, see https://datatracker.ietf.org/doc/html/rfc7230#section-3.3 . And a 200 OK reply to a GET request as in the playground example in the issue description, is allowed to have a payload. So I don't think that the response that is defined to have no payload exception applies here?

fraenkel

fraenkel commented on May 17, 2021

@fraenkel
Contributor

Thanks for catching that. It's would seem it's not just method bit also status code.

bradfitz

bradfitz commented on May 18, 2021

@bradfitz
Contributor

Yeah, when I said "HEAD primarily" I was being lazy and referring to:

// bodyAllowedForStatus reports whether a given response status code
// permits a body. See RFC 7230, section 3.3.
func bodyAllowedForStatus(status int) bool {
        switch {
        case status >= 100 && status <= 199:
                return false
        case status == 204:
                return false
        case status == 304:
                return false
        }
        return true
}

Which we use in various places in both net/http and http2 already, e.g.:

                if clen == "" && rws.handlerDone && bodyAllowedForStatus(rws.status) && (len(p) > 0 || !isHeadResp) {
                        clen = strconv.Itoa(len(p))
                }
        if w.req.Method != "HEAD" && w.contentLength != -1 && w.bodyAllowed() && w.contentLength != w.written {
                // Did not write enough. Avoid getting out of sync.                                                        
                return false
        }

I see net/http has checks for writing too much:

	if w.contentLength != -1 && w.written > w.contentLength {
                return 0, ErrContentLength
        }

But I don't see anything in net/http.(*response).finishReuest about writing too little.

added a commit that references this issue on Oct 6, 2021
f72ff5d
gopherbot

gopherbot commented on Oct 6, 2021

@gopherbot
Contributor

Change https://golang.org/cl/354141 mentions this issue: http2: return unexpected eof on empty response with non-zero content length

added 2 commits that reference this issue on Oct 7, 2021
973a1b9
11b92cc
added a commit that references this issue on Jun 22, 2022
f78f23f
locked and limited conversation to collaborators on Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      Participants

      @bradfitz@networkimprov@fraenkel@gopherbot@MichaelEischer

      Issue actions

        net/http: No error with http2 client on empty reply body and content-length != 0 · Issue #46071 · golang/go