net/http: customize limit on number of 1xx responses

[Acconut]

Proposal Details

In HTTP, a server can send multiple responses for one requests: zero or more informational responses (1xx) and one final response (2xx, 3xx, 4xx, 5xx). Go's HTTP client is capable of receiving those informational responses and exposes them to users via net/http/httptrace.ClientTrace.Got1xxResponse. However, the client has a default limit of reading only up to 5 responses. Any additional 1xx response will trigger a net/http: too many 1xx informational responses error.

go/src/net/http/transport.go

Lines 2328 to 2329 in 8db1310

	num1xx := 0 // number of informational 1xx headers received
	const max1xxResponses = 5 // arbitrary bound on number of informational responses

go/src/net/http/transport.go

Lines 2354 to 2357 in 8db1310

	num1xx++
	if num1xx > max1xxResponses {
	return nil, errors.New("net/http: too many 1xx informational responses")
	}

The code comments and the original commit (d88b137) mention that the limit of 5 responses is arbitrary. If the limit is reached, the entire request is stopped and the client cannot receive the final response (2xx etc) anymore. This is problematic for applications, where the server repeatedly sends informational responses. 5 is a sensible default value for nearly all applications, but it would be helpful if this limit could be customized to allow more or even an unlimited amount of responses.

One option for implementing this, would be to add another field to the net/http.Client struct. Setting it to a zero value keeps the current limit of 5 responses, while any other non-zero value sets the limit accordingly.

Background

In the HTTP working group of the IETF we are discussing a draft on resumable uploads. We are considering including a feature where the server can repeatedly send 1xx responses to inform the client about the upload progress. In these scenarios, the client sends data in the request body and repeatedly receives progress information in the 1xx responses. This progress information can be used to release data that is buffered on the client-side.

Example

Below you can find a brief program reproducing this behavior. The client sends a request to a server which responds with 10 1xx responses:

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"net/http/httptest"
	"net/http/httptrace"
	"net/textproto"
	"time"
)

func main() {
	ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		for i := 0; i < 10; i++ {
			w.Header().Set("X-Progress", fmt.Sprintf("%d%%", i*10))
			w.WriteHeader(150)
			<-time.After(100 * time.Millisecond)
		}

		w.WriteHeader(200)
	}))
	defer ts.Close()

	ctx := httptrace.WithClientTrace(context.Background(), &httptrace.ClientTrace{
		Got1xxResponse: func(code int, header textproto.MIMEHeader) error {
			fmt.Println("Progress:", header.Get("X-Progress"))

			return nil
		},
	})

	req, err := http.NewRequestWithContext(ctx, "GET", ts.URL, nil)
	if err != nil {
		log.Fatal(err)
	}

	client := ts.Client()
	res, err := client.Do(req)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(res.Status)
}

The client receives the first 5 1xx responses, but then errors out. The final response is not received by the client.

$ go run clients/go/test/example.go 
Progress: 0%
Progress: 10%
Progress: 20%
Progress: 30%
Progress: 40%
2024/01/09 10:05:56 Get "http://127.0.0.1:52073": net/http: HTTP/1.x transport connection broken: net/http: too many 1xx informational responses
exit status 1

If the limit could be raised, the client could receive all informational and the final response without an error.

[seankhliao]

cc @neild

[royfielding]

This is a bug in the Go client. HTTP is expected to have many interim responses, depending on the nature of the request. There is no reasonable limit on number of valid interim responses. Setting a time limit overall might make sense for capacity reasons, but this default limit is nuts.

[joakime]

Wow! @royfielding commenting on an HTTP user-agent bug on github. Hard to get more authoritative about the HTTP spec.

[neild]

A configurable limit is too fiddly; we shouldn't require users to twiddle a knob to get reasonable behavior.

We generally consider hostile servers to be less of a concern than hostile clients, but no limit might be a bit much; a server shouldn't be able to send an arbitrary amount of data in response to a request without the client's knowledge.

Perhaps a good compromise might be something along the lines of: No more than N 1xx responses, resetting the counter after a time interval and after every response byte read. (So interleaving an arbitrary number of 1xx responses with response bytes is acceptable, as is sending 1xx responses with no data but below some rate.)

Whatever we do will need to be synchronized between the HTTP/1 and HTTP/2 paths, which both implement the same no-more-than-5 logic at this time.

[Acconut]

Thanks for the comments!

we shouldn't require users to twiddle a knob to get reasonable behavior.

This would be an ideal scenario, but I am not not sure if it's possible in this case.

No more than N 1xx responses, resetting the counter after a time interval

That would be an option. For example, with this approach we could allow 5 responses in 5 seconds?

after every response byte read. (So interleaving an arbitrary number of 1xx responses with response bytes is acceptable, as is sending 1xx responses with no data but below some rate.)

I am not sure this is possible. As far as I know interim responses and the final response cannot be interleaved. When the client starts reading the final response body, no more interim responses will be sent anyway. This should be the case for HTTP/1.1 and also HTTP/2 from reading https://httpwg.org/specs/rfc9113.html#HttpFraming.

a server shouldn't be able to send an arbitrary amount of data in response to a request without the client's knowledge.

Would it be possible to reuse net/http.Transport.MaxResponseHeaderBytes for this purpose? It could be the limit on the size of all response headers combined (i.e. interim response headers plus the final response header). The default value would allow some interim responses to be sent while protecting against too many. While in use cases like mine, we could increase the limit to allow more interim responses.

Right now MaxResponseHeaderBytes is applied on a per-response basis and is reset for every interim or final response:

go/src/net/http/transport.go

Line 2358 in 704401f

pc.readLimit = pc.maxHeaderResponseSize() // reset the limit

Whatever we do will need to be synchronized between the HTTP/1 and HTTP/2 paths, which both implement the same no-more-than-5 logic at this time.

Yss, I agree on this.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

@neild do you still believe that #65035 (comment) is the right answer here?

Do we know what other HTTP client libraries do, especially ones that try to grapple with misbehaving servers, slowloris attacks, and so on?

[neild]

I spent a few minutes attempting to figure out what other clients do. I looked at Python's urllib3 and requests, Rust's hyper and reqwest, and libcurl. I have not managed to figure out from the documentation of any of them how you read 1xx responses, and I haven't tried delving into the code to see if there are any limits.

I'm still not enthused by a configurable knob for number of 1xx responses; you shouldn't need to twiddle settings to get good behavior. As @Acconut points out, 1xx responses can't be interleaved with data (I'd thought this was possible for HTTP/2, but no), so my suggestion above doesn't work, but we could change our rule to no more than N 1xx responses per second.

Another idea might be to permit unlimited 1xx responses, but only if the user is consuming them. Then if the user subscribes to 1xx responses, it's on them to figure out how many are too many.

Right now, the API for consuming 1xx responses is very clunky--you use net/http/httptrace and install a client trace handler for Got1xxResponse events. I've been thinking that we should have a more convenient API; perhaps we should add one (a new http.Request.On1xxResponse field containing a callback, perhaps?) and disable the 1xx limit for anyone using it, rather than changing the httptrace semantics. That's probably a separate proposal, though.

In the short term, I don't see a problem with changing the no-more-than-5 limit to no-more-than-5-per-second. (Or some number >5, it was chosen arbitrarily so far as I can tell.)