net/http: transferWriter.doBodyCopy() calls io.Copy() resulting in many buffer allocations

[mmatczuk]

What version of Go are you using (go version)?

$ go version go1.19.4 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE="on"
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/michal/Library/Caches/go-build"
GOENV="/Users/michal/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/michal/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/michal/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/michal/sdk/go1.19.4"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/michal/sdk/go1.19.4/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/7q/nyynjpwj5p19npby48ykjpx00000gn/T/go-build3980036370=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have HTTP proxy server proxying plain HTTP 1.1 requests based on google/martian. When I profile the memory allocations I see that transferWriter.doBodyCopy() calls io.Copy() resulting in many buffer allocations. The profile graph is attached below.

I think doBodyCopy() could use copyBufPool we have in response.ReadFrom() to avoid the allocations.

[gopherbot]

Change https://go.dev/cl/456435 mentions this issue: net/http: use copyBufPool in transferWriter.doBodyCopy()

[neild]

I wonder if io.Copy should use a pool to allocate buffers.

[gopherbot]

Change https://go.dev/cl/456555 mentions this issue: io: allocate copy buffers from a pool

[gopherbot]

Change https://go.dev/cl/467095 mentions this issue: Revert "io: allocate copy buffers from a pool"