transport: Avoid buffer copies when reading Data frames #8657

arjan-bal · 2025-10-16T15:16:36Z

This change incorporates changes from golang/go#73560 to split reading HTTP/2 frame headers and payloads. If the frame is not a Data frame, it's read through the standard library framer as before. For Data frames, the payload is read directly into a buffer from the buffer pool to avoid copying it from the framer's buffer.

Testing

For 1 MB payloads, this results in ~4% improvement in throughput.

# test command
go run benchmark/benchmain/main.go -benchtime=60s -workloads=streaming \
   -compression=off -maxConcurrentCalls=120 -trace=off \
   -reqSizeBytes=1000000 -respSizeBytes=1000000 -networkMode=Local -resultFile="${RUN_NAME}"

# comparison
go run benchmark/benchresult/main.go streaming-before streaming-after  
               Title       Before        After Percentage
            TotalOps        87536        91120     4.09%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op   4074102.92   4070489.30    -0.09%
           Allocs/op        83.60        76.55    -8.37%
             ReqT/op 11671466666.67 12149333333.33     4.09%
            RespT/op 11671466666.67 12149333333.33     4.09%
            50th-Lat  78.209875ms  75.159943ms    -3.90%
            90th-Lat 117.764228ms   107.8697ms    -8.40%
            99th-Lat 146.935704ms 139.069685ms    -5.35%
             Avg-Lat  82.310691ms  79.073282ms    -3.93%
           GoVersion     go1.24.7     go1.24.7
         GrpcVersion   1.77.0-dev   1.77.0-dev

For smaller payloads, the difference in minor.

go run benchmark/benchmain/main.go -benchtime=60s -workloads=streaming \
   -compression=off -maxConcurrentCalls=120 -trace=off \
   -reqSizeBytes=100 -respSizeBytes=100 -networkMode=Local -resultFile="${RUN_NAME}"

go run benchmark/benchresult/main.go streaming-before streaming-after 
               Title       Before        After Percentage
            TotalOps     21490752     21477822    -0.06%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op      1902.92      1902.94     0.00%
           Allocs/op        29.21        29.21     0.00%
             ReqT/op 286543360.00 286370960.00    -0.06%
            RespT/op 286543360.00 286370960.00    -0.06%
            50th-Lat    352.505µs    352.247µs    -0.07%
            90th-Lat    433.446µs    434.907µs     0.34%
            99th-Lat    536.445µs    539.759µs     0.62%
             Avg-Lat    333.403µs    333.457µs     0.02%
           GoVersion     go1.24.7     go1.24.7
         GrpcVersion   1.77.0-dev   1.77.0-dev

RELEASE NOTES:

transport: Avoid a buffer copy when reading data.

codecov · 2025-10-16T15:20:46Z

Codecov Report

❌ Patch coverage is 92.52336% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.96%. Comparing base (b8a0fc9) to head (00586ba).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/transport/http_util.go	93.25%	4 Missing and 2 partials ⚠️
internal/transport/http2_client.go	87.50%	0 Missing and 1 partial ⚠️
internal/transport/http2_server.go	90.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8657      +/-   ##
==========================================
+ Coverage   81.92%   81.96%   +0.03%     
==========================================
  Files         416      417       +1     
  Lines       40789    40871      +82     
==========================================
+ Hits        33418    33498      +80     
- Misses       6000     6007       +7     
+ Partials     1371     1366       -5

Files with missing lines	Coverage Δ
internal/transport/http2_client.go	`92.12% <87.50%> (-0.15%)`	⬇️
internal/transport/http2_server.go	`90.83% <90.00%> (-0.30%)`	⬇️
internal/transport/http_util.go	`94.10% <93.25%> (-0.35%)`	⬇️

... and 26 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

easwars · 2025-10-16T19:44:40Z

internal/transport/http_util.go

 	offset    int
 	batchSize int
-	conn      net.Conn
+	conn      io.ReadWriter


Why does this have to a io.ReadWriter? Can it simply be an io.Writer?

Yes, io.Writer is more appropriate here. I wanted to replace the net.Conn with a simpler interface to fake it in tests. I used io.ReadWriter in the constructor of the grpc framer (newFramer) and propagated the same type to the called functions. Changed now.

easwars · 2025-10-16T20:00:42Z

internal/transport/http_util.go

+		// DATA frame is received whose stream identifier
+		// field is 0x0, the recipient MUST respond with a
+		// connection error (Section 5.4.1) of type
+		// PROTOCOL_ERROR.


How are we returning a PROTOCOL_ERROR in this case?

The code in our http2 client and server that calls readFrame() does a type assertion for http2.StreamError and only in that case closes the stream with a protocol error. But it checks to find the stream to close, and if the stream ID here is 0, it won't find a stream, and therefore will end up closing the transport.

It should have been returning an http2.ConnectionError with code http2.ErrCodeProtocol, similar to the std lib framer. I fixed it to do the same now and updated the test.

The code in our http2 client and server that calls readFrame() does a type assertion for http2.StreamError and only in that case closes the stream with a protocol error. But

The std lib framer returns 2 kinds of errors:

Connection Errors: These should result in the entire http2 connection being closed. These are the majority of errors.

Stream Errors: These should result in only a single stream being closed, keeping the connection alive. Presently stream errors are only returned during parsing of window update and header frames.

Here, we need to return a connection error because stream ID 0 is special in HTTP/2. It is reserved for frames that apply to the entire connection rather than an individual request/response stream. Stream 0 is the "control channel" for the whole HTTP/2 connection. We can't close stream 0 while keeping the connection alive.

easwars · 2025-10-16T20:15:01Z

internal/transport/http_util.go

+	}
+	if fh.Flags.Has(http2.FlagDataPadded) {
+		if fh.Length == 0 {
+			return io.ErrUnexpectedEOF


Does it make sense to wrap io.ErrUnexpectedEOF with some additional context like "padded bit is set, but frame length is 0"?

I kept it as is for consistency with the std lib framer, which returns an unwrapped error here.

easwars · 2025-10-16T20:33:21Z

internal/transport/http_util_test.go

+			// Write the frame using the provided function.
+			writeErr := tt.w(fr)
+			if writeErr != nil {
+				t.Fatalf("tt.w() returned unexpected error: %v", writeErr)


Nit: tt.w() is not very readable when it shows up in the log of a failing test. Maybe something more descriptive here please.

Renamed to writeFrames.

arjan-bal

I realised that the changes in the std lib framer are not imported in g3 os this PR can't be merged in gRPC yet. I've added a Blocked label to indicate this.

arjan-bal · 2025-10-21T07:57:45Z

internal/transport/http_util_test.go

+			// Write the frame using the provided function.
+			writeErr := tt.w(fr)
+			if writeErr != nil {
+				t.Fatalf("tt.w() returned unexpected error: %v", writeErr)


Renamed to writeFrames.

arjan-bal · 2025-10-21T08:15:42Z

internal/transport/http_util.go

+		// DATA frame is received whose stream identifier
+		// field is 0x0, the recipient MUST respond with a
+		// connection error (Section 5.4.1) of type
+		// PROTOCOL_ERROR.


It should have been returning an http2.ConnectionError with code http2.ErrCodeProtocol, similar to the std lib framer. I fixed it to do the same now and updated the test.

The code in our http2 client and server that calls readFrame() does a type assertion for http2.StreamError and only in that case closes the stream with a protocol error. But

The std lib framer returns 2 kinds of errors:

Connection Errors: These should result in the entire http2 connection being closed. These are the majority of errors.

Stream Errors: These should result in only a single stream being closed, keeping the connection alive. Presently stream errors are only returned during parsing of window update and header frames.

Here, we need to return a connection error because stream ID 0 is special in HTTP/2. It is reserved for frames that apply to the entire connection rather than an individual request/response stream. Stream 0 is the "control channel" for the whole HTTP/2 connection. We can't close stream 0 while keeping the connection alive.

arjan-bal · 2025-10-21T08:20:16Z

internal/transport/http_util.go

+	}
+	if fh.Flags.Has(http2.FlagDataPadded) {
+		if fh.Length == 0 {
+			return io.ErrUnexpectedEOF


I kept it as is for consistency with the std lib framer, which returns an unwrapped error here.

arjan-bal · 2025-10-21T08:38:00Z

internal/transport/http_util.go

 	offset    int
 	batchSize int
-	conn      net.Conn
+	conn      io.ReadWriter


Yes, io.Writer is more appropriate here. I wanted to replace the net.Conn with a simpler interface to fake it in tests. I used io.ReadWriter in the constructor of the grpc framer (newFramer) and propagated the same type to the called functions. Changed now.

arjan-bal added this to the 1.77 Release milestone Oct 16, 2025

arjan-bal added Type: Performance Performance improvements (CPU, network, memory, etc) Area: Transport Includes HTTP/2 client/server and HTTP server handler transports and advanced transport features. labels Oct 16, 2025

arjan-bal force-pushed the copyless-data-frame-read branch from b5f777e to ae8c8df Compare October 16, 2025 16:54

arjan-bal self-assigned this Oct 16, 2025

arjan-bal force-pushed the copyless-data-frame-read branch 2 times, most recently from efe661b to 778af0a Compare October 16, 2025 18:53

arjan-bal added 4 commits October 17, 2025 00:40

changes no tests

9875111

tests

a6f1b72

cleanup

3f98471

eliminate extra heap alloc

5fa01cd

arjan-bal force-pushed the copyless-data-frame-read branch from 778af0a to 45e856f Compare October 16, 2025 19:12

bump x/net to golang.org/x/net@63d1a51

5ec4a4e

arjan-bal force-pushed the copyless-data-frame-read branch from 45e856f to 5ec4a4e Compare October 16, 2025 19:15

arjan-bal removed their assignment Oct 16, 2025

arjan-bal requested review from dfawley and easwars and removed request for easwars October 16, 2025 19:21

arjan-bal assigned easwars and dfawley Oct 16, 2025

easwars reviewed Oct 16, 2025

View reviewed changes

easwars assigned arjan-bal and unassigned easwars Oct 16, 2025

arjan-bal added 2 commits October 21, 2025 14:14

fix error codes and interface types

6820fee

Improve testing of error cases

00586ba

arjan-bal commented Oct 21, 2025

View reviewed changes

arjan-bal added the Status: Blocked label Oct 21, 2025

arjan-bal assigned easwars Oct 21, 2025

arjan-bal removed their assignment Oct 21, 2025

transport: Avoid buffer copies when reading Data frames #8657

Are you sure you want to change the base?

transport: Avoid buffer copies when reading Data frames #8657

Conversation

arjan-bal commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Uh oh!

codecov bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arjan-bal left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

arjan-bal commented Oct 16, 2025 •

edited

Loading

codecov bot commented Oct 16, 2025 •

edited

Loading