gRPC Streaming From C# Resulting in GoAway error from server

[psbang]

In the solution attached below, we're trying to make gRPC requests to the Google GenerativeAI API, which is supposed to return a streaming response. However, this is inconsistently successful, as a significant fraction of the streaming responses are cut short with a GoAway error (seen in the txt output file also attached below). In the output, you can see the GoAway comes with output frame 13, but we have noticed that the frame number at which this happens, if at all, is also not consistent.

To reproduce our issue, install open the console app included in the attached ZIP file. You will need an API key from https://ai.google.dev/tutorials/setup and fill it in where the // TODO is. Then, run the console app. The app will make 100 calls to the generative AI API with a pre-filled prompt, and at the end you will see how many calls failed (ranges from anywhere between 1 failure to up to 26). I create the proto file using the structures I need from the publicly available protobuf here.

The gRPC calls to this server are consistently successful from the Python client, but we haven't gotten it to succeed 100% of the time with C#. We believe that there might be a flag or header that we're missing to prevent the server from sending us the GoAway, but we don't know what it could be. Please advise.

GrpcStream.zip-20240111T225040Z-001.zip
SampleGrpcStreamOutput.txt

[erdalsivri]

I should mention that we tried passing a custom SocketsHttpHandler with various Keepalive values (especially KeepAlivePingPolicy.WithActiveRequests) but still couldn't get it to a reliable state. It is a fair point to say the server has a bug but GenerativeAI API team points us to the working Python client to prove the server is working fine. We did test the python client with a similar loop and it's never failed as consistently as the C# client. So we are thinking there might be some configuration mismatch between the client and the server or a misbehavior on the client side that cause server to reject the request after some time.

[JamesNK]

Hi

This is the same problem as #2361. You can read that issue for more details.

Adding this code to the start of the client app disables RTT pings fixes the problem:

AppContext.SetSwitch("System.Net.SocketsHttpHandler.Http2FlowControl.DisableDynamicWindowSizing", true);

[karelz]

@psbang are you using .NET 6.0 or would a fix in .NET 8.0 be sufficient for you?

[KirillKurdyukov]

@psbang are you using .NET 6.0 or would a fix in .NET 8.0 be sufficient for you?

Hello!
I also ran into this problem when using an L4 load balancer.
when can I expect a fix in stable versions?

Error:

Transport exception: Status(StatusCode="Unavailable", Detail="Error reading next message. HttpIOException: The response ended prematurely while waiting for the next frame from the server. (ResponseEnded)", DebugException="System.Net.Http.HttpIOException: The response ended prematurely while waiting for the next frame from the server. (ResponseEnded)")