[API Proposal]: Support error codes for HttpRequestException

[searchmon]

Edited by @antonfirsov
Update 1: changes based on feedback - #76644 (comment)

Background and motivation

HttpRequestException is thrown under various conditions but it does not provide an error code to differentiate those conditions.
Currently, it provides 'string Message' property inherited from Exception type and it can contain various error messages:

• The HTTP response headers length exceeded the set limit
• Cannot write more bytes to the buffer
• The response ended prematurely
• Received chunk header length could not be parsed
• Received an invalid status
• Received an invalid header line
• Received an invalid header name
• The server returned an invalid or unrecognized response
• etc.

The above strings are defined in https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Http/src/Resources/Strings.resx

If we want to pragmatically handle exceptions based on the cause, it would be better dealing with error codes rather than dealing with string values. For example, when we create a dashboard for different exceptions encountered, aggregating by error codes would make more sense that doing by string values. And we could categorize some errors into a group for a special handling.

Many other exceptions like SocketException and Win32Exception have an error code field already.
Would it make sense to provide a new property for HttpRequestException to hold error codes?

API Proposal

public class HttpRequestException : Exception
{
    // -- New properties --
    public HttpRequestError HttpRequestError { get; }

    // -- New constructors --
    public HttpRequestException(HttpRequestError httpRequestError, string? message) {}
    public HttpRequestException(HttpRequestError httpRequestError, string? message, Exception? inner) {}
    public HttpRequestException(HttpRequestError httpRequestError, string? message, Exception? inner, HttpStatusCode? statusCode) {}
    // ALTERNATIVE:
    public HttpRequestException(HttpRequestError httpRequestError, string? message, Exception? inner = null, HttpStatusCode? statusCode = null) {}
}

// IOException subtype to throw from response read streams
public class HttpIOException : IOException  // ALTERNATIVE name: HttpResponseReadException
{
    public HttpRequestError HttpRequestError { get; }
    public HttpResponseReadException(HttpRequestError httpRequestError, string? message = null, Exception? innerException = null) {}
} 

// The EXISTING HttpProtocolException should be the subtype of this new exception type
public class HttpProtocolException : HttpResponseReadException // WAS: IOException
{
}

// Defines disjoint error categories with high granularity.
public enum HttpRequestError
{
    Undefined = 0,                          // Uncategorized/generic error -OR-
                                            // the underlying HttpMessageHandler does not implement HttpRequestError

    NameResolutionError,                    // DNS request failed
    ConnectionError,                        // Transport-level error during connection
    TransportError,                         // Transport-level error after connection
    SecureConnectionError,                  // SSL/TLS error
    HttpProtocolError,                      // HTTP 2.0/3.0 protocol error occurred
    UnsupportedExtendedConnect,             // Extended CONNECT for WebSockets over HTTP/2 is not supported.
                                            // (SETTINGS_ENABLE_CONNECT_PROTOCOL has not been sent).
    VersionNegotiationError,                // Cannot negotiate the HTTP Version requested
    UserAuthenticationError,                // Authentication failed with the provided credentials
    ProxyTunnelError,

    StatusNotSuccess,                       // Set by EnsureSuccessStatusCode() in case of non-success status code.
    InvalidResponse,                        // General error in response/malformed response
    
    // ALTERNATIVE name: InvalidResponseHeaders
    InvalidResponseHeader,                  // Error with response headers
    
    // OPTIONAL TO NOT INCLUDE: The one's below are specific cases of errors with the response.
    // We may consider merging them into "InvalidResponse" and "InvalidResponseHeader"
    ResponseEnded,                          // Received EOF
    // These depend on SocketsHttpHandler configuration
    ContentBufferSizeExceeded,              // Response Content size exceeded MaxResponseContentBufferSize
    ResponseHeaderExceededLengthLimit,      // Response Header length exceeded MaxResponseHeadersLength
}

API Usage

With HttpClient methods:

try
{
    using var response = await httpClient.GetStringAsync(url);
}
catch (HttpRequestException ex) when (ex.HttpRequestError is HttpRequestError.ConnectionError or
                                                             HttpRequestError.SecureConnectionError or
                                                             HttpRequestError.NameResolutionError)
{
    // Retry
}

With response read Stream:

using var response = await httpClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
using var responseStream = await response.Content.ReadAsStreamAsync();
try
{
    await responseStream.ReadAsync(buffer);
}
catch (HttpResponseReadException ex)
{
    switch (ex.HttpRequestError)
    {
        // ....
    }
}

Risks

Any change to the error categorization is a breaking change, meaning that if we introduce new error codes for some users, we would break others.

Mobile handlers would need to implement their error mapping separately, meaning that we would lack platform consistency until that point. Whenever it's impossible to detect some error categories on those platforms, they would need to report a more generic error code eg. InvalidResponse instead of ResponseEnded. This is a challenge for users who need to include mobile targets in their cross-platform HttpClient code.

Since requests and connections are decoupled, connection-establishment errors (NameResolutionError, ConnectionError and SecureConnectionError) will be only reported if the originating request is still in the queue, otherwise we will swallow them. Observing these error codes is not a reliable way to observe ALL connection-level errors happening under the hood. Some users may find this surprising.

Alternative Designs

• Move errors that can occur during response content read to a separate HttpResponseReadError enum. HttpResponseReadException would hold values from that separate enum.
  • Upside: This might make it easier for users to implement unified error handling for the two API scenarios (HttpClient and Stream).
  • Downsides: might be harder to follow. Some error cases like TransportError or ProtocolError can occur in bothe cases, meaning that enum values would be shared.
• Reduce the granularity of the error codes, eg. avoid defining cases like ContentBufferSizeExceeded, and threat them as InvalidResponse.
  • Upside: Easier to implement and maintain.
  • Downside: we won't be able to handle valid error scenarios like the one mentioned in [API Proposal]: Support error codes for HttpRequestException #76644 (comment)
• Use exception subclasses instead of error codes for better extensibility. The most practical approach would be to keep HttpRequestException sealed, and define a class hierarchy under IOException (which then should be aways InnerException of HttpRequestException).
  • Downside: too many subclasses, such hierarchy is unprecedented in the BCL

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

HttpRequestException is thrown under various conditions but it does not provide an error code to differentiate those conditions.
Currently, it provides 'string Message' property inherited from Exception type and it can contain various error messages:

• The HTTP response headers length exceeded the set limit
• Cannot write more bytes to the buffer
• The response ended prematurely
• Received chunk header length could not be parsed
• Received an invalid status
• Received an invalid header line
• Received an invalid header name
• The server returned an invalid or unrecognized response
• etc.

The above strings are defined in https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Http/src/Resources/Strings.resx

If we want to pragmatically handle exceptions based on the cause, it would be better dealing with error codes rather than dealing with string values. For example, when we create a dashboard for different exceptions encountered, aggregating by error codes would make more sense that doing by string values. And we could categorize some errors into a group for a special handling.

Many other exceptions like SocketException and Win32Exception have an error code field already.
Would it make sense to provide a new property for HttpRequestException to hold error codes?

API Proposal

namespace System.Collections.Generic;

public class HttpRequestException : Exception
{
    .....
    public HttpRequestError HttpRequestErrorCode;
   .....
}

API Usage

try {
  await httpClient.SendAsync(...);
}
catch (HttpRequestException ex)
{
  switch (ex.HttpRequestErrorCode)
  {
    case: HttpRequestError.InvalidStatus: 
      // do something
      break;
    default:
      break;
  }
}

Alternative Designs

No response

Risks

No response

Author:	searchmon
Assignees:	-
Labels:	api-suggestion, area-System.Net.Http
Milestone:	-

[Symbai]

HttpRequestException has StatusCode property which already gives you the http status alias error code. (edit: Since .NET 5)

[ManickaP]

I'd love to see a comparison with other HTTP stack to see if they provide error codes like this or how do they report similar errors.
Disadvantage of this is that it somewhat puts our internal workings on a public API - the newly exposed error codes depend on our internal logic. So if we were to re-haul HTTP parsing for example, we'd still need to do it in way to provide the same error codes in same situations or introduce new ones and obsolete the old ones. Whether this is limiting or not, needs to be determine. We might attempt to choose the error codes in way that it is common for any HTTP logic - for that we should compare with other stacks. For example WinHTTP does expose something similar: https://learn.microsoft.com/en-us/windows/win32/winhttp/error-messages and it also has several obsoleted error codes.

[searchmon]

HttpRequestException.StatusCode is HttpStatusCode type and didn't help in my cases where its value is null.

Another point is that, though HttpRequestException can be a leaf exception, it often has IOException as an inner exception.
, which also misses a proper error code. (attached an example)

[ManickaP]

Triage: we'll decide if/how to solve this in 8.0 timeframe and if we'll do something, it'll aim at 8.0 release.

[antonfirsov]

Related requests from the past: #15567, #21447, #30896

[MihaZupan]

Another case where a more strongly-typed mechanism would be an improvement is the use of Exception.Data to communicate error conditions as part of the new H2 WebSockets feature.

runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs

Line 441 in 96b849e

ex.Data["HTTP2_ENABLED"] = false;

runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs

Line 1074 in 96b849e

exception.Data["SETTINGS_ENABLE_CONNECT_PROTOCOL"] = false;

Consuming this is rather ugly in YARP.