-
Notifications
You must be signed in to change notification settings - Fork 5k
The server returned an invalid or unrecognized response" error when using System.Net.Http.HttpClient #19825
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
cc: @karelz The repro code posted above is a just a piece of the total repro. What server is this talking to? In order to repro the problem, we need a complete solution. The error being reported is about parsing the server response. So, it is important to use the same server to debug this issue. |
@corinas can you please provide more details here? Thanks! |
We've also encountered this issue. We only seem to get the unrecognized response when making HEAD requests to a server which serves a redirect.
We use WebListener from namespace WebApplication1
{
public class Program
{
public static void Main(string[] args)
{
var host = new WebHostBuilder()
.UseWebListener()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseUrls("http://test.com:5000")
.UseStartup<Startup>()
.Build();
host.Run();
}
}
} Startup.cs namespace WebApplication1
{
public class Startup
{
public void Configure(IApplicationBuilder app)
{
app.Run(context =>
{
if (context.Request.GetDisplayUrl().EndsWith("redirect.html"))
{
context.Response.StatusCode = 200;
return Task.CompletedTask;
}
context.Response.Redirect("/redirect.html");
return Task.CompletedTask;
});
}
}
} And created a console application which makes the requests. namespace ConsoleApp3
{
class Program
{
static void Main()
{
for (var i = 0; i < 10000; i++)
{
using (var httpClient = new HttpClient())
{
var request = new HttpRequestMessage(HttpMethod.Head, new Uri("http://test.com:5000"));
var response = httpClient.SendAsync(request).Result;
}
}
}
}
} |
@JosVerburg how is your repro and callstack related to the original one here? |
@JosVerburg Please put together a complete, simple, repro with a Visual Studio 2015 or 2017 solution. You can simply zip up the solution and post it here. That would be the best way for us to investigate. |
And the reason for a complete repro is that I tried your sample code above and was unable to put together a complete solution that would launch // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory) |
@davidsh I've setup a repo with the solution at https://github.com/JosVerburg/ReproduceUnrecognizedResponse. @karelz To me it seems like the same issue, if you still can't reproduce it with the example to confirm I'll make a separate issue. Additional information:
Response:
|
@JosVerburg why do you think it is the same issue? What is similar? (the callstack looks different to me) |
OK, I updated the callstacks in top post and latest reply and now I see the similarities. he inner exception in interesting in your case. Sorry for the confusion ... |
I've seen this issue consistently on Windows Server 2012 R2 running on .net core 1.0. I'm using ASP.NET Core Proxy to proxy a number of requests to another application running on WebListener. Proxy internally uses HttpClient. From the looks of it (applications set up and stacktrack), it the same issue as reported by @JosVerburg. Is there anything I can contribute in order to get a diagnosis of this issue? |
We've also tried to reproduce the issue with wget. Fiddler also doesn't indicate an invalid response. |
I've written a test client for full framework, see https://github.com/devatwork/ReproduceUnrecognizedResponse. Interestingly enough full framework also thinks this is an issue on the server side as well. Stack trace:
This seems to suggest that the server application returns an invalid response. In both Jos's case and my case the server application is using WebListener/HttpSysServer. The HTTP response messages are identical according to a Fiddler trace I've run. |
RFC 2626 and RFC 2731 states that a head request MUST NOT return a response body. See also the last paragraph of https://tools.ietf.org/html/rfc2616#section-4.3. Now give the HTTP request/response pair reported by @JosVerburg:
As to my current understanding of the HTTP 1.1 spec, the response is indeed invalid. If that is indeed the case, the problem is server-side either in http.sys, HttpSysServer or in ASP.NET Core and not client-side. Any thoughts or flaws in my reasoning? |
This is interesting. I tried out the revised repro. The one using .NET Framework for client size seems to throw the I have sent the trace logs to the WinHTTP team for analysis. Is it possible that the server is buggy and only sometimes sends content body payload in the response on the HEAD request? |
I've observed the same behavior. The .NET Core client seems to throw after a seemingly random number of iterations. The Fiddler logs did not show any difference in response. It looks like a timing issue.
I think there is also a server bug, see RFC 2616 section 9.4 and the HTTP observed by @JosVerburg and me. If you look at the application code of the server application, there is very little room for error. Is you change the repro WebApp to Also, this does not explain why the .NET Core HttpClient throws after some time. If the response is indeed invalid, the client should have thrown right away, just like the desktop behavior, right? |
We have finished the analysis of this issue. The problem is due to the server violating the RFC by returning an entity-body. It is sending a 0-byte chunk. The response to a HEAD request should be to send absolutely no bytes of data for the response body. The reason for the difference in behavior between .NET Framework and .NET Core is due to race conditions and architectural differences. .NET Framework will fail right away after the first request. It fails, by-design, with a ProtocolViolationException. .NET Core (on Windows) uses native WinHTTP. This is the explanation from the WinHTTP team about why it is failing later on (due to race condition).
We are closing this issue. Please open up a new issue with the ASP.NET team regarding WebListener. |
I've reported this to the ASP.NET Core team, see aspnet/HttpSysServer#339. Hopefully they will be able to address the issue soon. @davidsh: I don't think we've confirmed that the original issue reported by @corinas and @Petermarcu is actually caused by the same issue, so we may want to wait for their input? |
I ran the .net core test client on Linux (1.1.1-runtime docker image) connecting to the web application running on Windows and it does not fail at all. So the behavior is different on each platform/runtime combination. Although the root cause is on the server side, IMHO the clients should consistently report that the server returned a malformed HTTP 1.1 response. I'm interested to understand whether behavior differences are considered bugs or not. |
We don't consider this a bug. The difference is due to architectural differences and race conditions in the platform OS APIs that are used for HTTP. As indicated above, the server is violating the RFC and in those cases, some behaviors of the client can be undefined or variable in these edge cases.
If there an actionable repro that demonstrates this same error but without involving a HEAD request with invalid entity-body response content, please open up a new issue. It will make things easier for tracking. |
@davidsh I am getting the same error and created a new issue for it dotnet/corefx#21314 |
I have the same error as well while trying to connect to an SSE stream. This should be considered a bug whether or not the server is enforcing RFC protocols, mainly because well if your a client developer the framework should be flexible enough to allow me to get around this problem. As it stands there is no resolution for this issue at all. To see the issue, try to connect to This API Call and use the header Accept: text/event-stream, HttpClient will throw an exception every single time. It's easy to reproduce, what's odd is Fiddler returns the result back just fine. |
Hi, ALso How to debug this since it occurs randomly :( program.cs public class Program
{
public static void Main(string[] args)
{
var config = new ConfigurationBuilder()
.AddJsonFile(Path.Combine(Directory.GetCurrentDirectory(),"hosting.json"), optional: false)
.Build();
var useHttps = false;
bool.TryParse(config !=null ? config["useHttps"] : "false", out useHttps);
IWebHost host = null;
if (useHttps)
{
var fileInfo = new FileInfo(config["certName"]);
X509Certificate2 cert = new X509Certificate2(fileInfo.FullName, config["certPwd"]);
host = new WebHostBuilder()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseStartup<Startup>()
.UseKestrel(options =>
{
options.Listen(IPAddress.Loopback, 5000);
options.Listen(IPAddress.Any, 4430, listenOptions =>
{
listenOptions.UseHttps(cert);
});
})
.UseIISIntegration()
.Build();
}
else
{
host = new WebHostBuilder()
.UseStartup<Startup>()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseIISIntegration()
.UseUrls(config["url"])
.Build();
}
host.Run();
}
} HTTP network file public class NetworkExtensions
{
public static IConfiguration Configuration { get; set; }
public static ILogger Log { get; set; }
/// <summary>
/// Flurl to get api
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="requestDto"></param>
/// <param name="rateLimit">This param is used to Perform OKTA api Retry </param>
/// <returns></returns>
public static async Task<T> Get<T>(OktaRequestDto requestDto, OKTARateLimit rateLimit = null)
{
using (MiniProfiler.Current.Step("Okta : Get"))
{
// OKTA code execution starts here.
string url = requestDto.BaseUrl
.AppendPathSegment(requestDto.ApiName + requestDto.Query);
// Handle all querystring append.
if (requestDto.QueryStrings != null && requestDto.QueryStrings.Count > 0)
{
foreach (var querystring in requestDto.QueryStrings.Keys)
{
//FLURL is encoding the value, To ensure the value is passed correct, added true param to stop default behavior
url = url.SetQueryParam(querystring, requestDto.QueryStrings[querystring], true);
}
}
var response = await url.WithHeader("Authorization", requestDto.ApiKey).GetJsonAsync<T>();
Log.Information("response => " + JsonConvert.SerializeObject(response));
return response;
catch (FlurlHttpException ex)
{
// there is an OKTA exception, return the default value, The exception is captured in the FLURLConfig
OneLoginException ole = new OneLoginException(ex, requestDto);
throw ole;
}
catch (Exception ex)
{
throw ex;
}
}
}
} Error received:
[EDIT] Add C# syntax highlighting by @karelz |
I'm getting this error as well with aspnetcore 2.1 preview2 as well, using azure app service.
|
I'm experiencing the same issue on asp net core targetting net461 framework. The HttpClient we're using is System.Net.Http 4.3.3. Also, we recently have updated to reuse HttpClient and experiencing this error intermittently. |
@saibaskaran57 @bmarder the original issue was fixed in the server. You should not hit the same problem if you run against 2.0+ server (bug aspnet/HttpSysServer#339 was fixed in 2.0.0). |
Hi All, We are facing similar issue, when we request Web API end point with large data, it waits for 5 minutes and throws below Exception. There is no timeout configured at server side and we have client request timeoout is 20minutes. Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware: An unhandled exception has occurred: One or more errors occurred. (An error occurred while sending the request.) |
@rajugurrapu this is a closed issue so it won't be seen.please open a fresh issue for your problem, including version details, code snippet, stack, and URL if special. |
FYI: @rajugurrapu created new bug dotnet/corefx#30040 |
Getting this bug into the right place: dotnet/core#321
@corinas
Hello,
I am facing a problem when making concurrent requests to a web api when using dotnet core. Sometimes i'm getting the below error; the error is not happening all the time (usually it happens once or twice in 600 concurrent calls).
Here's the test method I use for testing:
When this error occurs, the response looks like this:
{StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: , Headers:{}}
Calling the same method from a testing project that uses dotnet framework 4.5 does not cause any issue.
Could you please advise?
Thanks!
Corina
The text was updated successfully, but these errors were encountered: