Fully buffer XML, JSON, and Razor request and ViewComponentResultExecutor response bodies

[Tratcher]

Is your feature request related to a problem? Please describe.

Sync IO is a frequent source of thread pool starvation. As such #5120 bans sync IO operations for each server by default. Unfortunately the MVC XML and JSON serializers and deserializers rely on sync IO, as does the Razor view engine.

Describe the solution you'd like

Fully buffer the request and response bodies for the XML, Newtonsoft.JSON, and Razor scenarios. The new JSON APIs for 3.0 should provide async implementations. The current implementations already rely on partial buffering, but perform sync IO if those buffers fill/drain.

This has a frequently requested side benefit of allowing the app to recover from response serialization errors and return a meaningful error message.

Here are a few example test failures:

Microsoft.AspNetCore.Mvc.FunctionalTests.JsonOutputFormatterTests.SerializableErrorIsReturnedInExpectedFormat...
System.InvalidOperationException : Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true.
 at Microsoft.AspNetCore.TestHost.AsyncStreamWrapper.Read(Byte[] buffer, Int32 offset, Int32 count) in /_/src/Hosting/TestHost/src/AsyncStreamWrapper.cs:line 50
   at System.IO.Stream.ReadByte()
   at Microsoft.AspNetCore.Mvc.Infrastructure.NonDisposableStream.ReadByte() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/NonDisposableStream.cs:line 134
   at System.Xml.EncodingStreamWrapper.ReadBOMEncoding(Boolean notOutOfBand)
   at System.Xml.EncodingStreamWrapper..ctor(Stream stream, Encoding encoding)
   at System.Xml.XmlUTF8TextReader.SetInput(Stream stream, Encoding encoding, XmlDictionaryReaderQuotas quotas, OnXmlDictionaryReaderClose onClose)
   at System.Xml.XmlDictionaryReader.CreateTextReader(Stream stream, Encoding encoding, XmlDictionaryReaderQuotas quotas, OnXmlDictionaryReaderClose onClose)
   at Microsoft.AspNetCore.Mvc.Formatters.XmlDataContractSerializerInputFormatter.CreateXmlReader(Stream readStream, Encoding encoding) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Formatters.Xml/XmlDataContractSerializerInputFormatter.cs:line 213
   at Microsoft.AspNetCore.Mvc.Formatters.XmlDataContractSerializerInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Formatters.Xml/XmlDataContractSerializerInputFormatter.cs:line 159
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/ModelBinding/Binders/BodyModelBinder.cs:line 157
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/ModelBinding/ParameterBinder.cs:line 258
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Controllers/ControllerBinderDelegateProvider.cs:line 76

Microsoft.AspNetCore.Mvc.FunctionalTests.XmlSerializerFormattersWrappingTest.ValidationProblem...
System.InvalidOperationException : Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true.
   at Microsoft.AspNetCore.TestHost.ResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count) in /_/src/Hosting/TestHost/src/ResponseStream.cs:line 155
   at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.FlushInternal(Boolean flushEncoder) in /_/src/Http/WebUtilities/src/HttpResponseStreamWriter.cs:line 336
   at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.Flush() in /_/src/Http/WebUtilities/src/HttpResponseStreamWriter.cs:line 282
   at System.Xml.XmlEncodedRawTextWriter.Flush()
   at System.Xml.XmlWellFormedWriter.Flush()
   at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o, XmlSerializerNamespaces namespaces, String encodingStyle, String id)
   at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o)
   at Microsoft.AspNetCore.Mvc.Formatters.XmlSerializerOutputFormatter.Serialize(XmlSerializer xmlSerializer, XmlWriter xmlWriter, Object value) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Formatters.Xml/XmlSerializerOutputFormatter.cs:line 257
   at Microsoft.AspNetCore.Mvc.Formatters.XmlSerializerOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Formatters.Xml/XmlSerializerOutputFormatter.cs:line 237
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultAsync(IActionResult result) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 137
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1089
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContext context) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1192
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1071
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 850
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 790
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContext context) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1146
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 752
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 122
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 80

Microsoft.AspNetCore.Mvc.FunctionalTests.InputFormatterTests.ValidationUsesModelMetadataFrom...
System.InvalidOperationException : Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true.
  at Microsoft.AspNetCore.TestHost.AsyncStreamWrapper.Read(Byte[] buffer, Int32 offset, Int32 count) in /_/src/Hosting/TestHost/src/AsyncStreamWrapper.cs:line 50
   at Microsoft.AspNetCore.WebUtilities.HttpRequestStreamReader.ReadIntoBuffer() in /_/src/Http/WebUtilities/src/HttpRequestStreamReader.cs:line 321
   at Microsoft.AspNetCore.WebUtilities.HttpRequestStreamReader.Read(Char[] buffer, Int32 index, Int32 count) in /_/src/Http/WebUtilities/src/HttpRequestStreamReader.cs:line 168
   at Newtonsoft.Json.JsonTextReader.ReadData(Boolean append, Int32 charsRequired) in /_/Src/Newtonsoft.Json/JsonTextReader.cs:line 343
   at Newtonsoft.Json.JsonTextReader.ReadData(Boolean append) in /_/Src/Newtonsoft.Json/JsonTextReader.cs:line 272
   at Newtonsoft.Json.JsonTextReader.ParseValue() in /_/Src/Newtonsoft.Json/JsonTextReader.cs:line 1665
   at Newtonsoft.Json.JsonTextReader.Read() in /_/Src/Newtonsoft.Json/JsonTextReader.cs:line 419
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter) in /_/Src/Newtonsoft.Json/JsonReader.cs:line 1233
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) in /_/Src/Newtonsoft.Json/Serialization/JsonSerializerInternalReader.cs:line 149
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Formatters.Json/JsonInputFormatter.cs:line 349
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/ModelBinding/Binders/BodyModelBinder.cs:line 157
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/ModelBinding/ParameterBinder.cs:line 258
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Controllers/ControllerBinderDelegateProvider.cs:line 76
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ControllerActionInvoker.cs:line 382
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 790
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContext context) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1146
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 752
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 122

Microsoft.AspNetCore.Mvc.FunctionalTests.ViewComponentFromServicesTest.ViewComponents...
System.InvalidOperationException : Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true.
  at Microsoft.AspNetCore.TestHost.ResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count) in /_/src/Hosting/TestHost/src/ResponseStream.cs:line 155
   at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.FlushInternal(Boolean flushEncoder) in /_/src/Http/WebUtilities/src/HttpResponseStreamWriter.cs:line 336
   at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.Dispose(Boolean disposing) in /_/src/Http/WebUtilities/src/HttpResponseStreamWriter.cs:line 301
   at System.IO.TextWriter.Dispose()
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewComponentResultExecutor.ExecuteAsync(ActionContext context, ViewComponentResult result) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.ViewFeatures/ViewComponentResultExecutor.cs:line 126
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultAsync(IActionResult result) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 137
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResultFilterAsync[TFilter,TFilterAsync]() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1089
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContext context) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1192
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1071
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 850
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 790
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContext context) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 1146
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 752
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 122
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync() in /_/src/Mvc/src/Microsoft.AspNetCore.Mvc.Core/Infrastructure/ResourceInvoker.cs:line 80
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext) in /_/src/Http/Routing/src/EndpointMiddleware.cs:line 42
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext) in /_/src/Http/Routing/src/EndpointRoutingMiddleware.cs:line 78
   at Microsoft.AspNetCore.Mvc.FunctionalTests.CultureReplacerMiddleware.Invoke(HttpContext context) in /_/src/Mvc/test/Microsoft.AspNetCore.Mvc.FunctionalTests/Infrastructure/CultureReplacerMiddleware.cs:line 45
   at Microsoft.AspNetCore.TestHost.HttpContextBuilder.<>c__DisplayClass14_0.<<SendAsync>b__0>d.MoveNext() in /_/src/Hosting/TestHost/src/HttpContextBuilder.cs:line 74

@davidfowl @mkArtakMSFT @pranavkm

[mkArtakMSFT]

@rynowak, thoughts?
We've been avoiding buffering the responses and requests and that was the direction we had taken in communication with customers in the past. Should this change?

[davidfowl]

Lets discuss this with @DamianEdwards and @Tratcher. We discussed making this configurable so that it would blow up if it went above some threshold by default and it could be configurable per request otherwise.

[Tratcher]

We may have enough control over razor to make it do smarter async flushing rather than full buffering. E.g. FlushAsync between tags if the buffer is more than 75% full.

[Tratcher]

Offline design notes from @danroth27 and @davidfowl:
Customers don't want a response buffering size limit, it's just going to fail, get raised to MaxInt, and then cause OutOfMemory exceptions.
Proposal: Use the same file backed buffering mechanic as we do for requests, buffer the full response, and then flush once the serializer is done. HttpAbstractions can provide the buffer stream implementation, but MVC needs to flush it since it's the one that knows when the serilaizer is done.

[halter73]

I hope this doesn't get used as an excuse to continue allowing Razor to do sync writes/flushes. Buffering is a decent mitigation, but not nearly as good as top-to-bottom async I/O.

[rynowak]

@halter73 do you have a (non-glib) proposal for what that looks like?

For context, we already do something internally that might map pretty well to pipelines. If it is a good fit, and all 3.0 users can run on pipelines then I'd like to see what it would take to migrate Razor.

[Tratcher]

@rynowak pipelines are available in a few places now for you to start prototyping with.

[Tratcher]

The sync IO change has been merged with a few local opt-outs for things like the XML serilaizer. These should still be addressed.