Investigate options to allow pooling of JSON serialization infrastructure

[DamianEdwards]

Related:

• Consider caching Utf8JsonWriter and PooledByteBufferWriter on a thread static in JsonSerializer.Serialize runtime#69889
• Implement a heuristic for using fast-path serialization in streaming JsonSerializer methods. runtime#78646
• Understand allocations discrepancies in Json benchmarks #45330

Our JSON-related extension methods on HttpResponse today and the inherent JSON serialization implemented by Minimal APIs (either directly from route handler returned objects or the JSON IResult types) call through the JSON serializers async code paths. These paths don't implement any pooling for the underlying infrastructure like Utf8JsonWriter resulting in repeated allocations of those types, which for small payloads can represent the majority of allocations for the related requests.

We should explore options to enable these features to either pool the underlying types or instead go through the synchronous code paths of JsonSerializer instead, as they implement pooling under the covers, with appropriate heuristics or options to ensure expectations around memory allocations, back pressure, etc. are maintained.

Note it's possible existing ongoing work in the runtime will yield some benefits here already.

[captainsafia]

Triage: @BrennanConroy Is this something that you are taking a look at?

Update triage: Brennan has updated benchmark apps to pool the UTF8Writer and used the source generator for message payloads and this led to improvements in the number of allocations that occur from the JSON middleware.

Also merged a PR for minimal APIs that reduces allocations for minimal API scenarios via #45359 and Brennan is working on a change to remove boxing struct types in the serialization.

Stick this in .NET 8 since there's work in progress on this front.

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.