Add ValueTask to corefx

[benaadams]

ValueTask being developed for System.Threading.Tasks.Channels in corefxlab is very valuable on its own and should be included independently of Channels (and earlier).

ReadAsync is defined to return a ValueTask<T>, a new struct type included in this library. ValueTask<T> is a discriminated union of a T and a Task<T>, making it allocation-free for ReadAsync<T> to synchronously return a T value it has available (in contrast to using Task.FromResult<T>, which needs to allocate a Task<T> instance). ValueTask<T> is awaitable, so most consumption of instances will be indistinguishable from with a Task<T>. If a Task<T> is needed, one can be retrieved using ValueTask<T>'s AsTask() method, which will either return the Task<T> it stores internally or will return an instance created from the T it stores.

[stephentoub]

This is something we plan to do, it's just a matter of when and how. It's in the Channels library right now as a placeholder. You can see there's also a copy of it used internally in corefx, at least in one place.

[davidfowl]

We're going to do some measurements and get some concrete numbers. This could be huge for us (asp.net) in certain scenarios.

[mgravell]

Oh hell yes. Would use that. Would be great (although I imagine that ship has sailed) if libs like ADO.NET could use that for scenarios where data is probably available, like:

while(await reader.ReadAsync(cancel)) {
    ...
}

And of course sockets etc (it reminds me of the cheap return option of the non-task async IO API), and streams.

I can't see of any nice way of retrofitting this onto all the existing code without making the APIs ugly, but: yes. So much yes. I spend my life hunting allocations.

[benaadams]

Might loose some benefits with the Stream Async methods needing Task<int> returns; amended my Stream Api proposal https://github.com/dotnet/coreclr/issues/2179 to use ValueTask<int> rather than Task<int>; other non-value Tasks are ok as they can use cached Tasks

[JonHanna]

Glancing through this code is like when you open a present and only then realise you'd been wanting this thing you hadn't known existed, for some time.

[stephentoub]

Glad you like it :)

[omariom]

👍

[JonHanna]

Are there plans to let async be used with this? Having an async use the non-task-building constructor when it goes through a path that doesn't hit await would be awesome. Having it do so when it does go through await but only ever got a non-task-containing ValueTask even more so.

[svick]

@JonHanna I think you can generalize that to just "completed synchronously". That includes not hitting any awaits, awaiting non-task ValueTasks, awaiting already completed Tasks and awaiting any other awaitable whose IsCompleted returned true.

[stephentoub]

Are there plans to let async be used with this?

No concrete plans, but it's being discussed (cc: @jaredpar, @MadsTorgersen). There is value in doing so. However, using a ValueTask<TResult> instead of a Task<TResult> isn't a pure win. There is overhead associated with it (for example, it's a field bigger, which means you're not only passing back an extra field from a method, if you're awaiting one of these it's likely going to increase the size of an async caller's state machine object by the size of a field), and that's pure overhead without benefit in the case where the operation does complete asynchronously. That means developers really need to think carefully about switching to such a ValueTask<TResult> when exposing it in an API, and if they're optimizing it, they likely also care about other involved overheads. As such, it's likely that a careful developer who cares about such costs would choose to use a non-async method that wraps an async one, with that wrapper doing argument validation, checking for a fast path, etc., e.g.

public ValueTask<Something> BarAsync(string arg)
{
    if (arg == null) throw new ArgumentNullException("arg");
    return CanUseFastPath(arg) ?
        new ValueTask<Something>(ComputeSync(arg)) :
        new ValueTask<Something>(BarAsyncCore(arg));
}

private async Task<Something> BarAsyncCore(string arg) { ... }

That's not to say there isn't value in being able to write:

public ValueTask<Something> BarAsync(string arg)
{
    if (arg == null) throw new ArgumentNullException("arg");
    return BarAsyncCore(arg);
}

private async ValueTask<Something> BarAsynccore(string arg) { ... }

There obviously is. I'm simply highlighting it to point out that ValueTask<TResult> should only be used after careful consideration, and with such careful consideration, you're likely also considering other optimizations that might lead you away from this. And of course there's cost involved in developing such a feature to the language, in developers needing to learn about it, etc.

[JonHanna]

I think you can generalize that to just "completed synchronously"

In terms of desire, certainly. In terms of implementation I could see one being simpler than the other.

That means developers really need to think carefully about switching to such a ValueTask when exposing it in an API, and if they're optimizing it, they likely also care about other involved overheads.

If it's ever attempted and abandoned I'd still love to know how close it came to having the first example you give perform like the second. (If it's ever attempted and adopted I'll likely be doing my own experiments on the results to decide when I should and shouldn't use it anyway).

[stephentoub]

I would like to add this to corefx as part of a new library: System.Threading.Tasks.Extensions.dll.

Here’s a commit containing the library; I'll submit it as a PR once approved:
stephentoub/corefx@217376f

Here’s the API surface area being added:
https://github.com/stephentoub/corefx/blob/217376f756071bfd124b5408538457f411c7865c/src/System.Threading.Tasks.Extensions/ref/System.Threading.Tasks.Extensions.cs

The motivation for adding ValueTask is performance. We still encourage folks to use Task and Task<TResult> by default as the return type from async methods. However, for cases where the async method returns a Task<TResult>, very frequently returns synchronously, and is invoked on such a hot path that the cost of allocating the Task<TResult> for the synchronous result is prohibitive, ValueTask<TResult> may be used instead. ValueTask<TResult> is a struct-based discrimated union of a Task<TResult> and a TResult. If the method completes successfully synchronously, then it doesn’t need to allocate a Task<TResult>, and can instead just return a ValueTask<TResult> created from the raw TResult. Otherwise, it can return a ValueTask<TResult> created from the Task<TResult>.

Existing examples:

• We already had a strong use case for this in corefx and have an internal version of it used in System.Xml (we can rip that out and use this library instead once it’s available): Huge allocation reduction in XmlReader/Writer's async implementation corefx#1048
• I used this in the System.Threading.Tasks.Channels prototype on corefxlab, where it’s used in public surface area in one place for a significant performance benefit (again, once this lib is available, I’d replace the Channels one): https://github.com/dotnet/corefxlab/blob/master/src/System.Threading.Tasks.Channels/src/System/Threading/Tasks/Channels/IChannel.cs#L30
• ASP.NET has been looking at using it in a few places.

[benaadams]

it's a field bigger, which means you're not only passing back an extra field from a method

If the ValueTask return is always the last statement in each method in the call chain; will this type play well with jit tail call optimizations? Best I can seem to find is from 2009; which isn't RyuJit.

http://blogs.msdn.com/b/clrcodegeneration/archive/2009/05/11/tail-call-improvements-in-net-framework-4.aspx

• The caller or callee returns a value type.

We changed the JIT to recognize this case as well. Specifically the X64 calling convention dictates that for value types that don’t fit in a register (1,2,4, or 8 byte sized value types), that the caller should pass a ‘hidden’ argument called the return buffer. The CLR 2 JIT would pass a new return buffer to the callee, and then copy the data from the callee’s return buffer to the caller’s return buffer. The CLR 4 runtime now recognizes when it is safe to do so, and then just passes the caller’s return buffer into the callee as its return buffer. This means even when we don’t do a tail call, the code has at least one less copy, and it enables one more case to be ‘easy’!

Which would suggest sizeof(type+ptr) > 8 byte; it wouldn't be copied at each step of the stack unwind at least?

[mgravell]

I'd trail a tail-call for an unnecessary allocation in a heartbeat...

The bigger problem (for me, at least) is that a lot of the obvious places
to use this are in existing Task-based APIs: sockets, file-streams, ADO.NET
data-readers - the places that have probably read more than one "thing" and
can hand back synchronously most of the time. Without getting this down
into those layers, much of the useful scenarios may be limited.
On 30 Nov 2015 6:00 pm, "Ben Adams" notifications@github.com wrote:

If the ValueTask return is always the last statement in each method in the
call chain; will this type play well with jit tail call optimizations?

—
Reply to this email directly or view it on GitHub
https://github.com/dotnet/corefx/issues/4708#issuecomment-160706827.