Developers can use CollectionsMarshal ref accessors for Dictionary<TKey, TValue>

[benaadams]

Edited by @layomia. Original proposal by @benaadams (click to view)

They aren't "safe" operations as the backing store can change if items are added, removed, etc and the changes aren't tracked

namespace System.Runtime.InteropServices
{
    public partial static class CollectionsMarshal
    {
        public ref TValue ValueRef<TKey, TValue>(Dictionary<TKey, TValue> dict, TKey key);
    }
}

However it is useful to get a reference to modify struct TValue entries without a full copy and replace update.

/cc @jkotas

/cc @stephentoub is it valid for ConcurrentDictionary?

---

CollectionsMarshal ref accessors for Dictionary<TKey, TValue>

Attempting to obtain a value in a dictionary, and adding it if not present, is a common scenario in .NET applications. The existing mechanism for achieving this today is by using Dictionary<TKey, TValue>.TryGetValue(Tkey key, out TValue value), and adding the value to the dictionary if not present. This causes duplicate hash lookups:

if (!dictionary.TryGetValue(key, out MyType myValue)) // hash lookup
{
    myValue = CreateValue();
    dictionary.Add(key, myValue); // duplicate hash lookup
}

Another scenario is updating struct values in dictionaries. The existing pattern for achieving this causes struct copies and duplicate hash lookups, which potentially have non-trivial performance costs for large structs:

struct LargeStruct
{
    // Other members...
    
    public int MyInt { get; set; }

    // Other members...
}

LargeStruct myValue = dictionary[key]; // hash lookup, struct copy
myValue.MyInt++;
dictionary[key] = myValue; // another hash lookup, another struct copy

Motivation

• Provide a mechanism which avoids duplicate lookups when obtaining dictionary values which may not be present.
• Provide a mechanism which avoids copies and duplicate lookups when mutating struct dictionary values.

API proposal

namespace System.Runtime.InteropServices
{
    public static class CollectionsMarshal
    {
        /// <summary>
        ///   Gets a reference to the value associated with the specified key, or throws a KeyNotFoundException
        /// </summary>
        public static ref TValue GetValueRef<TKey, TValue>(Dictionary<TKey, TValue> dictionary, [NotNull] TKey key);

        /// <summary>
        ///   Gets a reference to the value associated with the specified key, or returns Unsafe.NullRef<TValue>
        /// </summary>
        public static ref TValue TryGetValueRef<TKey, TValue>(Dictionary<TKey, TValue> dictionary, [NotNull] TKey key, out bool exists);

        /// <summary>
        ///   Gets a reference to the value associated with the specified key, or inserts it with value default(TValue).
        /// </summary>
        public static ref TValue GetValueRefOrAddDefault<TKey, TValue>(Dictionary<TKey, TValue> dictionary, [NotNull] TKey key, out bool exists);
    }
}

CollectionsMarshal is an unsafe class that provides a set of methods to access the underlying data representations of collections.

API usages

Updating struct value in dictionary: KeyNotFoundException thrown when key not present in dictionary

This pattern is helpful when caller wants to optimally update a struct in a scenario where the key being absent is an error state. Creating a value and adding it to the dictionary, if not already present, is not desired.

try
{
    ref MyType value = CollectionsMarshal.GetValueRef(dictionary, key);
    value.MyInt++;
}
catch (KeyNotFoundException exception)
{
    // Handle exception
}

Unsafe.NullRef<TValue>() returned when key not present in dictionary

This pattern satisfies both the optimal struct value update and optimal "get or add" value scenarios.

ref MyType value = CollectionsMarshal.TryGetValueRef(dictionary, key, out bool exists);

if (exists)
{
    value.MyInt++;
}
else
{
    ref value = new MyType() { MyInt = 1 };
}

default(TValue) returned when key not present in dictionary

This pattern also satisfies both the optimal struct value update and optimal "get or add" value scenarios. A struct default value always being instantiated may cause the TryGetValueRef to be preferred, depending on the perf scenario.

ref MyType value = CollectionsMarshal.GetValueRefOrDefault(dictionary, key, out bool exists);
value.MyInt++;

if (exists)
{
    // Do something if I care that the key already existed.
}

Alternative design

GetOrAdd methods, similar to those on ConcurrentDictionary<TKey, TValue>, were proposed in #15059.

public class Dictionary<TKey, TValue>
{
    public TValue GetOrAdd([NotNull] TKey key, Func<TKey, TValue> valueFactory);
}

/* OR */

public class Dictionary<TKey, TValue>
{
    public TValue GetOrAdd<TState>([NotNull] TKey key, TState state, Func<TKey, TState, TValue> valueFactory);
}

Upsides

• More discoverable/friendly than the relatively advanced APIs in this proposal.
• Follows precendent in the BCL i.e. ConcurrentDictionary<TKey, TValue>.

Downsides

• Not pay-for-play. Since the new methods would live in System.Private.CoreLib, generic expansion issues due to struct-based generics will affect all users of Dictionary<TKey, TValue>, not just those that use them.
  • An argument against this point is that the methods can live in an extensions class, but that won't provide any perf benefits since they won't be able to access non-public members.
• Doesn't address the issue of copying when mutating large struct values.

Open questions

Should the out bool exists parameter on TryGetValueRef be removed?

Since a call to Unsafe.IsNullRef<T>(ref value) can indicate whether the value exists in the dictionary, the second method could simply be:

public static class CollectionsMarshal
{
    /// <summary>
    ///   Gets a reference to the value associated with the specified key, or returns Unsafe.NullRef<TValue>
    /// </summary>
    public static ref TValue TryGetValueRef<TKey, TValue>(Dictionary<TKey, TValue> dictionary, [NotNull] TKey key);
}

Usage

ref MyType value = CollectionsMarshal.TryGetValueRef(dictionary, key);

if (!Unsafe.IsNullRef(ref value))
{
    value.MyInt++;
}
else
{
    ref value = new MyType() { MyInt = 1 };
}

Any concerns about API bloat in the CollectionsMarshal type?

The generic expansion highlighted in the GetOrAdd-based alternative doesn't apply much here since the new methods will live in CollectionsMarshal and will be pay-for-play. However, are there concerns about bloating the type?

One answer here states as follows:

I'd think to the same extent as any other/similar type such as MemoryMarshal, Unsafe, etc.

As it is today, we have 1 method in it and this proposal is adding 2-3 more. We only have a limited number of collection types in S.P.Corelib and a more limited set where returning a ref makes sense, so I don't think we are at risk of overloading it.

[jkotas]

Same comment https://github.com/dotnet/corefx/issues/31597#issuecomment-410452767 applies here.

[benaadams]

Updated

[vcsjones]

@benaadams

public ref TValue ValueRef<TKey, TValue>(Dictionary<TKey, TValue> TKey key);

Did you mean to give the dictionary a parameter name and have two different parameters? Like

public ref TValue ValueRef<TKey, TValue>(Dictionary<TKey, TValue> dictionary, TKey key);

[safern]

@benaadams does it make sense to create 1 issue for: #27061 and this guy, to propose adding CollectionsMarshal class and having some rationale and usage, to bring it over to API review?

Marking as 3.0 for now to not loose attention and then we can decide if we bring them individually or together as 1 issue to API Review.

[danmoseley]

Not required for 3.0 release

[terrajobst]

@benaadams same question as on the other issue for CollectionMarshal: what are compelling scenarios for these APIs?

[benaadams]

struct LargeStruct
{
   // ...
   int Value;
}

1 hash lookup, no copies to update a struct

dict.ValueRef(key).Value++; // hash lookup

vs

2 hash lookup, 2 copies to update a struct

var s = dict[key]; // hash lookup, struct copy
s.Value++;
dict[key] = s; // hash lookup, struct copy

[terrajobst]

So if we were to expose an API that would allow you to cache the hash key, would this address your need? IOW, between struct copies and hash lookups, what hurts more?

[benaadams]

The hashing isn't particularly expensive (e.g. if its an int key there is no cost); its then taking the mod of that and traversing the arrays while doing the lookup (i.e. the search).

The struct copies are expensive as if you want to modify one property of the struct its disproportionately expensive.

I do accept its inherently unsafe if you mix it with other operations on the list or dictionary; and then try to keep using the ref, so it would be nice to add something like a [CallerMustBeUnsafe] attribute dotnet/roslyn#8663 so it can only be done in an unsafe block

[terrajobst]

The hashing isn't particularly expensive (e.g. if its an int key there is no cost); its then taking the mod of that and traversing the arrays while doing the lookup (i.e. the search).

The struct copies are expensive as if you want to modify one property of the struct its disproportionately expensive.

That makes sense.

it would be nice to add something like a [CallerMustBeUnsafe] attribute

Other people might feel differently but the whole point is that this should be a good enough speed bump. Putting this on a type in System.Runtime.InteropServices that isn't an extension method is sufficient to me. We could prefix the APIs with Unsafe too, but I'm not sure we're helping much with this.

The larger question is: where do we draw the line with escape hatches like this? In a sense, we're chasing the 1% cases here, but I'm a bit scared that we're creating a mess the more APIs like this we're exposing throughout the BCL. And I don't have a good handle on this where we hold the bar.

Do you have any real workload where this would show up? Of course, one can always fabricate a micro benchmark; I don't doubt there are cases where this API would speed things up. The point is: how much does this matter in real code?

[benaadams]

Looking at these apis more holistically:

For the dictionary you can achieve a similar; but safer, effect by wrapping the struct in a class then using the class handle. Its safer because the reference will remain valid across dictionary operations; and then a "live" reference can be more easily passed around and stored.

For the list List<T> AsSpan https://github.com/dotnet/corefx/issues/31597 would enable the same ability (working from the returned Span<T> which returns ref); but also be a more useful api.

For the dictionary using a class indirection doesn't cause too much issue because the data layout isn't important to the calling code (as how its internally stored in dictionary can't be consumed).

For list; that its a straight in-order array is useful, as then a consumer if they had access to it, can do things like vectorize the processing of the data, which a class indirection would break.

So in preference I'd drop these apis and add the List<T> AsSpan https://github.com/dotnet/corefx/issues/31597 one as it enables the same scenario as this api, plus other opportunities like vectoization (which this doesn't).

e.g. using the other api to achieve the same

AsSpan(list)[index].Value++; // though likely would cache the span if it was in a loop

vs

ItemRef(list, index).Value++

[tannergooding]

@benaadams, could you update this proposal to just be for Dictionary?

Seeing as we already reviewed/approved the List<T> changes here: https://github.com/dotnet/corefx/issues/31597 (and implemented https://source.dot.net/#System.Private.CoreLib/CollectionsMarshal.cs,18)