diff --git a/docs/standard/memory-and-spans/index.md b/docs/standard/memory-and-spans/index.md new file mode 100644 index 0000000000000..c31d2a40e7c8d --- /dev/null +++ b/docs/standard/memory-and-spans/index.md @@ -0,0 +1,50 @@ +--- +title: "Memory and spans" +ms.date: "10/03/2018" +ms.technology: dotnet-standard +helpviewer_keywords: + - "Memory" + - "Span" + - buffers" + - "pipeline processing" +author: "rpetrusha" +ms.author: "ronpet" +--- +# Memory- and span-related types + +Starting with .NET Core 2.1, .NET includes a number of interrelated types that represent a contiguous, strongly-typed region of arbitrary memory. These include: + +- , a type that is used to access a contiguous region of memory. A instance can be backed by an array of type `T`, a , a buffer allocated with [stackalloc](~/docs/csharp/language-reference/keywords/stackalloc.md), or a pointer to unmanaged memory. Because it has to be allocated on the stack, it has a number of restrictions. For example, a field in a class cannot be of type , nor can span be used in asynchronous operations. + +- , an immutable version of the structure. + +- , a contiguous region of memory that is allocated on the managed heap rather than the stack. A instance can be backed by an array of type `T` or a . Because it can be stored on the managed heap, has none of the limitations of . + +- , an immutable version of the structure. + +- , which allocates strongly-typed blocks of memory from a memory pool to an owner. instances can be rented from the pool by calling and released back to the pool by calling . + +- , which represents the owner of a block of memory and controls its lifetime management. + +- , an abstract base class that can be used to replace the implementation of so that can be backed by additional types, such as safe handles. is intended for advanced scenarios. + +- , a wrapper for a particular number of array elements starting at a particular index. + +- , a collection of extension methods for converting strings, arrays, and array segments to blocks. + +> [!NOTE] +> For earlier frameworks, and are available in the [System.Memory NuGet package](https://www.nuget.org/packages/System.Memory/). + +For more information, see the namespace. + +## Working with memory and span + +Because the memory- and span-related types are typically used to store data in a processing pipeline, it is important that developers follow a set of best practices when using , , and related types. These best practices are documented in [Memory\ and Span\ usage guidelines](memory-t-usage-guidelines.md). + +## See also + +- +- +- +- +- \ No newline at end of file diff --git a/docs/standard/memory-and-spans/memory-t-usage-guidelines.md b/docs/standard/memory-and-spans/memory-t-usage-guidelines.md new file mode 100644 index 0000000000000..6c6904e75a732 --- /dev/null +++ b/docs/standard/memory-and-spans/memory-t-usage-guidelines.md @@ -0,0 +1,353 @@ +--- +title: "Memory<T> and Span<T> usage guidelines" +ms.date: "10/01/2018" +helpviewer_keywords: + - "Memory<T> and Span<T> best practices" + - "using Memory<T> and Span<T>" +author: "rpetrusha" +ms.author: "ronpet" +--- +# Memory\ and Span\ usage guidelines + +.NET Core includes a number of types that represent an arbitrary contiguous region of memory. .NET Core 2.0 introduced and , which are lightweight memory buffers that can be backed by managed or unmanaged memory. Because these types can be stored on the stack, they are unsuitable for a number of scenarios, including asynchronous method calls. .NET Core 2.1 adds a number of additional types, including , , , and . Like , and its related types can be backed by both managed and unmanaged memory. Unlike , can only be stored on the managed heap. + +Both and are buffers of structured data that can be used in pipelines. That is, they are designed so that some or all of the data can be efficiently passed to components in the pipeline, which can process them and optionally modify the buffer. Because and its related types can be accessed by multiple components or by multiple threads, it's important that developers follow some standard usage guidelines to produce robust code. + +## Owners, consumers, and lifetime management + +Since buffers can be passed around between APIs, and since buffers can sometimes be accessed from multiple threads, it's important to consider lifetime management. There are three core concepts: + +- **Ownership**. The owner of a buffer instance is responsible for lifetime management, including destroying the buffer when it's no longer in use. All buffers have a single owner. Generally the owner is the component that created the buffer or that received the buffer from a factory. Ownership can also be transferred; **Component-A** can relinquish control of the buffer to **Component-B**, at which point **Component-A** may no longer use the buffer, and **Component-B** becomes responsible for destroying the buffer when it's no longer in use. + +- **Consumption**. The consumer of a buffer instance is allowed to use the buffer instance by reading from it and possibly writing to it. Buffers can have one consumer at a time unless some external synchronization mechanism is provided. Note that the active consumer of a buffer isn't necessarily the buffer's owner. + +- **Lease**. The lease is the length of time that a particular component is allowed to be the consumer of the buffer. + +The following pseudo-code example illustrates these three concepts. It includes a `Main` method that instantiates a buffer of type , calls the `WriteInt32ToBuffer` method to write the string representation of an integer to the buffer, and then calls the `DisplayBufferToConsole` method to display the value of the buffer. + +```csharp +using System; + +class Program +{ + // Write 'value' as a human-readable string to the output buffer. + void WriteInt32ToBuffer(int value, Buffer buffer); + + // Display the contents of the buffer to the console. + void DisplayBufferToConsole(Buffer buffer); + + // Application code + static void Main() + { + var buffer = CreateBuffer(); + try { + int value = Int32.Parse(Console.ReadLine()); + WriteInt32ToBuffer(value, buffer); + DisplayBufferToConsole(buffer); + } + finally { + buffer.Destroy(); + } + } +} +``` + +The `Main` method creates the buffer (in this case an instance) and so is its owner. Therefore, `Main` is responsible for destroying the buffer when it's no longer in use. It does this by calling the buffer's method. (The method here actually clears the buffer's memory; the structure doesn't actually have a method that destroys the buffer.) + +The buffer has two consumers, `WriteInt32ToBuffer` and `DisplayBufferToConsole`. There is only one consumer at a time (first `WriteInt32ToBuffer`, then `DisplayBufferToConsole`), and neither of the consumers owns the buffer. Note also that "consumer" in this context doesn't imply a read-only view of the buffer; consumers can modify the buffer's contents, as `WriteInt32ToBuffer` does, if given a read/write view of the buffer. + +The `WriteInt32ToBuffer` method has a lease on (can consume) the buffer between the start of the method call and the time the method returns. Similarly, `DisplayBufferToConsole` has a lease on the buffer while it's executing, and the lease is released when the method unwinds. (There is no API for lease management; a "lease" is a conceptual matter.) + +## Memory\ and the owner/consumer model + +As the [Owners, consumers, and lifetime management](#owners-consumers-and-lifetime-management) section notes, a buffer always has an owner. .NET Core supports two ownership models: + +- A model that supports single ownership. A buffer has a single owner for its entire lifetime. + +- A model that supports ownership transfer. Ownership of a buffer can be transferred from its original owner (its creator) to another component, which then becomes responsible for the buffer's lifetime management. That owner can in turn transfer ownership to another component, and so on. + +You use the interface to explicitly manage the ownership of a buffer. supports both ownership models. The component that has an reference owns the buffer. The following example uses an instance to reflect the ownership of an buffer. + +[!code-csharp[ownership](~/samples/snippets/standard/buffers/memory-t/owner/owner.cs)] + +We can also write this example with the [`using`](~/docs/csharp/language-reference/keywords/using-statement.md): + +[!code-csharp[ownership-using](~/samples/snippets/standard/buffers/memory-t/owner-using/owner-using.cs)] + +In this code: + +- The `Main` method holds the reference to the instance, so the `Main` method is the owner of the buffer. + +- The `WriteInt32ToBuffer` and `DisplayBufferToConsole` methods accept xref:System.Memory%601> as a public API. Therefore, they are consumers of the buffer. And they only consume it one at a time. + +Although the `WriteInt32ToBuffer` method is intended to write a value to the buffer, the `DisplayBufferToConsole` method isn't. To reflect this, it could have accepted an argument of type . For additional information on , see [Rule #2: Use ReadOnlySpan\ or ReadOnlyMemory\ if the buffer should be read-only](#rule-2). + +### "Ownerless" Memory\ instances + +You can create a instance without using . In this case, ownership of the buffer is implicit rather than explicit, and only the single-owner model is supported. You can do this by: + +- Calling one of the constructors directly, passing in a `T[]`, as the following example does. + +- Calling the [String.AsMemory](xref:System.MemoryExtensions.AsMemory(System.String)) extension method to produce a `ReadOnlyMemory` instance. + +[!code-csharp[ownerless-memory](~/samples/snippets/standard/buffers/memory-t/ownerless/ownerless.cs)] + +The method that initially creates the instance is the implicit owner of the buffer. Ownership cannot be transferred to any other component because there is no instance to facilitate the transfer. (As an alternative, you can also imagine that the runtime's garbage collector owns the buffer, and all methods just consume the buffer.) + +## Usage guidelines + +Because a memory block is owned but is intended to be passed to multiple components, some of which may operate upon a particular memory block simultaneously, it is important to establish guidelines for using both and . Guidelines are necessary because: + +- It is possible for a component to retain a reference to a memory block after its owner has released it. + +- It is possible for a component to operate on a buffer at the same time that another component is operating on it, in the process corrupting the data in the buffer. + +- While the stack-allocated nature of optimizes performance and makes the preferred type for operating on a memory block, it also subjects to some major restrictions restrictions. It is important to know when to use a and when to use . + +The following are our recommendations for successfully using and its related types. Note that guidance that applies to and also applies to and unless we explicitly note otherwise. + +**Rule #1: For a synchronous API, use Span\ instead of Memory\ as a parameter if possible.** + + is more versatile than and can represent a wider variety of contiguous memory buffers. also offers better performance than >. Finally, you can use the property to convert a instance to a , although Span\-to-Memory\ conversion isn't possible. So if your callers happen to have a instance, they'll be able to call your methods with parameters anyway. + +Using a parameter of type instead of type also helps you write a correct consuming method implementation. You'll automatically get compile-time checks to ensure that you're not attempting to access the buffer beyond your method's lease (more on this later). + +Sometimes, you'll have to use a parameter instead of a parameter, even if you're fully synchronous. Perhaps an API that you depend accepts only arguments. This is fine, but be aware of the tradeoffs involved when using synchronously. + + instance begins when the method is entered, and it ends when the method exits. Consider the following example, which calls `Log` in a loop based on input from the console. + +[!code-csharp[void-returning](~/samples/snippets/standard/buffers/memory-t/void-returning/void-returning.cs#1)] + +If `Log` is a fully synchronous method, this code will behave as expected because there is only one active consumer of the memory instance at any given time. +But imagine instead that `Log` has this implementation. + +```csharp +// !!! INCORRECT IMPLEMENTATION !!! +static void Log(ReadOnlyMemory message) +{ + // Run in background so that we don't block the main thread while performing IO. + Task.Run(() => { + StreamWriter sw = File.AppendText(@".\input-numbers.dat"); + sw.WriteLine(message); }); +} +``` + +In this implementation, `Log` violates its lease because it still attempts to use the instance in the background after the original method has returned. The `Main` method could mutate the buffer while `Log` attempts to read from it, which could result in data corruption. + +There are several ways to resolve this: + +- The `Log` method can return a instead of `void`, as the following implementation of the `Log` method does. + + [!code-csharp[task-returning](~/samples/snippets/standard/buffers/memory-t/task-returning2/task-returning2.cs#1)] + +- `Log` can instead be implemented as follows: + + [!code-csharp[defensive-copy](~/samples/snippets/standard/buffers/memory-t/task-returning/task-returning.cs#1)] + +**Rule #4: If your method accepts a Memory\ and returns a Task, you must not use the Memory\ instance after the Task transitions to a terminal state.** + +This is just the async variant of Rule #3. The `Log` method from the earlier example can be written as follows to comply with this rule: + +[!code-csharp[task-returning-async](~/samples/snippets/standard/buffers/memory-t/void-returning-async/void-returning-async.cs#1)] + +Here, "terminal state" means that the task transitions to a completed, faulted, or canceled state. In other words, "terminal state" means "anything that would cause await to throw or to continue execution." + +This guidance applies to methods that return , , , or any similar type. + +**Rule #5: If your constructor accepts Memory\ as a parameter, instance methods on the constructed object are assumed to be consumers of the Memory\ instance.** + +Consider the following example: + +```csharp +class OddValueExtractor { + public OddValueExtractor(ReadOnlyMemory input); + public bool TryReadNextOddValue(out int value); +} + +void PrintAllOddValues(ReadOnlyMemory input) +{ + var extractor = new OddValueExtractor(input); + while (extractor.TryReadNextOddValue(out int value)) + { + Console.WriteLine(value); + } +} +``` + +Here, the `OddValueExtractor` constructor accepts a `ReadOnlyMemory` as a constructor parameter, so the constructor itself is a consumer of the `ReadOnlyMemory` instance, and all instance methods on the returned value are also consumers of the original `ReadOnlyMemory` instance. This means that `TryReadNextOddValue` consumes the `ReadOnlyMemory` instance, even though the instance isn't passed directly to the `TryReadNextOddValue` method. + +**Rule #6: If you have a settable Memory\-typed property (or an equivalent instance method) on your type, instance methods on that object are assumed to be consumers of the Memory\ instance.** + +This is really just a variant of Rule #5. This rule exists because property setters or equivalent methods are assumed to capture and persist their inputs, so instance methods on the same object may utilize the captured state. + +The following example triggers this rule: + +```csharp +class Person +{ + // Settable property. + public Memory FirstName { get; set; } + + // alternatively, equivalent "setter" method + public SetFirstName(Memory value); + + // alternatively, a public settable field + public Memory FirstName; +} +``` + +**Rule #7: If you have an IMemoryOwner\ reference, you must at some point dispose of it or transfer its ownership (but not both).** + +Since a instance may be backed by either managed or unmanaged memory, the owner must call when work performed on the instance is complete. Alternatively, the owner may transfer ownership of the instance to a different component, at which point the acquiring component becomes responsible for calling at the appropriate time (more on this later). + +Failure to call the method may lead to unmanaged memory leaks or other performance degradation. + +This rule also applies to code that calls factory methods like . The caller becomes the owner of the returned and is responsible for disposing of the instance when finished. + +**Rule #8: If you have an IMemoryOwner\ parameter in your API surface, you are accepting ownership of that instance.** + +Accepting an instance of this type signals that your component intends to take ownership of this instance. Your component becomes responsible for proper disposal according to Rule #7. + +Any component that transfers ownership of the instance to a different component should no longer use that instance after the method call completes. + +> [!IMPORTANT] +> If your constructor accepts as a parameter, its type should implement , and your method should call . + +**Rule #9: If you're wrapping a synchronous p/invoke method, your API should accept Span\ as a parameter.** + +According to Rule #1, is generally the correct type to use for synchronous APIs. You can pin instances via the [`fixed`](~/docs/csharp/language-reference/keywords/fixed-statement.md) keyword, as in the following example. + +```csharp +using System.Runtime.InteropServices; + +[DllImport(...)] +private static extern unsafe int ExportedMethod(byte* pbData, int cbData); + +public unsafe int ManagedWrapper(Span data) +{ + fixed (byte* pbData = &MemoryMarshal.GetReference(data)) + { + int retVal = ExportedMethod(pbData, data.Length); + + /* error checking retVal goes here */ + + return retVal; + } +} +``` + +In the previous example, `pbData` can be null if, for example, the input span is empty. If the exported method absolutely requires that `pbData` be non-null, even if `cbData` is 0, the method can be implemented as follows: + +```csharp +public unsafe int ManagedWrapper(Span data) +{ + fixed (byte* pbData = &MemoryMarshal.GetReference(data)) + { + byte dummy = 0; + int retVal = ExportedMethod((pbData != null) ? pbData : &dummy, data.Length); + + /* error checking retVal goes here */ + + return retVal; + } +} +``` + +**Rule #10: If you're wrapping an asynchronous p/invoke method, your API should accept Memory\ as a parameter.** + +Since you cannot use the [`fixed`](~/docs/csharp/language-reference/keywords/fixed-statement.md) keyword across asynchronous operations, you use the method to pin instances, regardless of the kind of contiguous memory the instance represents. The following example shows how to use this API to perform an asynchronous p/invoke call. + +```csharp +using System.Runtime.InteropServices; + +[UnmanagedFunctionPointer(...)] +private delegate void OnCompletedCallback(IntPtr state, int result); + +[DllImport(...)] +private static extern unsafe int ExportedAsyncMethod(byte* pbData, int cbData, IntPtr pState, IntPtr lpfnOnCompletedCallback); + +private static readonly IntPtr _callbackPtr = GetCompletionCallbackPointer(); + +public unsafe Task ManagedWrapperAsync(Memory data) +{ + // setup + var tcs = new TaskCompletionSource(); + var state = new MyCompletedCallbackState { + Tcs = tcs + }; + var pState = (IntPtr)GCHandle.Alloc(state; + + var memoryHandle = data.Pin(); + state.MemoryHandle = memoryHandle; + + // make the call + int result; + try { + result = ExportedAsyncMethod((byte*)memoryHandle.Pointer, data.Length, pState, _callbackPtr); + } catch { + ((GCHandle)pState).Free(); // cleanup since callback won't be invoked + memoryHandle.Dispose(); + throw; + } + + if (result != PENDING) + { + // Operation completed synchronously; invoke callback manually + // for result processing and cleanup. + MyCompletedCallbackImplementation(pState, result); + } + + return tcs.Task; +} + +private static void MyCompletedCallbackImplementation(IntPtr state, int result) +{ + GCHandle handle = (GCHandle)state; + var actualState = (MyCompletedCallbackState)state; + handle.Free(); + actualState.MemoryHandle.Dispose(); + + /* error checking result goes here */ + + if (error) { actualState.Tcs.SetException(...); } + else { actualState.Tcs.SetResult(result); } +} + +private static IntPtr GetCompletionCallbackPointer() +{ + OnCompletedCallback callback = MyCompletedCallbackImplementation; + GCHandle.Alloc(callback); // keep alive for lifetime of application + return Marshal.GetFunctionPointerForDelegate(callback); +} + +private class MyCompletedCallbackState +{ + public TaskCompletionSource Tcs; + public MemoryHandle MemoryHandle; +} +``` + +## See also + +- +- +- diff --git a/docs/toc.md b/docs/toc.md index 4d5d81e32a918..c2c96562d12a6 100644 --- a/docs/toc.md +++ b/docs/toc.md @@ -42,6 +42,8 @@ #### [Asynchronous Programming Patterns](standard/asynchronous-programming-patterns/) ### [Parallel Programming](standard/parallel-programming/) ### [Threading](standard/threading/) +## [Memory and span-related types](standard/memory-and-spans/index.md) +### [Memory\ and Span\ usage guidelines](standard/memory-and-spans/memory-t-usage-guidelines.md) ## [Native interoperability](standard/native-interop.md) ## [Collections and Data Structures](standard/collections/) ## [Numerics in .NET](standard/numerics.md)