Guidelines for working with Memory<T> and Span<T> #8055

rpetrusha · 2018-10-03T22:59:00Z

Guidelines for working with Memory<T> and Span<T>

Fixes #4823

@ahsonkhan @stephentoub @GrabYourPitchforks

BillWagner

This is really good @rpetrusha It's a clear explanation of these types and when to choose each.

I had a few suggestions for topics to introduce earlier that would make it more clear.

Overall, a great addition.

BillWagner · 2018-10-04T15:24:37Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+.NET Core includes a number of types that represent an arbitrary contiguous region of memory. .NET Core 2.0 introduced <xref:System.Span%601> and <xref:System.ReadOnlySpan%601>, which are lightweight memory buffers stored on the stack that can be backed by managed or unmanaged memory. Their stack-only storage makes these types unsuitable for a number of scenarios, including asynchronous method calls. .NET Core 2.1 adds a number of additional types, including <xref:System.Memory%601>, <xref:System.ReadOnlyMemory%601>, <xref:System.Buffers.IMemoryOwner%601>, and <xref:System.Buffers.MemoryPool%601>. Like <xref:System.Span%601>, <xref:Memory%601> and its related types can be backed by both managed and unmanaged memory. Unlike <xref:System.Span%601>, <xref:System.Memory%601> can be stored on the managed heap.
+
+Both <xref:System.Span%601> and <xref:System.Memory%601> are buffers of structured data that can be used in pipelines. That is, they are designed so that some or all of the data can be efficiently passed to components in the pipeline, which can process them and optionally modify the buffer. Because <xref:System.Memory%601> and its related types can be accessed  by multiple components or by multiple threads, it is important that developers follow some standard usage guidelines to produce robust code.


nit: double space "accessed by"
nit: "it is" => "it's"

BillWagner · 2018-10-04T15:26:16Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+Since buffers can be passed around between APIs, and since buffers can sometimes be accessed from multiple threads, it is important to consider lifetime management. There are three core concepts:
+
+- **Ownership**. The owner of a buffer instance is responsible for lifetime management, including destroying the buffer when it is no longer in use. All buffers have a single owner. Generally the owner is the component that created the buffer or that received the buffer from a factory. Ownership can also be transferred; **Component-** can relinquish control of the buffer to **Component-B**, at which point **Component-A** may no longer use the buffer, and **Component-B** becomes responsible for destroying the buffer when it is no longer in use.


it is => it's (2 occurrences)
"Component-" should be "Component-A"

BillWagner · 2018-10-04T15:27:25Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+- **Ownership**. The owner of a buffer instance is responsible for lifetime management, including destroying the buffer when it is no longer in use. All buffers have a single owner. Generally the owner is the component that created the buffer or that received the buffer from a factory. Ownership can also be transferred; **Component-** can relinquish control of the buffer to **Component-B**, at which point **Component-A** may no longer use the buffer, and **Component-B** becomes responsible for destroying the buffer when it is no longer in use.
+
+- **Consumption**. The consumer of a buffer instance is allowed to use the buffer instance by reading from it and possibly writing to it. Buffers can have one consumer at a time unless some external synchronization mechanism is provided. Note that the active consumer of a buffer is not necessarily the buffer's owner.


is not => isn't

Contractions don't seem to be my strong point, @BillWagner.

BillWagner · 2018-10-04T15:35:16Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+}
+```
+
+The `Main` method creates the buffer (in this case an <xref:System.Span%601> instance) and so is its owner. Therefore, `Main` is responsible for destroying the buffer when it's no longer in use. It does this by calling the buffer's <xref:System.Span%601.Clear?displayProperty=nameWithType> method. (The <xref:System.Span%601.Clear> method here actually clears the buffer's memory; the <xref:System.Span%601> structure does not actually have a method that destroys the buffer.)


does not => doesn't

BillWagner · 2018-10-04T15:45:20Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+The buffer has two consumers, `WriteInt32ToBuffer` and `DisplayBufferToConsole`. There is only one consumer at a time (first `WriteInt32ToBuffer`, then `DisplayBufferToConsole`), and neither of the consumers owns the buffer. Note also that "consumer" in this context does not imply a read-only view of the buffer; consumers can modify the buffer's contents, as `WriteInt32ToBuffer does, if given a read/write view of the buffer.
+
+The `WriteInt32ToBuffer` method has a lease on (can consume) the buffer between the start of the method call and the time the method returns. Similarly, `DisplayBufferToConsole` has a lease on the buffer while it is executing, and the lease is released when the method unwinds. (There is no API for lease management; a "lease" is simply a conceptual matter.)


nit: consider removing "simply" in the last clause.

BillWagner · 2018-10-04T17:13:24Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+void DisplayBufferToConsole(ReadOnlySpan<char> buffer);
+```
+
+The `DisplayBufferToConsole` method now works with virtually every buffer type imaginable: `T[]`, [stackalloc](~/docs/csharp/language-reference/keywords/stackalloc.md), and so on. You can even pass a <xref:System.String> directly into it!


The use of stackalloc feels wrong as written. Stackalloc needs some expression as its target, like an array or a span. Suggestion: "storage allocated with stackalloc"

BillWagner · 2018-10-04T17:51:16Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+The `WriteInt32ToBuffer` method has a lease on (can consume) the buffer between the start of the method call and the time the method returns. Similarly, `DisplayBufferToConsole` has a lease on the buffer while it is executing, and the lease is released when the method unwinds. (There is no API for lease management; a "lease" is simply a conceptual matter.)
+
+## Memory\<T> and the owner/consumer model


This section took me a couple reads for the light bulbs to come on.

Consider starting this section by introducing 2 models: A model that enables ownership transfer, and a model that assumes a single owner. Then, introduce the IMemoryOwner interface as the mechanism to implement those models. (Its absence assumes the second model).

Thanks, @BillWagner; this is a really helpful comment. I've addressed it and committed. I'll address your following comment either later this evening or tomorrow.

BillWagner · 2018-10-04T17:56:39Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+The method that initially creates the <xref:System.Memory%601> instance is the implicit owner of the buffer. Ownership cannot be transferred to any other component because there is no <xref:System.Buffers.IMemoryOwner%601> instance to facilitate the transfer. (As an alternative, you can also imagine that the runtime's garbage collector owns the buffer, and all methods just consume the buffer.)
+
+## Usage guidelines


Consider a short section on Memory vs. Span and the lifetime / storage location implications. Many of the rules below make it clear that these distinctions are important. Introducing them and explaining them here would make the guidance more clear.

mairaw · 2018-10-08T22:25:18Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+The `Main` method creates the buffer (in this case an <xref:System.Span%601> instance) and so is its owner. Therefore, `Main` is responsible for destroying the buffer when it's no longer in use. It does this by calling the buffer's <xref:System.Span%601.Clear?displayProperty=nameWithType> method. (The <xref:System.Span%601.Clear> method here actually clears the buffer's memory; the <xref:System.Span%601> structure doesn't actually have a method that destroys the buffer.)
+
+The buffer has two consumers, `WriteInt32ToBuffer` and `DisplayBufferToConsole`. There is only one consumer at a time (first `WriteInt32ToBuffer`, then `DisplayBufferToConsole`), and neither of the consumers owns the buffer. Note also that "consumer" in this context doesn't imply a read-only view of the buffer; consumers can modify the buffer's contents, as `WriteInt32ToBuffer does, if given a read/write view of the buffer.


missing closing backtick after WriteInt32ToBuffer

svick · 2018-10-10T14:04:12Z

docs/standard/memory-and-spans/index.md

+---
+# Memory- and span-related types
+
+Starting with .NET Core 2.1, .NET includes a number of interrelated types that represent a contiguous, strongly-typed region of arbitrary memory. These include:


I believe Span and Memory are available on older frameworks too, through the System.Memory NuGet package, even if they are less efficient and have less supporting APIs. Is that worth a short mention here?

svick · 2018-10-10T15:07:21Z

docs/standard/memory-and-spans/index.md

+
+Starting with .NET Core 2.1, .NET includes a number of interrelated types that represent a contiguous, strongly-typed region of arbitrary memory. These include:
+
+- <xref:System.Span%601?displayProperty=nameWithType>, a contiguous region of memory that is allocated on the stack rather than the managed heap. This allows for enhanced performance. A <xref:System.Span%601> instance can be backed by an array of type \<T>, a <xref:System.String>, a buffer allocated with [stackalloc](~/docs/csharp/language-reference/keywords/stackalloc.md), or a pointer to unmanaged memory. Because it is allocated on the stack, it has a number of restrictions. For example, a field in a class cannot be of type <xref:System.Span%601>, nor can span be used in asynchronous operations. 


System.Span, a contiguous region of memory that is allocated on the stack rather than the managed heap.

Even though this is expanded on in the rest of the paragraph, I think it's very confusing to say this, because the memory itself does not have to be allocated on the stack. Maybe say something like "a type that can be used to access a contiguous region of memory"?

This allows for enhanced performance.

As I understand it, the stack-only restriction is not about performance (a regular struct would be just as efficient). It's about safety.

an array of type <T>

Why use angle brackets here (and also below)? Shouldn't it be "an array of type T" instead?

Because it is allocated on the stack

Wouldn't it be better to say that it has to be allocated on the stack?

svick · 2018-10-10T19:28:48Z

docs/standard/memory-and-spans/index.md

+
+- <xref:System.Span%601?displayProperty=nameWithType>, a contiguous region of memory that is allocated on the stack rather than the managed heap. This allows for enhanced performance. A <xref:System.Span%601> instance can be backed by an array of type \<T>, a <xref:System.String>, a buffer allocated with [stackalloc](~/docs/csharp/language-reference/keywords/stackalloc.md), or a pointer to unmanaged memory. Because it is allocated on the stack, it has a number of restrictions. For example, a field in a class cannot be of type <xref:System.Span%601>, nor can span be used in asynchronous operations. 
+
+- <xref:System.ReadOnlySpan%601?displayProperty=nameWithtype>, an immutable version of the <xref:System.Span%601> structure.


If we're using the same terms as .Net collections, then ReadOnlySpan is not immutable, just read-only, because some other code can change the backing memory.

The same applies to ReadOnlyMemory below.

svick · 2018-10-10T19:35:52Z

docs/standard/memory-and-spans/index.md

+
+- <xref:System.ReadOnlySpan%601?displayProperty=nameWithtype>, an immutable version of the <xref:System.Span%601> structure.
+
+- <xref:System.Memory%601?displayProperty=nameWithType>, a contiguous region of memory that is allocated on the managed heap rather than the stack. A <xref:System.Memory%601> instance can be backed by an array or type \<T> and a <xref:System.String>. Because it can be stored on the managed heap, <xref:System.Memory%601> has none of the limiations of <xref:System.Span%601>.


a contiguous region of memory that is allocated on the managed heap rather than the stack

I think this is not correct: Memory itself is a struct, so it can be allocated on the stack. And the backing memory can't be stackalloced, that's true, but it also doesn't have to be allocated on the managed heap, if you use a custom MemoryManager<T>.

A Memory instance can be backed by an array or type <T> and a String.

Should this be: "A Memory instance can be backed by an array of type <T> or a String."?

It also does not mention MemoryManager<T>, is the mention in its own item below sufficient?

svick · 2018-10-10T19:43:31Z

docs/standard/memory-and-spans/index.md

+
+- <xref:System.Buffers.MemoryManager%601>, an abstract base class that can be used to replace the implementation of <xref:System.Memory%601> so that <xref:System.Memory%601> can be backed by additional types, such as safe handles. <xref:System.Buffers.MemoryManager%601> is intended for advanced scenarios.
+
+- <xref:System.ArraySegment%601>, a wrapper for a particular number of array elements starting at a particular index.


Is there ever a reason to use ArraySegment if you can use Memory or Span? If not, should that be mentioned here?

svick · 2018-10-11T12:35:10Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+This rule also applies to code that calls factory methods like <xref:System.Buffers.MemoryPool%601.Rent%2A?displayProperty=nameWithType>. The caller becomes the owner of the returned <xref:System.Buffers.IMemoryOwner%601> and is responsible for disposing of the instance when finished.
+
+**Rule #8: If you have an IMemoryOwner\<T> parameter in your API surface are, you are accepting ownership of that instance.**


Typo: extra "are"

svick · 2018-10-11T12:37:43Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+Any component that transfers ownership of the <xref:System.Buffers.IMemoryOwner%601> instance to a different component should no longer use that instance after the method call completes.
+
+> [!IMPORTANT]
+> If your constructor accepts <xref:System.Buffers.IMemoryOwner%601> as a parameter, it should implement <xref:System.IDisposable>, and your <xref:System.IDisposable.Dispose%2A> method should call <xref:System.Buffers.MemoryPool%601.Dispose%2A?displayProperty=nameWithType>.


A constructor can't implement IDisposable, so I think this should be something like:

If your constructor accepts IMemoryOwner as a parameter, the type should implement IDisposable

svick · 2018-10-11T12:44:19Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+public unsafe int ManagedWrapper(Span<byte> data)
+{
+    fixed (byte* pbData = &MemoryMarshal.GetReference(data))


In C# 7.3, you can write just fixed (byte* pbData = data).

I think it would make sense to include a note about that in this rule. (Along with a link on how to enable point releases of C#.)

svick · 2018-10-11T12:45:53Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+
+**Rule #10: If you're wrapping an asynchronous p/invoke method, your API should accept Memory\<T> as a parameter.**
+
+Since you cannot use the [`fixed`](~/docs/csharp/language-reference/keywords/fixed-statement.md) keyword across asynchronous operations, you use the <xref:System.Memory%601.Pin%2A?displayProperty=nameWithType> method to pin <xref:System.Memory%601> instances, regardless of the kind of contiguous memory the instance represents. The following example shows how to do use this API to perform an asynchronous p/invoke call.


Typo: do use

svick · 2018-10-11T12:48:56Z

docs/standard/memory-and-spans/memory-t-usage-guidelines.md

+    var state = new MyCompletedCallbackState {
+        Tcs = tcs
+    };
+    var pState = (IntPtr)GCHandle.Alloc();


I think this should be: var pState = (IntPtr)GCHandle.Alloc(state);.

BillWagner · 2018-11-14T16:14:41Z

I approved the changes, as I had no further issues @rpetrusha

I'll let you work through with @mairaw and others if more needs to be done.

Ron Petrusha added 7 commits August 10, 2018 09:25

Began new draft

3146c72

Provisional commit

a4a5aa7

Additional commit

27dc564

Additional text

4f86c3c

Additional commit

5adcd15

Additional revisions

07d8008

Added top-level article, updated TOC

d7d297f

rpetrusha self-assigned this Oct 3, 2018

rpetrusha added the 🏁 Release: .NET Core 2.1 label Oct 3, 2018

rpetrusha added this to the Sprint 142 (9/24/2018 - 10/12/2018) milestone Oct 3, 2018

BillWagner requested changes Oct 4, 2018

View reviewed changes

Ron Petrusha added 4 commits October 4, 2018 15:03

Addressed review comments

496527d

Addressed review comments

d7e36c2

Addressed review comments, fixed build errors

335d30a

Fixed more build issues

105903f

rpetrusha changed the title ~~Guidelines for working with Memory\<T> and Span\<T>~~ Guidelines for working with Memory<T> and Span<T> Oct 6, 2018

Fixed more build issues

3d02931

mairaw reviewed Oct 8, 2018

View reviewed changes

mairaw requested review from stephentoub, ahsonkhan and GrabYourPitchforks October 8, 2018 22:26

Added missing backtick

fb78e8d

svick reviewed Oct 11, 2018

View reviewed changes

Ron Petrusha added 2 commits November 8, 2018 10:45

Addressed some review comments

dcc3dd0

Additional changes

db3d167

BillWagner approved these changes Nov 14, 2018

View reviewed changes

A few additional changes

ac7925a

rpetrusha merged commit 548ee5d into dotnet:master Nov 15, 2018

rpetrusha deleted the memory branch November 15, 2018 17:01

tdykstra mentioned this pull request Nov 22, 2019

Example out of sync with text in Memory and Span usage guidelines article #15934

Closed


		.NET Core includes a number of types that represent an arbitrary contiguous region of memory. .NET Core 2.0 introduced <xref:System.Span%601> and <xref:System.ReadOnlySpan%601>, which are lightweight memory buffers stored on the stack that can be backed by managed or unmanaged memory. Their stack-only storage makes these types unsuitable for a number of scenarios, including asynchronous method calls. .NET Core 2.1 adds a number of additional types, including <xref:System.Memory%601>, <xref:System.ReadOnlyMemory%601>, <xref:System.Buffers.IMemoryOwner%601>, and <xref:System.Buffers.MemoryPool%601>. Like <xref:System.Span%601>, <xref:Memory%601> and its related types can be backed by both managed and unmanaged memory. Unlike <xref:System.Span%601>, <xref:System.Memory%601> can be stored on the managed heap.

		Both <xref:System.Span%601> and <xref:System.Memory%601> are buffers of structured data that can be used in pipelines. That is, they are designed so that some or all of the data can be efficiently passed to components in the pipeline, which can process them and optionally modify the buffer. Because <xref:System.Memory%601> and its related types can be accessed by multiple components or by multiple threads, it is important that developers follow some standard usage guidelines to produce robust code.


		Since buffers can be passed around between APIs, and since buffers can sometimes be accessed from multiple threads, it is important to consider lifetime management. There are three core concepts:

		- Ownership. The owner of a buffer instance is responsible for lifetime management, including destroying the buffer when it is no longer in use. All buffers have a single owner. Generally the owner is the component that created the buffer or that received the buffer from a factory. Ownership can also be transferred; Component- can relinquish control of the buffer to Component-B, at which point Component-A may no longer use the buffer, and Component-B becomes responsible for destroying the buffer when it is no longer in use.


		- Ownership. The owner of a buffer instance is responsible for lifetime management, including destroying the buffer when it is no longer in use. All buffers have a single owner. Generally the owner is the component that created the buffer or that received the buffer from a factory. Ownership can also be transferred; Component- can relinquish control of the buffer to Component-B, at which point Component-A may no longer use the buffer, and Component-B becomes responsible for destroying the buffer when it is no longer in use.

		- Consumption. The consumer of a buffer instance is allowed to use the buffer instance by reading from it and possibly writing to it. Buffers can have one consumer at a time unless some external synchronization mechanism is provided. Note that the active consumer of a buffer is not necessarily the buffer's owner.


		The buffer has two consumers, `WriteInt32ToBuffer` and `DisplayBufferToConsole`. There is only one consumer at a time (first `WriteInt32ToBuffer`, then `DisplayBufferToConsole`), and neither of the consumers owns the buffer. Note also that "consumer" in this context does not imply a read-only view of the buffer; consumers can modify the buffer's contents, as `WriteInt32ToBuffer does, if given a read/write view of the buffer.

		The `WriteInt32ToBuffer` method has a lease on (can consume) the buffer between the start of the method call and the time the method returns. Similarly, `DisplayBufferToConsole` has a lease on the buffer while it is executing, and the lease is released when the method unwinds. (There is no API for lease management; a "lease" is simply a conceptual matter.)


		The `WriteInt32ToBuffer` method has a lease on (can consume) the buffer between the start of the method call and the time the method returns. Similarly, `DisplayBufferToConsole` has a lease on the buffer while it is executing, and the lease is released when the method unwinds. (There is no API for lease management; a "lease" is simply a conceptual matter.)

		## Memory\<T> and the owner/consumer model


		The method that initially creates the <xref:System.Memory%601> instance is the implicit owner of the buffer. Ownership cannot be transferred to any other component because there is no <xref:System.Buffers.IMemoryOwner%601> instance to facilitate the transfer. (As an alternative, you can also imagine that the runtime's garbage collector owns the buffer, and all methods just consume the buffer.)

		## Usage guidelines


		The `Main` method creates the buffer (in this case an <xref:System.Span%601> instance) and so is its owner. Therefore, `Main` is responsible for destroying the buffer when it's no longer in use. It does this by calling the buffer's <xref:System.Span%601.Clear?displayProperty=nameWithType> method. (The <xref:System.Span%601.Clear> method here actually clears the buffer's memory; the <xref:System.Span%601> structure doesn't actually have a method that destroys the buffer.)

		The buffer has two consumers, `WriteInt32ToBuffer` and `DisplayBufferToConsole`. There is only one consumer at a time (first `WriteInt32ToBuffer`, then `DisplayBufferToConsole`), and neither of the consumers owns the buffer. Note also that "consumer" in this context doesn't imply a read-only view of the buffer; consumers can modify the buffer's contents, as `WriteInt32ToBuffer does, if given a read/write view of the buffer.


		Starting with .NET Core 2.1, .NET includes a number of interrelated types that represent a contiguous, strongly-typed region of arbitrary memory. These include:

		- <xref:System.Span%601?displayProperty=nameWithType>, a contiguous region of memory that is allocated on the stack rather than the managed heap. This allows for enhanced performance. A <xref:System.Span%601> instance can be backed by an array of type \<T>, a <xref:System.String>, a buffer allocated with [stackalloc](~/docs/csharp/language-reference/keywords/stackalloc.md), or a pointer to unmanaged memory. Because it is allocated on the stack, it has a number of restrictions. For example, a field in a class cannot be of type <xref:System.Span%601>, nor can span be used in asynchronous operations.


		- <xref:System.Span%601?displayProperty=nameWithType>, a contiguous region of memory that is allocated on the stack rather than the managed heap. This allows for enhanced performance. A <xref:System.Span%601> instance can be backed by an array of type \<T>, a <xref:System.String>, a buffer allocated with [stackalloc](~/docs/csharp/language-reference/keywords/stackalloc.md), or a pointer to unmanaged memory. Because it is allocated on the stack, it has a number of restrictions. For example, a field in a class cannot be of type <xref:System.Span%601>, nor can span be used in asynchronous operations.

		- <xref:System.ReadOnlySpan%601?displayProperty=nameWithtype>, an immutable version of the <xref:System.Span%601> structure.


		- <xref:System.ReadOnlySpan%601?displayProperty=nameWithtype>, an immutable version of the <xref:System.Span%601> structure.

		- <xref:System.Memory%601?displayProperty=nameWithType>, a contiguous region of memory that is allocated on the managed heap rather than the stack. A <xref:System.Memory%601> instance can be backed by an array or type \<T> and a <xref:System.String>. Because it can be stored on the managed heap, <xref:System.Memory%601> has none of the limiations of <xref:System.Span%601>.


		- <xref:System.Buffers.MemoryManager%601>, an abstract base class that can be used to replace the implementation of <xref:System.Memory%601> so that <xref:System.Memory%601> can be backed by additional types, such as safe handles. <xref:System.Buffers.MemoryManager%601> is intended for advanced scenarios.

		- <xref:System.ArraySegment%601>, a wrapper for a particular number of array elements starting at a particular index.


		This rule also applies to code that calls factory methods like <xref:System.Buffers.MemoryPool%601.Rent%2A?displayProperty=nameWithType>. The caller becomes the owner of the returned <xref:System.Buffers.IMemoryOwner%601> and is responsible for disposing of the instance when finished.

		Rule #8: If you have an IMemoryOwner\<T> parameter in your API surface are, you are accepting ownership of that instance.


		Rule #10: If you're wrapping an asynchronous p/invoke method, your API should accept Memory\<T> as a parameter.

		Since you cannot use the [`fixed`](~/docs/csharp/language-reference/keywords/fixed-statement.md) keyword across asynchronous operations, you use the <xref:System.Memory%601.Pin%2A?displayProperty=nameWithType> method to pin <xref:System.Memory%601> instances, regardless of the kind of contiguous memory the instance represents. The following example shows how to do use this API to perform an asynchronous p/invoke call.

Guidelines for working with Memory<T> and Span<T> #8055

Guidelines for working with Memory<T> and Span<T> #8055

Uh oh!

Conversation

rpetrusha commented Oct 3, 2018