From 7d240c97a66809fe4edd10e239a19c7001e5f126 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Tue, 8 Sep 2020 23:25:27 +0100 Subject: [PATCH 1/6] Add data interchange section, using DLPack This is incomplete, but there seems to be agreement that DLPack is the best candidate for an interchange mechanism. So let's document that, and improve the content as we go. See https://github.com/data-apis/consortium-feedback/issues/1 for the RFC about standardizing on DLPack. Design discussion should be picked up there. --- spec/assumptions.md | 2 ++ spec/design_topics/data_interchange.md | 40 ++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/spec/assumptions.md b/spec/assumptions.md index 2e5f6f07d..20267d2db 100644 --- a/spec/assumptions.md +++ b/spec/assumptions.md @@ -18,6 +18,8 @@ made in the API standard. For example, JIT compilers may require output dtypes of functions to be predictable from input dtypes only rather than input values. +.. _assumptions-dependencies: + ## Dependencies The only dependency that's assumed in this standard is that on Python itself. diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index bf21592ba..ef2476e76 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -1 +1,41 @@ # Data interchange mechanisms + +This section discusses the mechanism to convert one type of array into another. +As discussed in the :ref:`assumptions-dependencies ` section, +_functions_ provided by an array library are not expected to operate on +_array types_ implemented by another library. Instead, the array can be +converted to a "native" array type. + +The interchange mechanism must offer the following: + +1. Data access via a protocol that describes the memory layout of the array + in an implementation-independent manner. + _Rationale: any number of libraries must be able to exchange data, and no + particular package must be needed to do so._ +2. Support for all dtypes in this API standard (see :ref:`data-types`). +3. Device support. It must be possible to determine on what device the array + that is to be converted lives. + _Rationale: there are CPU-only, GPU-only, and multi-device array types; + it's best to support these with a single protocol (with separate + per-device protocols it's hard to figure out unambiguous rules for which + protocol gets used, and the situation will get more complex over time + as TPU's and other accelerators become more widely available)._ +4. Moving data from one device to another must raise by default, and be + explicitly enabled by the user. + _Rationale: data transfer is typically very expensive, and hence must not + happen silently._ +5. Zero-copy semantics where possible, making a copy only if needed (e.g. + when data is not contiguous in memory). + _Rationale: performance._ +6. A stable C ABI. + _Rationale: all prominent existing array libraries are implemented in + C/C++, and are released independently from each other. Hence a stable C + ABI is required for packages to work well together._ + +The best candidate for this protocol is DLPack. See the +[RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1) +for details. + +TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). + +TODO: specify the expected behaviour with copy/view/move/shared-memory semantics in detail. From e65de2799df434e870a8efdf6d61a4778ef9ef3f Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Sat, 12 Sep 2020 17:08:48 +0100 Subject: [PATCH 2/6] Update the recommendation to raise on device transfers. --- spec/design_topics/data_interchange.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index ef2476e76..63804f28d 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -20,14 +20,10 @@ The interchange mechanism must offer the following: per-device protocols it's hard to figure out unambiguous rules for which protocol gets used, and the situation will get more complex over time as TPU's and other accelerators become more widely available)._ -4. Moving data from one device to another must raise by default, and be - explicitly enabled by the user. - _Rationale: data transfer is typically very expensive, and hence must not - happen silently._ -5. Zero-copy semantics where possible, making a copy only if needed (e.g. +4. Zero-copy semantics where possible, making a copy only if needed (e.g. when data is not contiguous in memory). _Rationale: performance._ -6. A stable C ABI. +5. A stable C ABI. _Rationale: all prominent existing array libraries are implemented in C/C++, and are released independently from each other. Hence a stable C ABI is required for packages to work well together._ @@ -39,3 +35,15 @@ for details. TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). TODO: specify the expected behaviour with copy/view/move/shared-memory semantics in detail. + + +.. note:: + + If an array that is accessed via the interchange protocol lives on a + device that the requesting library does not support, one of two things + must happen: moving data to another device, or raising an exception. + Device transfers are typically expensive, hence doing that silently can + lead to hard to detect performance issues. Hence it is + recommended to raise an exception, and let the user explicitly enable + device transfers via, e.g., a `force=False` keyword that they can set to + `True`. \ No newline at end of file From c92d27eef4ad654952f7adef44987f62094a5d88 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Sat, 12 Sep 2020 23:58:35 +0100 Subject: [PATCH 3/6] Add note on alternatives to DLPack (buffer protocol, `__cuda_array_interface__`) --- spec/design_topics/data_interchange.md | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index 63804f28d..40ae1c143 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -23,7 +23,7 @@ The interchange mechanism must offer the following: 4. Zero-copy semantics where possible, making a copy only if needed (e.g. when data is not contiguous in memory). _Rationale: performance._ -5. A stable C ABI. +5. A Python-side and a C-side interface, the latter with a stable C ABI. _Rationale: all prominent existing array libraries are implemented in C/C++, and are released independently from each other. Hence a stable C ABI is required for packages to work well together._ @@ -32,6 +32,26 @@ The best candidate for this protocol is DLPack. See the [RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1) for details. +.. note:: + + The main alternatives to DLPack are device-specific methods: + + - The [buffer protocol](https://docs.python.org/dev/c-api/buffer.html) on CPU + - `__cuda_array_interface__` for CUDA, specified in the Numba documentation + [here](https://numba.pydata.org/numba-doc/0.43.0/cuda/cuda_array_interface.html) + (Python-side only at the moment) + + An issue with device-specific protocols are: if two libraries both + support multiple device types, in which order should the protocols be + tried? A growth in the number of protocols to support each time a new + device gets supported by array libraries (e.g. TPUs, AMD GPUs, emerging + hardware accelerators) also seems undesirable. + + In addition to the above argument, it is also clear from adoption patterns that DLPack + has the widest support. The buffer protocol, despite being a lot older and standardized as part of Python itself via PEP 3118, hardly has any support from array libraries. + CPU interoperability is mostly dealt with via the NumPy-specific `__array__`. + + TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). TODO: specify the expected behaviour with copy/view/move/shared-memory semantics in detail. @@ -46,4 +66,5 @@ TODO: specify the expected behaviour with copy/view/move/shared-memory semantics lead to hard to detect performance issues. Hence it is recommended to raise an exception, and let the user explicitly enable device transfers via, e.g., a `force=False` keyword that they can set to - `True`. \ No newline at end of file + `True`. + From ea860430e320eab1345e3e920df99e5368e63d73 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Sun, 13 Sep 2020 20:11:32 +0100 Subject: [PATCH 4/6] Another small addition to clarify what `__array__` does --- spec/design_topics/data_interchange.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index 40ae1c143..0d9f60212 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -27,6 +27,7 @@ The interchange mechanism must offer the following: _Rationale: all prominent existing array libraries are implemented in C/C++, and are released independently from each other. Hence a stable C ABI is required for packages to work well together._ +6. An The best candidate for this protocol is DLPack. See the [RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1) @@ -49,7 +50,9 @@ for details. In addition to the above argument, it is also clear from adoption patterns that DLPack has the widest support. The buffer protocol, despite being a lot older and standardized as part of Python itself via PEP 3118, hardly has any support from array libraries. - CPU interoperability is mostly dealt with via the NumPy-specific `__array__`. + CPU interoperability is mostly dealt with via the NumPy-specific `__array__` (which, + when called, means the object it is attached to must return a `numpy.ndarray` containing + the data the object holds). TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). From 9b4823f175842eb4c38aef12023ace490e56c6e6 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Sun, 13 Sep 2020 20:14:51 +0100 Subject: [PATCH 5/6] Reflow text in data interchange file --- spec/design_topics/data_interchange.md | 30 +++++++++++++------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index 0d9f60212..d9334fd68 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -35,7 +35,7 @@ for details. .. note:: - The main alternatives to DLPack are device-specific methods: + The main alternatives to DLPack are device-specific methods: - The [buffer protocol](https://docs.python.org/dev/c-api/buffer.html) on CPU - `__cuda_array_interface__` for CUDA, specified in the Numba documentation @@ -48,11 +48,13 @@ for details. device gets supported by array libraries (e.g. TPUs, AMD GPUs, emerging hardware accelerators) also seems undesirable. - In addition to the above argument, it is also clear from adoption patterns that DLPack - has the widest support. The buffer protocol, despite being a lot older and standardized as part of Python itself via PEP 3118, hardly has any support from array libraries. - CPU interoperability is mostly dealt with via the NumPy-specific `__array__` (which, - when called, means the object it is attached to must return a `numpy.ndarray` containing - the data the object holds). + In addition to the above argument, it is also clear from adoption + patterns that DLPack has the widest support. The buffer protocol, despite + being a lot older and standardized as part of Python itself via PEP 3118, + hardly has any support from array libraries. CPU interoperability is + mostly dealt with via the NumPy-specific `__array__` (which, when called, + means the object it is attached to must return a `numpy.ndarray` + containing the data the object holds). TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). @@ -62,12 +64,10 @@ TODO: specify the expected behaviour with copy/view/move/shared-memory semantics .. note:: - If an array that is accessed via the interchange protocol lives on a - device that the requesting library does not support, one of two things - must happen: moving data to another device, or raising an exception. - Device transfers are typically expensive, hence doing that silently can - lead to hard to detect performance issues. Hence it is - recommended to raise an exception, and let the user explicitly enable - device transfers via, e.g., a `force=False` keyword that they can set to - `True`. - + If an array that is accessed via the interchange protocol lives on a + device that the requesting library does not support, one of two things + must happen: moving data to another device, or raising an exception. + Device transfers are typically expensive, hence doing that silently can + lead to hard to detect performance issues. Hence it is recommended to + raise an exception, and let the user explicitly enable device transfers + via, e.g., a `force=False` keyword that they can set to `True`. From 7f69239ef791c56a8a2d9e14d946fd058d0811b7 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Sun, 13 Sep 2020 21:26:10 +0100 Subject: [PATCH 6/6] minor cleanup --- spec/design_topics/data_interchange.md | 1 - 1 file changed, 1 deletion(-) diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index d9334fd68..b8ded4eaf 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -27,7 +27,6 @@ The interchange mechanism must offer the following: _Rationale: all prominent existing array libraries are implemented in C/C++, and are released independently from each other. Hence a stable C ABI is required for packages to work well together._ -6. An The best candidate for this protocol is DLPack. See the [RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1)