define graph.devices #854
define graph.devices #854
Conversation
|
Thanks @philloooo for Chromium prototyping and making this PR! It looks good to me. As discussed in WebML WG Teleconference – 22 May 2025, Windows ML was announced as an evolution of DirectML. It would be good to know the implementability of Windows ML (ONNX Runtime). |
For the record, that has been strongly contested for the overloaded meaning of it in #815, which led to the proposal to use "gpu" or "gpu-like" labels. However, I think we could experimentally use the name "npu" here -- when we note that it might mean many different HW implementations with wildly different capabilities. However, NPU is gaining more and more the meaning of being power efficient, low-throughput, moderately low-latency acceleration. We could diversify that naming later. We should record (as developer use cases) what (programming) benefits could a client get from knowing that a node in a compiled graph is executing e.g. on NPU. As elaborated in this comment, it might reflect a past decision made on a temporary condition (i.e. there could be hidden parameters we don't record). Anyway, I think this might be a good way to gather data/experience on how typical compiled graphs will look like, which will help improving the spec on the device selection (hints, for now, but later also capabilities ), so this proposal LGTM. |
fixes #836
hi, I've implemented this API on Chromium for both CoreML backend and DML backend and Tflite backend. So I am putting up the spec PR for discussion.
This re-introduces the MLDeviceType but in the context of a compiled graph. The use case shifts from letting users specify device preference to providing information to developers on where the graph would get run.
So I'd like to hear people's opinion on the naming for 'npu' under this context.
@fdwr @huningxin
Preview | Diff