Web Neural Network API as-is in WASI?

[anssiko]

There's an interesting proposal over at WebAssembly System Interface repo WebAssembly/WASI#59 we should discuss.

Quoting OP:

It might sound too early to be talking about such fancy high level APIs but I think a WASI runtime with hardware accelerated neural networks and related algorithms would have a huge immediate market since it's such a hot topic. Probably this is another of those times when copying the web or following it closely is a good thing, the Web Neural Network API(examples) is being defined to provide this APIs to JS developers but might work even better with WASI.

Follow-ups over at the WASI repo or here. Consider this an early exploration. Currently this CG is not chartered to consider WASI but that could change in the future given adequate support for such a direction.

Cc @huningxin and @zolkis

[huningxin]

There is a proposal for WASI-nn, a machine learning API for WASI: WebAssembly/WASI#272.

[anssiko]

FYI, added to the WebML CG Teleconference – 14 May 2020 agenda for discussion: https://github.com/webmachinelearning/meetings/blob/master/telcons/2020-05-14-agenda.md

[anssiko]

I had a good chat with @abrown working on https://github.com/WebAssembly/wasi-nn and he had questions regarding the WebNN API and its coupling with browser-specific primitives (e.g. Navigator object) and some IDL types.

@abrown this issue is a good one for fielding those questions.

(There's also another issue discussing WebNN API in a non-browser runtimes for JavaScript: #142)

[abrown]

Ideally, Wasm users would not have to worry about where their modules are going to be run (e.g. in a browser, in a standalone VM with WASI)--you can see these types of concerns in issues like WebAssembly/WASI#401. The issue is that web APIs and WASI are on separate tracks and run by separate groups of people so there is a possibility that the specifications diverge or are hard/impossible to run on the other side (e.g. the navigator object, browser-specific WebIDL types could be tricky to shim for WASI). My understanding from our talk is that WebNN should have few of these small shim problems, but I think we should make a concerted effort to try to reduce any big differences in the specifications. I think the big difference right now is that WebNN allows users to "build" up the inference graph whereas wasi-nn allows users to "load" the inference graph from an encoding. Any thoughts about that? I guess, in summary, I think we should do whatever is possible to minimize thrash for ML-specific users.

[pyu10055]

@abrown I think one of the reason WebNN did not go with loader API is that, the ML model format is not mature. IMO ML model format is quite different from image and video format, the content of the format is keep evolving (i.e. continuously adding new kernels). To ensure models running successfully in different platforms, you would need a high level framework to polyfill the missing ops.

Can you explain the difference between WASI and browser WASM? If WebNN is implemented in C++ can WASI and WASM take advantage of the same platform implementation with a JS binding?

[abrown]

ML model format is not mature [...] the content of the format is keep evolving

I agree and would add that the set of ML operators is also evolving (or so I hear). This is why we went down the "loader" path: the wasi-nn spec does not have to change when the formats change (avoiding thrash), whereas the WebNN spec must evolve alongside (i.e. try to keep up) with newly-added ML operators. Is this intuition correct?

Can you explain the difference between WASI and browser WASM?

A simple example: imagine a program that wants to "print stuff." A WASI-enabled Wasm module would have an import for fd_write and the standalone VM (e.g. NodeJS, Wasmtime) would provide the implementation to do the actual printing. In a browser, the import might be called something else; or, it might shim fd_write and provide printing using console.log.

This works because the WASI API and Web API for "printing" is roughly the same, if you squint. But if the APIs are significantly different (e.g. the "builder" vs "loader" dilemma), then the browser would have a tough time polyfilling wasi-nn's load operation and a standalone VM would have a tough time polyfilling WebNN's ML operators (e.g. gemm). If they converge, then things are easier; we could perhaps use the same C++ implementation for both like you mention.