Integration of WASI-NN #1521

tonibofarull · 2022-09-26T18:45:32Z

Initial integration of WASI-NN based on #1225.

tonibofarull · 2022-09-26T18:48:08Z

TODO

Specify limitations: only support of tflite, etc.
Improve safety checks. For example, using invalid formats.

tonibofarull · 2022-09-27T12:56:34Z

My team is planning to continuously support WASI-NN by adding support to GPU, etc.
@wenyongh how do we proceed with this? I have several questions:

Right now we haven't added this to the CI.
We are supporting only a subset of WASI-NN features.

wenyongh · 2022-09-28T01:40:35Z

My team is planning to continuously support WASI-NN by adding support to GPU, etc. @wenyongh how do we proceed with this? I have several questions:

Right now we haven't added this to the CI.

We are supporting only a subset of WASI-NN features.

@tonibofarull Sounds like it is not a short term work and may take some time to implement all the features, right? Normally we will develop such features under a development branch (like dev/socket, dev/refactor_interp_module_inst), and periodically merge code from that branch into main branch when a milestone is reached. I have created a branch dev/wasi_nn, if there are more features to implement, how about submitting the PR to that branch? And if a milestone is reached and you want to merge it into main, please notify us. (Or merge this PR to main, and the next PRs to dev/wasi_nn?)
And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on? Here are several samples:
#1336
#1384
#1468
Thanks.

wenyongh

Just comments about the scripts, license headers and so on, the C/Python source code logic isn't reviewed yet, will read it soon.

build-scripts/install_tensorflow.sh

build-scripts/runtime_lib.cmake

wenyongh · 2022-09-28T06:40:00Z

core/iwasm/libraries/wasi-nn/README.md

+1. Build the dockre image,
+
+```
+docker build -t wasi-nn -f core/iwasm/libraries/wasi-nn/test/Dockerfile .


Not sure whether the test is just a simple interface test (like unit test), or a sample? If it is a sample, please put it under <wamr_root>/samples, for example, samples/wasi-nn.

I'm testing multiple neural network models. Once it's ready for a release we can add a real sample!

core/iwasm/libraries/wasi-nn/logger.h

core/iwasm/libraries/wasi-nn/test/Dockerfile

core/iwasm/libraries/wasi-nn/test/build.sh

core/iwasm/libraries/wasi-nn/test/models/average.py

wenyongh · 2022-09-30T05:59:30Z

core/iwasm/libraries/wasi-nn/test/test_tensorflow.c

+
+    if (buffer == NULL) {
+        fputs("Memory error\n", stderr);
+        exit(2);


how about changing to return with an error id?

because here I'm doing the test of WASI-NN. As soon as there is an error I want to exit the program.

core/iwasm/libraries/wasi-nn/test/test_tensorflow.c

core/iwasm/libraries/wasi-nn/wasi_nn_tensorflow.cpp

core/iwasm/libraries/wasi-nn/wasi_nn_tensorflow.hpp

tonibofarull · 2022-09-30T16:01:57Z

Thank you, I'll take a look!

tonibofarull · 2022-10-04T14:07:09Z

And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on? Here are several samples:

Let me address your comments before we discuss this 🙇

* feat: "load module done" * fix: "pr fixes" * fix: "pr fixes"

* feat: "set input" * feat: "formatted files" * fix: "PR changes" * fix: renaming files

* feat: "set input" * feat: "formatted files" * feat: ouptut and compute functions * PR fixes * fix: PR fixes

* feat: draft opening file * feat: ongoing progress tflite * feat: working test * fix: setting wasi_nn flag * Remove legacy cmake and clean build file * max sum and average models tested and working * multiple outputs begining * refractor saving model * pr fixes * pr fixes * pr fixes * pr fixes * Fix dockerfile * Fix unnecesary modified files * Working inferencer and tests passing * Assert in all tests * Generation of models in the build * Formatting python scripts Co-authored-by: tonibofarull <[email protected]>

tonibofarull · 2022-10-10T10:41:58Z

I've fixed all the previous comments!

And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on?

I've created the issue here.

I have created a branch dev/wasi_nn, if there are more features to implement, how about submitting the PR to that branch? And if a milestone is reached and you want to merge it into main, please notify us. (Or merge this PR to main, and the next PRs to dev/wasi_nn?)

As stated in the issue, the first milestone corresponds to support TensorFlow in CPU and FP32, which are the features implemented in this PR.

Having the dev branch makes sense for having the future intermediate results!

wenyongh · 2022-10-11T01:48:14Z

I've fixed all the previous comments!

And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on?

I've created the issue here.

I have created a branch dev/wasi_nn, if there are more features to implement, how about submitting the PR to that branch? And if a milestone is reached and you want to merge it into main, please notify us. (Or merge this PR to main, and the next PRs to dev/wasi_nn?)

As stated in the issue, the first milestone corresponds to support TensorFlow in CPU and FP32, which are the features implemented in this PR.

Having the dev branch makes sense for having the future intermediate results!

Great! We will read the code again, many thanks!

core/iwasm/libraries/wasi-nn/README.md

core/iwasm/libraries/wasi-nn/test/test_tensorflow.c

xujuntwt95329

LGTM

wenyongh · 2022-10-12T04:12:24Z

@tonibofarull Many thanks for the fixing, I have merged the PR.
And I tried to compile the source code, and found that there are many compile warnings, could you help clear them?
And will you add a sample/wasi-nn in the next stage? Thanks.

[ 8%] Building C object CMakeFiles/libiwasm.dir/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/random.c.o
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:16:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:77:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
77 | attribute((import_module("wasi_nn")));
| ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:77:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:89:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
89 | attribute((import_module("wasi_nn")));
| ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:89:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:102:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
102 | attribute((import_module("wasi_nn")));
| ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:102:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:112:1: warning: ‘export_module’ attribute directive ignored [-Wattributes]
112 | attribute((import_module("wasi_nn")));
| ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:112:1: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:130:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
130 | attribute((import_module("wasi_nn")));
| ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:130:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c: In function ‘wasi_nn_load’:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:68:71: warning: declaration of ‘graph’ shadows a global declaration [-Wshadow]
68 | graph_encoding encoding, execution_target target, graph *graph)
| ~~~~~~~^~~~~
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:12:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_common.h:39:18: note: shadowed declaration is here
39 | typedef uint32_t graph;
| ^~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:95:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
95 | for (int i = 0; i < builder->size; ++i) {
| ^
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c: In function ‘wasi_nn_init_execution_context’:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:140:64: warning: declaration of ‘graph’ shadows a global declaration [-Wshadow]
140 | wasi_nn_init_execution_context(wasm_exec_env_t exec_env, graph graph,
| ~~~~~~^~~~~
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:12:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_common.h:39:18: note: shadowed declaration is here
39 | typedef uint32_t graph;
| ^~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c: In function ‘wasi_nn_set_input’:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:194:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
194 | for (int i = 0; i < dimensions.size; ++i) {
| ^
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:204:12: warning: declaration of ‘tensor’ shadows a global declaration [-Wshadow]
204 | tensor tensor = { .type = input_tensor->type,
| ^~~~~~
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:16:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:62:3: note: shadowed declaration is here
62 | } tensor;

tonibofarull · 2022-10-12T05:33:18Z

Yes, of course! Thank you very much 🥳

Initial integration of WASI-NN based on bytecodealliance#1225: - Implement the library core/iwasm/libraries/wasi-nn - Support TensorFlow, CPU, F32 at the first stage - Add cmake variable `-DWAMR_BUILD_WASI_NN` - Add test case based on Docker image and update document Refer to bytecodealliance#1573

yamt · 2025-05-29T23:54:50Z

core/iwasm/common/wasm_native.c

@@ -425,6 +428,13 @@ wasm_native_init()
        goto fail;
 #endif /* WASM_ENABLE_LIB_RATS */

+#if WASM_ENABLE_WASI_NN != 0
+    n_native_symbols = get_wasi_nn_export_apis(&native_symbols);
+    if (!wasm_native_register_natives("wasi_nn", native_symbols,


@tonibofarull where did this "wasi_nn" module name come from?
can you give me a reference to an exact version of the spec this was based on?
eg. commit id of wasi-nn repo
as far as i researched wasi-nn has always been using "wasi_ephemeral_nn".

It was an arbitrary name as I couldn't get an API list based on witx/wit.

ok. so this is not compatible with any versions of wasi-nn. right?

Compatibility is not guaranteed.

So, there will be no harm if we replace wasi-nn with wasi_ephemeral_nn in the repo?

Compatibility is not guaranteed.

well, actually incompatibility is guaranteed because of the different module name, isn't it?

tonibofarull force-pushed the WASI-NN branch from 039620a to b168c09 Compare September 26, 2022 18:56

tonibofarull marked this pull request as ready for review September 27, 2022 09:23

wenyongh reviewed Sep 28, 2022

View reviewed changes

wenyongh reviewed Sep 30, 2022

View reviewed changes

tonibofarull force-pushed the WASI-NN branch from e5f3604 to 433d51c Compare October 4, 2022 14:03

tonibofarull and others added 20 commits October 4, 2022 16:12

First initial design of wasi nn

0c5442d

feat: partial witx api of build and load

f00ef2f

feat: added tensorflow submodule

f786168

fix: witx functions fix

cb699fd

Update wasi nn definition

b1451d8

Installation script of tensorflow

3883ba6

Added export api in runtime

4ae7c97

Adding source in runtime creation

620e064

Update sample application to use load from wasi

a847d38

Update build and add testing dockerfile and run

0ef7092

Remove submodule

51d75c8

Fix dockerfile

97c9103

Adding cpp binding and proper installation

00ef633

fix: "compilation bug fixed"

bab3387

SP-193 load_function (#4)

98215a8

* feat: "load module done" * fix: "pr fixes" * fix: "pr fixes"

fix: clang format convention

3fc13d0

SP-222 set input (#5)

de1010a

* feat: "set input" * feat: "formatted files" * fix: "PR changes" * fix: renaming files

SP-222 set input (#6)

52c3310

* feat: "set input" * feat: "formatted files" * feat: ouptut and compute functions * PR fixes * fix: PR fixes

feat: security checking wasi-nn (#7)

316b372

tonibofarull added 7 commits October 4, 2022 16:12

Remove unused file

dd70322

Improve docs

6691f32

Same version

107272b

Dockerfile

a081e6b

Add logging and use deps folder

8cbb9b0

Move install into deps

13e88c3

Fix path

49cfeaf

tonibofarull force-pushed the WASI-NN branch from 433d51c to 49cfeaf Compare October 4, 2022 14:13

Improve install tensorflow

d3e45d6

tonibofarull force-pushed the WASI-NN branch from 0b9ca53 to 9cbc332 Compare October 4, 2022 15:06

Fix

040d17e

tonibofarull force-pushed the WASI-NN branch from 9cbc332 to 040d17e Compare October 4, 2022 15:25

tonibofarull added 5 commits October 4, 2022 17:58

Parameterize

80cf3b7

Update cmakelist

49a9553

Fix

9937612

Flatbuffer alpine incompatibility

cc42c57

Fix comments

d9cdd4b

wenyongh reviewed Oct 11, 2022

View reviewed changes

Fix

06e4362

tonibofarull requested a review from wenyongh October 11, 2022 15:23

xujuntwt95329 reviewed Oct 12, 2022

View reviewed changes

wenyongh merged commit e53ab91 into bytecodealliance:main Oct 12, 2022

tonibofarull deleted the WASI-NN branch December 20, 2022 11:29

yamt reviewed May 29, 2025

View reviewed changes

yamt mentioned this pull request Jun 4, 2025

wasi-nn: make wasi_ephemeral_nn default #4326

Closed

Integration of WASI-NN #1521

Integration of WASI-NN #1521

Uh oh!

Conversation

tonibofarull commented Sep 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tonibofarull commented Sep 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tonibofarull commented Sep 27, 2022

Uh oh!

wenyongh commented Sep 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wenyongh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tonibofarull commented Sep 30, 2022

Uh oh!

tonibofarull commented Oct 4, 2022

Uh oh!

tonibofarull commented Oct 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wenyongh commented Oct 11, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xujuntwt95329 left a comment

Choose a reason for hiding this comment

Uh oh!

wenyongh commented Oct 12, 2022

Uh oh!

tonibofarull commented Oct 12, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tonibofarull commented Sep 26, 2022 •

edited

Loading

tonibofarull commented Sep 26, 2022 •

edited

Loading

wenyongh commented Sep 28, 2022 •

edited

Loading

tonibofarull commented Oct 10, 2022 •

edited

Loading