Add WASM backend #1497

nebrelbug · 2019-04-05T02:51:13Z

TensorFlow.js version

1.0.4

Browser version

Chrome 73

Describe the problem or feature request

Has there been any further discussion about using WASM to optimize performance? I noticed there was discussion in #36, but it was decided to wait and eventually tfjs-node was created.

Possible Benefits of WASM

Run in any WASM environment
Maybe higher speed than WebGL if users have slow graphics?

The text was updated successfully, but these errors were encountered:

dsmilkov · 2019-04-05T17:52:12Z

Hi, thanks for filing this issue. A WASM backend is on our roadmap for this quarter. We can use this issue for updates.

no-1ne · 2019-04-07T00:22:18Z

Folks at intel have been making webml/webnn and recently made some amazing strides of hitting 31fps running coco SSD models.

intel/webml-polyfill#650 (comment)

They are implementing it natively mimicking the android NN API and for the changes to reach end user, they have to be merged into chromium source code.

Where as WASM backend would be awesome and I hope it can hit as much performance as they are able to

no-1ne · 2019-05-27T00:37:50Z

Hi folks update on this, curious to see the performance compared to tflite and tensorflow

dsmilkov · 2019-05-29T21:19:48Z

Thanks for the reference. We are closely following the amazing work that Intel has done, led by @huningxin. Our wasm backend will most likely not be able to reach that level of performance since we won't be able to access hardware acceleration beyond SIMD 128bit, but it will provide other benefits, such as:

acceleration for lower-end mobile devices that don't have webgl float support.
acceleration on higher-end devices through SIMD and threading.
consistent performance on cpu, agnostic of the JIT compilation.
open an avenue for adding iterative linear algebra primitives such as SVD, eigenvectors, matrix inversion, that cannot be done efficiently on the GPU.

We will start developing our WASM backend in the open, on GitHub, very soon. Stay tuned!

huningxin · 2019-05-30T05:42:39Z

Thanks for the update @dsmilkov . As you know, webml-polyfill currently reuses tfjs-core WebGL kernels for GPU implementation and compiles tf-lite kernels to WASM for CPU implementation. It's a great news that tfjs will have a WASM backend soon. It would simplify the webml-polyfill and benefit the webnn spec development. Highly looking forward to that.

chirva-ivan · 2019-08-13T09:34:34Z

Hey, guys. Are there any updates on this issue?

dsmilkov · 2019-08-13T13:55:59Z

We just started by adding the framework (skeleton) code around it: tensorflow/tfjs-core#1863 We will be slowly ramping up on this.

camsjams · 2019-09-05T18:25:20Z

While fully understanding that this a new and still experimental backend, are you planning on publishing the alpha/beta builds to npm?

dsmilkov · 2019-09-06T17:32:33Z

We are still very early in the process so publishing something to npm is not useful at this time. I'll ping this issue when we have more updates.

bhack · 2019-09-20T13:48:20Z

For out of browser WASI I just opened tensorflow/mlir#140 in the MLIR repo.

rthadur · 2019-10-15T01:19:56Z

This has been done and should be available in latest releases.Thank you.

tiziano88 · 2019-10-15T10:17:12Z

Great news! Which release has now Wasm support? And are there any instructions about it? Looking forward to trying it out.

nsthorat · 2019-10-15T11:47:12Z

This is actually still in progress we’ll update this when we’re ready to release :)

Annaero · 2019-11-08T14:59:44Z

This is actually still in progress we’ll update this when we’re ready to release :)

Hello, do you have any updates?

dsmilkov · 2019-12-05T20:28:24Z

Hi all,

We are excited to share that we just released our first alpha release of the WASM backend.
See the README.md in the tfjs-backend-wasm folder for how to use it. At this point we have about ~30 kernels - see all_kernels.ts for the list.

We would love to get your feedback, testing, and help as we are polishing towards a general release!

Thanks!

dsmilkov · 2019-12-05T20:29:06Z

Closing this issue since it's a general issue tracking our WASM release. Feel free to file separate issues if you run into problems.

whatevergeek · 2019-12-05T20:57:46Z

Thanks @dsmilkov
This is great news.

bhack · 2019-12-05T20:57:59Z

@dsmilkov Are you involved as team in https://bytecodealliance.org/?

camsjams · 2019-12-05T23:09:16Z

There appears to be a directory called wasm-out missing in the published npm version, I get an error:

Module not found: Error: Can't resolve '../wasm-out/tfjs-backend-wasm' in '/var/www/node_modules/@tensorflow/tfjs-backend-wasm/dist'

Here is a screenshot of installed items:

BUG This fixes #1497 (comment)

dsmilkov · 2019-12-05T23:29:15Z

Thanks for finding that out. We just released 1.4.0-alpha2 with the fix. Let us know how it goes.

camsjams · 2019-12-06T00:24:57Z

It looks to be working so far in the sense that our application hasn't crashed or thrown errors. I will you know how it goes 👍

no-1ne · 2019-12-06T00:40:40Z

Great new congrats on alpha,

Cameron, pls do share the performance improvements you notice during your experiments with wasm backend.

chirva-ivan · 2019-12-06T12:37:49Z

Did everything following the example from README and got an error. Do I need some special build of tfjs-core? Any Ideas?

nsthorat · 2019-12-06T14:46:18Z

@chirva-ivan which browser are you using? If you open the initialization error what does it say?

chirva-ivan · 2019-12-06T15:27:48Z

@nsthorat using Chrome 78

nsthorat · 2019-12-06T15:37:58Z

@chirva-ivan is it possible for you to stand up a demo for us to look at?

dsmilkov · 2019-12-06T15:38:42Z

Are you using Parcel? Can you share a small project/build script that we can reproduce? Thanks!

yashints · 2019-12-09T00:43:23Z

@dsmilkov Here is a codesandbox with above error:

https://codesandbox.io/s/naughty-montalcini-s0gjk

…rflow#1497

Make the WASM backend work in node. Also: - Enable node tests in CI - Reduce log outputs when running tests in core. - Fix #1497 (comment) (tfjs_backend_wasm.default is not a fuction) - Remove pre-install script which causes yarn of a third-party project that uses the wasm backend to spin indefinitely. FEATURE

yashints · 2019-12-10T23:34:30Z

@dsmilkov I am still getting the error in the browser, was the fix just for node?

dsmilkov · 2019-12-14T20:43:28Z

I just sent #2559 for review where I document how to use the WASM backend with bundlers and also added starter projects for webpack and parcel. See update readme from the PR.

Note also that you have to depend on the latest alpha 1.4.0-alpha3, and potentially we might need to release alpha4 since we made some updates to the browser field in package.json.

rthadur added comp:node.js type:feature New feature or request labels Apr 5, 2019

rthadur assigned nkreeger and dsmilkov Apr 5, 2019

dsmilkov added comp:wasm and removed comp:node.js labels Apr 5, 2019

dsmilkov changed the title ~~Improve Performance with WASM~~ Add WASM backend Apr 5, 2019

no-1ne mentioned this issue Apr 7, 2019

Tf.js acceleration on APU, NPU #1496

Closed

dsmilkov assigned nsthorat and unassigned nkreeger Sep 6, 2019

bhack mentioned this issue Sep 30, 2019

WASI tensorflow/mlir#140

Closed

rthadur closed this as completed Oct 15, 2019

nsthorat reopened this Oct 15, 2019

dsmilkov closed this as completed Dec 5, 2019

dsmilkov mentioned this issue Dec 5, 2019

[wasm] Do not ignore wasm-out since it's required directly #2500

Merged

dsmilkov added a commit that referenced this issue Dec 5, 2019

Do not ignore wasm-out since it's required directly (#2500)

ce5efd5

BUG This fixes #1497 (comment)

sapphi-red added a commit to sapphi-red/tfjs that referenced this issue Dec 9, 2019

[wasm] Fix 'tfjs_backend_wasm_1.default is not a function' refs tenso…

60167b0

…rflow#1497

sapphi-red mentioned this issue Dec 9, 2019

[wasm] Fix 'tfjs_backend_wasm_1.default is not a function' refs #1497 #2519

Closed

dsmilkov mentioned this issue Dec 9, 2019

[wasm] Enable execution in node #2520

Merged

Add WASM backend #1497

Add WASM backend #1497

Comments

nebrelbug commented Apr 5, 2019

TensorFlow.js version

Browser version

Describe the problem or feature request

dsmilkov commented Apr 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

no-1ne commented Apr 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

no-1ne commented May 27, 2019

Uh oh!

dsmilkov commented May 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

huningxin commented May 30, 2019

Uh oh!

chirva-ivan commented Aug 13, 2019

Uh oh!

dsmilkov commented Aug 13, 2019

Uh oh!

camsjams commented Sep 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dsmilkov commented Sep 6, 2019

Uh oh!

bhack commented Sep 20, 2019

Uh oh!

rthadur commented Oct 15, 2019

Uh oh!

tiziano88 commented Oct 15, 2019

Uh oh!

nsthorat commented Oct 15, 2019

Uh oh!

Annaero commented Nov 8, 2019

Uh oh!

dsmilkov commented Dec 5, 2019

Uh oh!

dsmilkov commented Dec 5, 2019

Uh oh!

whatevergeek commented Dec 5, 2019

Uh oh!

bhack commented Dec 5, 2019

Uh oh!

camsjams commented Dec 5, 2019

Uh oh!

dsmilkov commented Dec 5, 2019

Uh oh!

camsjams commented Dec 6, 2019

Uh oh!

no-1ne commented Dec 6, 2019

Uh oh!

chirva-ivan commented Dec 6, 2019

Uh oh!

nsthorat commented Dec 6, 2019

Uh oh!

chirva-ivan commented Dec 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nsthorat commented Dec 6, 2019

Uh oh!

dsmilkov commented Dec 6, 2019

Uh oh!

yashints commented Dec 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yashints commented Dec 10, 2019

Uh oh!

dsmilkov commented Dec 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dsmilkov commented Apr 5, 2019 •

edited

Loading

no-1ne commented Apr 7, 2019 •

edited

Loading

dsmilkov commented May 29, 2019 •

edited

Loading

camsjams commented Sep 5, 2019 •

edited

Loading

chirva-ivan commented Dec 6, 2019 •

edited

Loading

yashints commented Dec 9, 2019 •

edited

Loading

dsmilkov commented Dec 14, 2019 •

edited

Loading