wasm2js usage or usecase unclear #2263

advdv · 2019-07-25T10:06:09Z

Dear reader,

I'm either using the wasm2js tool wrong or I don't understand what it should be use for. I'm pretty new to the wasm ecosystem so my apologies if i'm asking something crazy here.

Basically I got a main.wasm file that I run fine in browsers that support wasm directly. But I would like to research the possibility of running it in older browsers as well. The description of the wasm2js tool in the README.md seems promising "compiles WebAssembly to JS" and is also hinted at in the wasmrust documentation: https://rustwasm.github.io/wasm-bindgen/examples/wasm2js.html

But whenever I try to run the resulting .js file in chrome or node it will give me errors such as Uncaught SyntaxError: Unexpected token .. This seems to indicate it is not actually valid Js as expected by the browser or node. Is there an extra compilation step i'm missing? Some sources on the internet mention that the tool outputs "almost asm.js" but its unclear to me what I should do with this information. Issue #1929 and https://v8.dev/blog/emscripten-llvm-wasm#javascript-output seem to suggest the tool is moving away from asm.js... what gives?

I'm building the wasm2js tool from source on the latest commit. But downloading it through the releases gives me the same mystery.

Any help on how to convert/transpile a wasm file to javascript that can be run in any browser would be appreciated.

Thank you!

The text was updated successfully, but these errors were encountered:

kripken · 2019-07-25T16:17:35Z

It should work as you expect - it's possible you've found a bug. Do you have a testcase you can provide?

One thing to note is that it by default emits an ES6 module. If your VM doesn't support that, it might give a syntax error on the module notation. If you don't want ES6 notation, see the --emscripten flag (which emits it in "emscripten-friendly notation", which in particular is not ES6). Another option might be to use babel to translate ES6 to ES5.

advdv · 2019-07-26T07:12:23Z

Thank you for the quick response. What i'm specifically trying to get converted to ES5 javascript is a WASM compiled binary of a simple Go program:

package main

func main() {
	println("hello, world")
}

The process of compiling Go to WASM is described in the official wiki here but i've also uploaded the resulting binary for download here. It comes in at a hefty 1.2MB because it includes the Go runtime but otherwise works fine when loaded into a modern browser using the following html:

<html>
  <head>
    <meta charset="utf-8">
    <script src="/internal/cmd/wasm_exec.js"></script>
    <script>
      const go = new Go();
      WebAssembly.instantiateStreaming(fetch("./main.wasm"), go.importObject).then((result) => {
        go.run(result.instance);
      });
    </script>
  </head>
  <body></body>
</html>

I then converted the main.wasm file to a js file like this wasm2js main.wasm -o main.js. This produces a file like this head -n 20 main.js:

import { runtime.wasmExit } from 'go';
import { runtime.wasmWrite } from 'go';
import { runtime.nanotime } from 'go';
import { runtime.walltime } from 'go';
import { runtime.scheduleTimeoutEvent } from 'go';
import { runtime.clearTimeoutEvent } from 'go';
import { runtime.getRandomData } from 'go';
import { getTempRet0 } from 'env';


  var scratchBuffer = new ArrayBuffer(8);
  var i32ScratchView = new Int32Array(scratchBuffer);
  var f32ScratchView = new Float32Array(scratchBuffer);
  var f64ScratchView = new Float64Array(scratchBuffer);
  
  function legalimport$wasm2js_scratch_load_i64() {
    if (typeof setTempRet0 === 'function') setTempRet0(i32ScratchView[1]);
    return i32ScratchView[0];
  }

I then first tried to load it in chrome (as a start) by changing the html to:

<html>
  <head>
    <meta charset="utf-8">
    <script src="/internal/cmd/wasm_exec.js"></script>
    <script src="/_examples/00-wasm2js/main.js"></script>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

This would give me the following error: Uncaught SyntaxError: Unexpected token { If I change the script tag to include the type="module" to ask chrome to load it as a ES6 module the error becomes Uncaught SyntaxError: Unexpected token ..

If I remove the import lines that include the dot the javascript will fail with the same error Uncaught SyntaxError: Unexpected token . but now at line 415004 where these imports are used.

Finally I ran wasm2js with the --emscripten flag (wasm2js main.wasm --emscripten -o main.js) and upon loading that into chrome it gives no error (Yay!) but doesn't run the program either. Or at least, doesn't show "hello, world" in the console as i would expect.

What I think is happening is that WASM output is looking for functions that are defined in the helper javascript script called 'wasm_exec.js' (src) and tries to 'import' them in a way that is not supported?

Maybe it possible to merge the wasm_exec.js with the output of wasm2js? I have some time to get this to work if you think it's feasible, what do you think? It would be really exciting to for the Go language and WASM as an intermediate format if we could get it to run on old browsers :)

kripken · 2019-07-26T17:06:08Z

Yes, I agree it's exciting to get Go working in browsers (old or new) :)

As an ES6 module, the full error looks like

$ node  --experimental-modules b.mjs
(node:234207) ExperimentalWarning: The ESM module loader is experimental.
a.mjs:1
import { runtime.wasmExit } from 'go';
                ^

SyntaxError: Unexpected token .
    at Loader.moduleStrategy (internal/modules/esm/translators.js:51:18)

The Go wasm file has imports with "." in the name, and those aren't valid JS identifiers. So the naive translation to JS doesn't work - those identifiers would need to be escaped, or the wasm not use such an identifier ("_" would be ok instead).

Fixing that is one option. Another option is to use the non-ES6 output mode, that I mentioned earlier. Something like

bin/wasm2js --emscripten ~/Downloads/temp/main.wasm  -o a.js -O

Then the JS provides an function instantiate(asmLibraryArg, wasmMemory, wasmTable) function. That assumes you pass in the memory and table, which isn't what that Go module expects, though. So some work would be necessary there too. But should be pretty easy.

I'd be happy to help out here, let me know.

…ized. fixes #2263. Adds a complete example of usage in the readme as well

kripken · 2019-07-27T20:30:59Z

Looks like the first option, properly escaping such imports, is easy to do, I opened #2267 with a fix, and also some docs that include a full example of using wasm2js output.

Ok, with that, I can run

$ bin/wasm2js -O ~/Downloads/temp/main.wasm -o hello_go.js

Then with these files it will load:

<body>
<script type="module" src="hello_go.js"></script>
<script type="module">
  import { run } from './hello_go.js';
  run();
</script>
</body>

// go.js
export var runtime_wasmExit = function() { console.log('exit') };
export var runtime_wasmWrite = function() { console.log('wasmWrite') };
export var runtime_nanotime = function() { console.log('nanotime') };
export var runtime_walltime = function() { console.log('walltime') };
export var runtime_scheduleTimeoutEvent = function() { console.log('schedule') };
export var runtime_clearTimeoutEvent = function() { console.log('clear') };
export var runtime_getRandomData = function() { console.log('random') };

// env.js
export var getTempRet0 = function() { return 0 };

(env is a module defined by wasm2js; here getTempRet0 isused to return the high 64 bits of a 64 bit value - apparently some go function returns an i64?)

I did need to edit hello_go.js a little, changing ./go.js instead of go, and likewise for env. I'm not sure if there's another solution, I don't know much about ES6 module loading.

With that, it seems to load and run! It calls those exports at least. (Those would need to be hooked up to wasm_exec.js that you mentioned, I guess; maybe rollup or such could combine it with the main go script?)

advdv · 2019-07-28T07:37:26Z

Thats super cool! The README improvements definitely make it more clear and I got the setup you described to work. I'll focus on getting the go.js exports hooked up and report back. From there ill experiment with some more complex programs to see how far it goes. My initial question for this issue has been answered so you can close it if you want. I can open a separate issue to discuss further developments around getting Go programs to run

kripken · 2019-07-28T17:56:52Z

@advanderveer great!

Ok, let's close this, but please keep us updated on progress with this - I'd love to see it working! Docs or a blogpost about "how to use wasm2js with Go" would be cool eventually, happy to collaborate on that if there's interest.

This fixes names that would be invalid in JS, like a.b. Turns out the Go compiler emits wasm with such imports. Also add some docs on how to use wasm2js. Fixes #2263

advdv · 2020-08-24T09:09:34Z

@kripken More than a year later I've finally came around to working this out a bit further! A proof-of-concept with a demo is available over here for anyone to check out: https://github.com/advanderveer/goes.

I don't have a personal blog to post this somewhere but i'm happy to take feedback here on in the issues of the repo. Let me know what you think!

kripken · 2020-08-25T17:23:31Z

@advanderveer Nice to see this work!

Looking at it, some thoughts:

I think the term "emscripten" is confusing, since the emscripten toolchain is never run (or am I missing some connection)? Maybe "wasm2js" for the JS mode?
About code size, were the builds all optimized? I can see main.wasm shrinks by 5% with wasm-opt -O, so maybe not. I can't check the wasm2js output directly, but did you add -O when running wasm2js which can help a lot? Aside from that, running a JS minifier on hello_go_after.min.js would help a lot too. Just a quick test of removing whitespace shrinks it by 25%, and using a proper minifier like closure or terser etc. could do a lot more.

kripken added a commit that referenced this issue Jul 27, 2019

wasm2js: Mangle import names properly, so illegal basenames are legal…

db07070

…ized. fixes #2263. Adds a complete example of usage in the readme as well

kripken mentioned this issue Jul 27, 2019

wasm2js: Mangle import names for JS #2267

Merged

kripken closed this as completed in #2267 Jul 28, 2019

kripken added a commit that referenced this issue Jul 28, 2019

wasm2js: Mangle import names for JS (#2267)

6ac5fa7

This fixes names that would be invalid in JS, like a.b. Turns out the Go compiler emits wasm with such imports. Also add some docs on how to use wasm2js. Fixes #2263

advdv mentioned this issue Oct 13, 2020

wasm: browser compatibility policy golang/go#28360

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wasm2js usage or usecase unclear #2263

wasm2js usage or usecase unclear #2263

advdv commented Jul 25, 2019

kripken commented Jul 25, 2019

advdv commented Jul 26, 2019

kripken commented Jul 26, 2019

kripken commented Jul 27, 2019

advdv commented Jul 28, 2019

kripken commented Jul 28, 2019

advdv commented Aug 24, 2020

kripken commented Aug 25, 2020

wasm2js usage or usecase unclear #2263

wasm2js usage or usecase unclear #2263

Comments

advdv commented Jul 25, 2019

kripken commented Jul 25, 2019

advdv commented Jul 26, 2019

kripken commented Jul 26, 2019

kripken commented Jul 27, 2019

advdv commented Jul 28, 2019

kripken commented Jul 28, 2019

advdv commented Aug 24, 2020

kripken commented Aug 25, 2020