The Generated wasm in GenJs project is different than existing output.wasm

[msmshazan]

I think system.bc is not getting linked properly
The existing output.wasm is 20kb
The generated output.wasm is 2kb

[schellingb]

The output I get when running the Makefile is around 20 kb (19882 bytes)

When running make.exe -B -n (to see what commands would be executed) this is what I see:

Copying loader.js to Release-wasm/loader.js ...
copy "loader.js" "Release-wasm/loader.js" >nul 2>nul
copy "loader.html" "Release-wasm/loader.html" >nul 2>nul
main.c
"D:/dev/wasm/llvm/clang" -DNDEBUG -Ofast -std=c99  -target wasm32 -nostdinc -D__EMSCRIPTEN__ -D_LIBCPP_ABI_VERSION=2 -fvisibility=hidden -fno-builtin -fno-exceptions -fno-threadsafe-statics -isystemD:/dev/wasm/system/include/libcxx -isystemD:/dev/wasm/system/include/compat -isystemD:/dev/wasm/system/include -isystemD:/dev/wasm/system/include/libc -isystemD:/dev/wasm/system/lib/libc/musl/arch/emscripten -MMD -MP -MF Release-wasm/main.d -o Release-wasm/main.o -c main.c
Linking Release-wasm/output.wasm ...
"D:/dev/wasm/llvm/wasm-ld" -strip-all -gc-sections -no-entry -allow-undefined -import-memory -export=__wasm_call_ctors -export=malloc -export=free -export=main  -o Release-wasm/output.wasm Release-wasm/main.o System.bc
"D:/dev/wasm/wasm-opt.exe" --legalize-js-interface -O3 Release-wasm/output.wasm -o Release-wasm/output.wasm

It properly appends System.bc to the command line of wasm-ld to link against it. And when I open loader.html in a browser it runs correctly, too.

Did you have System.bc created when running make for the first time?

[msmshazan]

Yes I have System.bc created which is about 733kb

[msmshazan]

This is the output of running make.exe -B -n

Linking Release-wasm/output.wasm ...
PS E:\shazan\website> make.exe -B -n
Copying loader.js to Release-wasm/loader.js ...
copy "loader.js" "Release-wasm/loader.js" >nul 2>nul
copy "loader.html" "Release-wasm/loader.html" >nul 2>nul
main.c
"E:/shazan/emsdk/upstream/bin/clang" -DNDEBUG -Ofast -std=c99  -target wasm32 -nostdinc -D__EMSCRIPTEN__ -D_LIBCPP_ABI_VERSION=2 -fvisibility=hidden -fno-builtin -fno-exceptions -fno-threadsafe-statics -isystemE:/shazan/emsdk/upstream/emscripten/system/include/libcxx -isystemE:/shazan/emsdk/upstream/emscripten/system/include/compat -isystemE:/shazan/emsdk/upstream/emscripten/system/include -isystemE:/shazan/emsdk/upstream/emscripten/system/include/libc -isystemE:/shazan/emsdk/upstream/emscripten/system/lib/libc/musl/arch/emscripten -MMD -MP -MF Release-wasm/main.d -o Release-wasm/main.o -c main.c
Linking Release-wasm/output.wasm ...
"E:/shazan/emsdk/upstream/bin/wasm-ld" -strip-all -gc-sections -no-entry -allow-undefined -import-memory -export=__wasm_call_ctors -export=malloc -export=free -export=main   -o Release-wasm/output.wasm Release-wasm/main.o System.bc
"E:/shazan/emsdk/upstream/bin/wasm-opt.exe" --legalize-js-interface -O3 Release-wasm/output.wasm -o Release-wasm/output.wasm

[msmshazan]

Do I need to set environmental variables like %PATH% for some folders?

[schellingb]

No it should not rely on any environment variable like PATH at all.

Your output of make -B -n exactly matches mine, it really should be fine.

Could you check the versions of your tools by running these lines?

E:\shazan\emsdk\upstream\bin\clang -v
E:\shazan\emsdk\upstream\bin\wasm-ld -v
E:\shazan\emsdk\upstream\bin\wasm-opt.exe --version

For me I get:

# D:\dev\wasm\llvm\clang -v
clang version 10.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: D:/dev/wasm/llvm

# D:\dev\wasm\llvm\wasm-ld -v
LLD 10.0.0

# D:\dev\wasm\wasm-opt.exe --version
wasm-opt version 92 (version_92)

[msmshazan]

This is what I get when I run those commands

E:\>E:\shazan\emsdk\upstream\bin\clang -v
clang version 11.0.0 (Cswircachegitchromium.googlesource.com-external-github.com-llvm-llvm--project 57d361bd2f3668eefa7028ad4a2d616317ed437c)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: E:\shazan\emsdk\upstream\bin
  Registered Targets:
    wasm32 - WebAssembly 32-bit
    wasm64 - WebAssembly 64-bit
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64

E:\>E:\shazan\emsdk\upstream\bin\wasm-ld -v
LLD 11.0.0 (Cswircachegitchromium.googlesource.com-external-github.com-llvm-llvm--project 57d361bd2f3668eefa7028ad4a2d616317ed437c)

E:\>E:\shazan\emsdk\upstream\bin\wasm-opt.exe --version
wasm-opt version 93 (version_93-13-g33ee4ccd4)

[schellingb]

I updated to a pre-release of llvm 11, it's not exactly the same version as you have but it should be similar at least. Yet, the output WASM file is correct for me.

Hmm. If it's really the llvm build then the build you have (that you got through Emscripten installation I think) is somehow different from official llvm builds.

Can you run the commands separate and check the output sizes? Something like this

# cd E:\shazan\website

# "E:/shazan/emsdk/upstream/bin/clang" -DNDEBUG -Ofast -std=c99  -target wasm32 -nostdinc -D__EMSCRIPTEN__ -D_LIBCPP_ABI_VERSION=2 -fvisibility=hidden -fno-builtin -fno-exceptions -fno-threadsafe-statics -isystemE:/shazan/emsdk/upstream/emscripten/system/include/libcxx -isystemE:/shazan/emsdk/upstream/emscripten/system/include/compat -isystemE:/shazan/emsdk/upstream/emscripten/system/include -isystemE:/shazan/emsdk/upstream/emscripten/system/include/libc -isystemE:/shazan/emsdk/upstream/emscripten/system/lib/libc/musl/arch/emscripten -MMD -MP -MF Release-wasm/main.d -o Release-wasm/main.o -c main.c
# dir Release-wasm\main.o

# "E:/shazan/emsdk/upstream/bin/wasm-ld" -strip-all -gc-sections -no-entry -allow-undefined -import-memory -export=__wasm_call_ctors -export=malloc -export=free -export=main   -o Release-wasm/output.wasm Release-wasm/main.o System.bc
# dir Release-wasm\output.wasm

# "E:/shazan/emsdk/upstream/bin/wasm-opt.exe" --legalize-js-interface -O3 Release-wasm/output.wasm -o Release-wasm/output.wasm
# dir Release-wasm\output.wasm

For me, the files at the 3 stages are the following:

2020/05/15  02:06             1'157 main.o
2020/05/15  02:07            25'548 output.wasm
2020/05/15  02:08            19'882 output.wasm

[msmshazan]

I got these files :

05/14/2020  11:10 PM             1,366 main.o
05/14/2020  11:11 PM             2,707 output.wasm
05/14/2020  11:12 PM             1,831 output.wasm

[schellingb]

Hmmmm, slowly running out of ideas.
Can you try replacing the System.bc you built with mine:
System.bc.zip

[msmshazan]

I replaced System.bc with yours still it outputted output.wasm as 2kb
Then I tried downloading llvm manually and installing it and pointing makefile to it instead of using the emsdk one and it works!
So I guess the emsdk's llvm doesn't properly work on windows?
Thanks for your help

[schellingb]

Thank you for trying all these things and finding that out.

Weird that emsdk's llvm is different in this, and without any errors or warnings even. This closes this issue but it really is not good that there are llvm versions out there that just don't work this way. I am quite curious, I'll try to see if I can get my hands on this exact llvm version myself.

[schellingb]

So I found out why, by looking at the clang output file with the tool wasm-objdump called with wasm-objdump.exe -x Release-wasm\main.o.

For official LLVM build clang it shows this:

Function: - func[5] sig=0 <main>
Code: - func[5] size=128 <main>
Custom: - 1: F <main> func=5 binding=global vis=hidden

For emsdk LLVM build clang it shows this:

Function: - func[5] sig=0 <__main_argc_argv>
Code: - func[5] size=128 <__main_argc_argv>
Custom: - 1: F <__main_argc_argv> func=5 binding=global vis=hidden

What?!?! It somehow renames the C function "main" to "__main_argc_argv". I have no idea why that could be. Other than that the file 100% is the same. Really weird.

But with main gone, the linker sees nothing of interest and strips out pretty much everything besides "add", "malloc" and "free".

[schellingb]

Well, it turns out to be intended!

Here's where they came up with this:
WebAssembly/tool-conventions#134

And here's where it was introduced into llvm:
llvm/llvm-project@00072c0

This is new to me, and the solution is rather disappointing.
Instead of exporting a symbol called "main", we need to export 4 symbols ("main", "__original_main", "__main_argc_argv" and "__main_void"). Then in the JavaScript loader it needs to check for all 4 and call them appropriately. So much for keeping things nice and tidy.

[schellingb]

So the behavior of emsdk's llvm will be the default in the next official llvm release.

If you're interested how to update the GenJs project to deal with it, you can check the latest commit I made in my newer project WAjic:
schellingb/wajic@bd041ce?diff=unified

This shows exactly the changes needed to be done in the loader JS and the Makefile.

If I find some more free time I'll update all the demos in this repository according to that.

Thanks for reporting this!! :-)