WebAssembly and Exception Handling (throw)

[anutosh491]

Hey all,

After getting clang-repl running in the browser, I worked on integration it with jupyterlite. Xeus-cpp, a C++ Jupyter kernel provides a way to integrate it. Here is a static link that can be used to try C++ completely in the browser (https://compiler-research.org/xeus-cpp/lab/index.html) . An example notebook xeus-cpp-lite-demo.ipynb has been provided to show what all can be acheived.

Coming back to the issue. I see we can't run throw (or a try catch blocking involving throw) while running clang-repl in the browser.

The debug logs tell me that this comes from dlopen

All this can be tried through the static link above.

[anutosh491]

Now the point is that for running clang-repl in the browser. This is the workflow taken

code -> PTU -> LLVM IR -> wasm object -> wasm binary -> loaded on top of main module using dlopen

1. So as it fails in the dlopen step, we know for sure that the LLVM IR is being produced and also a wasm binary is being produced (hopefully correctly)

Pasting them down below just for reference

i) LLVM IR (only relevant part)

@_ZTIi = external constant ptr
@llvm.global_ctors = appending global [1 x { i32, ptr, ptr }] [{ i32, ptr, ptr } { i32 65535, ptr @_GLOBAL__sub_I_incr_module_2, ptr null }]
define internal void @__stmts__0() #0 {
  %1 = call ptr @__cxa_allocate_exception(i32 4) #2
  store i32 1, ptr %1, align 16
  call void @__cxa_throw(ptr %1, ptr @_ZTIi, ptr null) #3
  unreachable
}
declare ptr @__cxa_allocate_exception(i32) #0
declare void @__cxa_throw(ptr, ptr, ptr) #0
; Function Attrs: noinline
define internal void @_GLOBAL__sub_I_incr_module_2() #1 {
  call void @__stmts__0()
  ret void
}

ii) wasm module

(module $incr_module_2.wasm
  (memory $env.memory (;0;) (import "env" "memory") 0)
  (table $env.__indirect_function_table (;0;) (import "env" "__indirect_function_table") 0 funcref)
  (global $__memory_base (;0;) (import "env" "__memory_base") i32)
  (global $__table_base (;1;) (import "env" "__table_base") i32)
  (func $__cxa_allocate_exception (;0;) (import "env" "__cxa_allocate_exception") (param i32) (result i32))
  (func $__cxa_throw (;1;) (import "env" "__cxa_throw") (param i32 i32 i32))
  (global $typeinfo for int (;2;) (import "GOT.mem" "_ZTIi") (mut i32))
  (func $__wasm_call_ctors (;2;) (export "__wasm_call_ctors")
    call $_GLOBAL__sub_I_incr_module_2
  )
  (func $__wasm_apply_data_relocs (;3;) (export "__wasm_apply_data_relocs")
  )
  (func $__stmts__0 (;4;)
    (local $var0 i32)
    i32.const 4
    call $__cxa_allocate_exception
    local.tee $var0
    i32.const 1
    i32.store
    local.get $var0
    global.get $typeinfo for int
    i32.const 0
    call $__cxa_throw
    unreachable
  )
  (func $_GLOBAL__sub_I_incr_module_2 (;5;)
    call $__stmts__0
  )
)

I think this looks correct to me !

[anutosh491]

Now coming back to the dloepn step. The debugger through chrome tools tells me that this is the last part where it ends up

emscripten/src/library_dylink.js

Lines 856 to 859 in 4c14f1f

	var init = moduleExports['__wasm_call_ctors'];
	if (init) {
	if (runtimeInitialized) {
	init();

Which means it is trying to execute this block I'd guess

  (func $__wasm_call_ctors (;2;) (export "__wasm_call_ctors")
    call $_GLOBAL__sub_I_incr_module_2
  )
  (func $__stmts__0 (;4;)
    (local $var0 i32)
    i32.const 4
    call $__cxa_allocate_exception
    local.tee $var0
    i32.const 1
    i32.store
    local.get $var0
    global.get $typeinfo for int
    i32.const 0
    call $__cxa_throw
    unreachable
  )
  (func $_GLOBAL__sub_I_incr_module_2 (;5;)
    call $__stmts__0
  )

But it isn't able to. Now __wasm_call_ctors calls _GLOBAL__sub_I_incr_module_5 which simply calls __stmts__0 ... So I am guessing its just not able to run __stmts__0 but I think even that is being framed correctly ?

[anutosh491]

cc @sbc100 @kripken

Here's what I thought might be going wrong.

1. Just as a sanity check I thought that I should confirm the presence of symbols in the final xcpp.wasm be built (the wasm binary out of xeus-cpp that acts as a main module)

(xeus-lite-host) anutosh491@Anutoshs-MacBook-Air build % wasm-objdump -x xcpp.wasm | grep __cxa
 - func[1] sig=10 <__cxa_find_matching_catch_2> <- env.__cxa_find_matching_catch_2
.....
 - func[9678] <__cxa_allocate_exception>
.....
 - global[3] <__cxa_throw>

(xeus-lite-host) anutosh491@Anutoshs-MacBook-Air build % wasm-objdump -x xcpp.wasm | grep _ZTIi
 - global[1701] i32 mutable=0 <_ZTIi> - init i32=409276
 - global[1701] -> "_ZTIi"

I think we have everything

2. I thought this might be a -fwasm-exceptions or -fexceptions thingy. I realized we build xeus-cpp with -fexceptions but llvm isn't using that (we obviously need to build llvm for wasm to get libclangInterpreter.a which facilitates using clang-repl in the web). So I tried this too but didn't help me in any way. Still get the same result.

If y'all are interested in the configuration, this is what i used.

emcmake cmake -DCMAKE_BUILD_TYPE=MinSizeRel         \
    -DBUILD_SHARED_LIBS=OFF                         \
    -DLLVM_HOST_TRIPLE=wasm32-unknown-emscripten    \
    -DLLVM_TARGETS_TO_BUILD="WebAssembly"           \
    -DLLVM_INCLUDE_BENCHMARKS=OFF                   \
    -DLLVM_INCLUDE_EXAMPLES=OFF                     \
    -DLLVM_INCLUDE_TESTS=OFF                        \
    -DLLVM_ENABLE_LIBEDIT=OFF                       \
    -DLLVM_ENABLE_PROJECTS="clang;lld"              \
    -DLLVM_ENABLE_THREADS=OFF                       \
    -DCLANG_ENABLE_STATIC_ANALYZER=OFF              \
    -DCLANG_ENABLE_ARCMT=OFF                        \
    -DCLANG_ENABLE_BOOTSTRAP=OFF                    \
    -DLLVM_ENABLE_ZSTD=OFF                          \
    -DLLVM_ENABLE_LIBXML2=OFF                       \
    -DCMAKE_CXX_FLAGS="-Dwait4=__syscall_wait4 -fexceptions" \
    ../llvm

Apart from adding the -fexceptions flag here ... everything is what we already use for getting the static link to work !

[sbc100]

Is there still an issue here?

You are correct that you need to make sure that -fwasm-exceptions is either used everywhere, or nowhere. You cannot mix code compiled with and without that flag. Did making that consistent fix your issue?

From your original backtrace looks like the code its trying to load a DLL called "const char*", which is very odd. Can you stop in the debugger and see why that might be? Is the name of file being loaded really "const char*"?

(BTW, you file bugs like this it would be very helpful if you could copy and paste the text rather than attaching screenshots. Using test make it much easier for use to search / copy / etc within the issue.)

[anutosh491]

Is there still an issue here?

Yes it is.

Did making that consistent fix your issue?

So I think we tried building the whole stack with -fexceptions (jupyterlite, xeus-cpp, llvm etc) and we haven't moved to -fwasm-exceptions yet as we thought using one of these for the whole toolchain would be enough !

Can you stop in the debugger and see why that might be? Is the name of file being loaded really "const char*"?

Is it ? So when we use clang-repl in the browser ever code block produced a file named incr_module_xx.wasm where xx is the code block number. So yeah don't think that's the file name here !

I think it, it might just be the exception ptr type or something (not sure). The below issue looks relevant here.
#6330

EDIT: Also just questioning my breakdown here. The wasm module generated looks correct to me and I think it is the init() call that I referred above that doesn't work ! Maybe someone could confirm that for me ?

[sbc100]

I think that fact that _dlopen_js is being called with the string "const char*" rather than the name of a DLL is really the clue. That looks really wrong.

Can you break at that callsite and see the string ptr value being passed to _dlopen_js? Presumably the user code passed a completely different string.. can you print the ptr value on C++ side too? It looks like dlopen is being called from side module with function names like $func917. I imagine somehow the DLL is confused about where its static data lives? Perhaps __memory_base was not correctly set when the DLL was loaded?

Can you try building you side modules with --profiling-funcs so you get useful functions names instead of $func917?

[anutosh491]

Hey @sbc100 sorry took some time to get back

But this the whole log when we try executing "throw 1;"

It points to the addModule function as expected where the dlopen is being called

https://github.com/llvm/llvm-project/blob/main/clang/lib/Interpreter/Wasm.cpp#L65

[anutosh491]

So this is what I see when I build xeus-cpp with Assertion=0 vs Assertions=1

1. Assertions=0 (looks like some exception ptr)
   

2. Assertions=1 (type being returned)

[anutosh491]

Also I don't see absolutely no difference in how dlopen is working for any cell that works vs the cell executing throw 1;
Its the same. The file name is also tmp/incr_module_2.wasm which should be the case. Case 1 is when some default case vs case 2 is with throw

At this point there is so much we can already do (check the example notebook https://github.com/compiler-research/xeus-cpp/blob/main/notebooks/xeus-cpp-lite-demo.ipynb) that not being able to use throw 1; seems very weird.

[anutosh491]

I executed stuff till this the final failure which comes up here

emscripten/src/lib/libdylink.js

Lines 857 to 860 in f9ca632

	var init = moduleExports['__wasm_call_ctors'];
	if (init) {
	if (runtimeInitialized) {
	init();

As soon as the debugger hits init() I get the error message.

Nothing really seems fishy till the end. I see memory_base is 0 here (not sure if that shouldn't be the case, looks ok to me)

[sbc100]

What is memorySize? memoryBase should only be zero if memorySize is also zero.

The memoryBase and tableBase are there the data segment and table segment for your DLL are stored. They will only be zero when you module has no memory segment or no table segment of its own.

You can see how much data and table space your module needs by looking at the dylink section of the module/DLL. Its always the first section in any wasm DLL:

$ ./emcc -sSIDE_MODULE test/hello_world.c
$ wasm-objdump -x a.out.wasm 

a.out.wasm:	file format wasm 0x1

Section Details:

Custom:
 - name: "dylink.0"
 - mem_size     : 15
 - mem_p2align  : 0
 - table_size   : 0
 - table_p2align: 0
...

Here you can see the hello world program, when compiled to DLL requires 15 bytes of memory and zero table slots.

[anutosh491]

This is what metadata has after getDylinkMetadata(binary)

var loadWebAssemblyModule = (binary, flags, libName, localScope, handle) => {
var metadata = getDylinkMetadata(binary);

memoryAlign: 0
memorySize: 0
neededDynlibs: []
tableAlign: 0
tableSize: 0
tlsExports: Set(0) {size: 0}
weakImports: Set(0) {size: 0}
[[Prototype]]: Object

Binary shows Int8Array(809) if that's relevant. So yeah I guess it is the getDylinkMetadata call that doesn't go as expected ?

Also I see you've mentioned about using wasm-objdump but I am not sure how to put it to use at runtime. As in every cell block gives me a side module (code -> PTU -> llvm IR -> incr_module_xx.so file -> incr_module_xx.wasm file -> loaded on top of main module using dlopen)

So the max I can do is go through the incr_module_xx.wasm file which comes out of to be something like this

(module $incr_module_2.wasm
  (memory $env.memory (;0;) (import "env" "memory") 0)
  (table $env.__indirect_function_table (;0;) (import "env" "__indirect_function_table") 0 funcref)
  (global $__stack_pointer (;0;) (import "env" "__stack_pointer") (mut i32))
  (global $__memory_base (;1;) (import "env" "__memory_base") i32)
  (global $__table_base (;2;) (import "env" "__table_base") i32)
  (func $__cxa_allocate_exception (;0;) (import "env" "__cxa_allocate_exception") (param i32) (result i32))
  (func $__cxa_throw (;1;) (import "env" "__cxa_throw") (param i32 i32 i32))
  (global $typeinfo for int (;3;) (import "GOT.mem" "_ZTIi") (mut i32))
  (global $__dso_handle (;4;) (export "__dso_handle") i32 (i32.const 0))
  (func $__wasm_call_ctors (;2;) (export "__wasm_call_ctors")
    call $_GLOBAL__sub_I_incr_module_2
  )
  (func $__stmts__0 (;3;)
    (local $var0 i32)
    i32.const 4
    call $__cxa_allocate_exception
    local.tee $var0
    i32.const 1
    i32.store
    local.get $var0
    global.get $typeinfo for int
    i32.const 0
    call $__cxa_throw
    unreachable
  )
  (func $_GLOBAL__sub_I_incr_module_2 (;4;)
    call $__stmts__0
  )
)

P.S: Also just for you to confirm for yourself that the error is coming out of dlopen itself (and also to maybe play around and debug any questions you might have) I think you can try running throw 1; through our static link and add breakpoints in the source files to see what's happening https://compiler-research.org/xeus-cpp/lab/index.html

[sbc100]

So incr_module_xx.wasm module looks like it actually doesn't have any data or table slots (I don't see any data segments or elem segments).

BTW you can run wasm-object on your incr_module_xx.wasm file to see the dylink section if you like. It not showing up in the wat disassembly that you attached above.

Can you set a breakpoint on the "Error loading dyanmic library" line and see inspect the exception (e) that is being thrown? What does the stack trace for that exception look like?

[anutosh491]

So incr_module_xx.wasm module looks like it actually doesn't have any data or table slots (I don't see any data segments or elem segments).

This is what I see for int x = 10;

This seems to have a data segment at the end. So I don't think this is the case for every cell. It's just throw which might be at fault.