dynamic linking verus module loading: what's the difference?

[jfbastien]

#53 discusses dynamic linking (post-V1), and PRs such as #71 document module loading (V1 feature). Speaking in-person, @lukewagner, @kg, @ncbray and I don't quite agree on the differences.

1. Does dynamic linking inply heap sharing? We seem to agree it does.
2. Is module loading just FII/syscall?
3. If V1 had dynamic linking, how would uses of module loading be different?
   • Would developers hack-in dynamic linking support on top of V1 using module loading?
4. Is module loading hindering non-browser uses?
   • If stdout and write were accessed through a dynamic library then it may be more portable than if it went through module loading and baked-in console.log.

Maybe we should even consider supporting dynamic linking for V1.

[kripken]

Dynamic linking in v1 would require it to be polyfillable, though? I thought that was the difference with module loading: module loading loads self-contain modules, without heap sharing, while dynamic linking allows heap and function pointer sharing, both of which we can't polyfill.

[kg]

I am not aware of any reason we can't polyfill dynamic linking in v1. We can't polyfill it to asm.js, perhaps but I'm not even convinced of that as a fact.

[sunfishcode]

As soon as we give people dynamic linking, in any manner whatsoever, we have an ABI, floor to ceiling.

Let's think carefully about whether we want to do that in v.1 or not.

[kripken]

@kg: very good point. We could polyfill it in multiple ways, if we do not limit ourselves to asm.js, or use asm.js but do not use asm.js function tables. And this would not be a regression over anything asm.js offers. So I don't see any problems there. If we think dynamic linking should be in v1, then those are viable options (with downsides).

But doing dynamic linking in v1 does mean a lot more work in v1, including specifying an ABI as @sunfishcode says. I guess I'm not sure of the motivation for moving dynamic linking to v1? Do we think it is more urgent now for some reason?

[sunfishcode]

It's my impression, including from today's meeting, that there's consensus that we don't want to define a full ABI in v.1, and as a consequence, that there won't be dynamic linking in v.1 as an official feature. Please correct me if I'm wrong.

It's ok if someone as an independent project were to build tools providing dynamic linking functionality. I don't think there'd be any reasonable expectation that we standardize whatever ABI they happened to use.

Consequently, Future Features is an appropriate milestone here, since that's currently how dynamic linking is categorized.

[kg]

The motivation from yesterday's discussion was roughly this:

Module loading is a lot like load-time dynamic linking. The overlap suggests that perhaps they should be unified, or people will look at Module Loading and use it as a shim to get many of the benefits of load-time dynamic linking. It's also possible people would want it enough to implement it at load time by processing executables.

We all agreed that those were legitimate concerns, but that it made sense to continue treating dynamic linking (both run-time and load-time) as a post-v1 feature. There was a shared PoV that public feedback (once we announce) might strongly motivate us to pull load-time dynamic linking into v1, but there's no reason to do that now. It's likewise possible that public feedback will de-emphasize dynamic linking as a feature because there will turn out to be less demand than we expected.

[jfbastien]

@sunfishcode agreed ABIs are a good reason to punt this post-V1.

@kg captured my and the group's views well in the comment above. I Heartily Endorse This Event Or Product.

[ncbray]

WASM-level dynamic linking does not require specifying a stable or standard ABI for C dynamic linking. You could get by with an ad-hoc C ABI by always recompiling all libraries with the same compiler and distributing new versions in lockstep.

I am not necessarily arguing that dynamic linking is a should-have for v1. Just that some of the rationale is incorrect because it conflates dynamic linking with long term ABI stability/compatibility and these are distinct issues. It would be acceptable to provide dynamic linking without any compatibility guarentees.

[sunfishcode]

With #682, we now have a design which should be able to support proper dynamic linking in the MVP. Note that we're still not ready to declare a stable ABI yet, as we'll still want to incoporate ABI-exposed features such as multiple return values, zero-cost EH, and TLS, when they become available.

Also, within the context of #682, we now have an answer to the top-level question here, which is that there is now no fundamental wasm-level difference. From a toolchain perspective, there will surely be a set of conventions for how dynamic libraries interoperate with the dynamic linker and the main executable, however it'll be built on the same concept of wasm modules, with imports and exports of functions, linear memory, globals, and tables.