Support multi-module 1-step Instantiate with dependencies

[flagxor]

We should support a version of instantiate that will continue to allow us to recommend it as the preferred instantiation method one dynamic linking / multiple modules become more popular.

WebAssembly.instantiateGroup({a:Promise<Response>, b:Promise<Response>, otherImports:importObject})
With this, the module at urlA could import the foo export of the module at urlB by importing from "b" "foo".

Questions:

• Do we want to support cycles or just DAGs?
• Do we want to support the BufferSource overload?
• How does this interact with ES6 modules? (or does it?)

[rossberg]

Just repeating my usual observation that allowing cycles is much more than a mere API addition. It would introduce full-fletched recursive modules into Wasm, which is a tricky design space with numerous potential implications on present and future versions of the language. (FWIW, recursive modules for typed languages are still an active and rather involved research topic.) We should be very careful not to do something hasty there.

So for a Milestone 2 feature it would be much safer to separate the problem of bulk instantiation from recursive linking and only allow DAGs.

One other nit: otherImports should not be part of the object but a separate parameter.

[lukewagner]

Agreed on starting with DAGs, just as a practical first step.

On the otherImports bit: good point. I had been thinking that it was simpler to put them all in the same "namespace", but thinking more about this: in the future it is likely we'd want to allow importing Module, ArrayBuffer, etc and then there would be an ambiguity: are we asking to instantiate these or just use them as import values. So yeah, definitely better to keep otherImports a separate parameter.

So the original reason I thought it was simple to keep them in the same namespace: let's say I'm instantiating some module in the DAG of some call instantiateGroup(modules, importObj) and that module imports "a" "b"; where do we look up "a"? We could merge modules and importObj into a single namespace, but then what about duplicates: do we require there are none or do we define precedence (e.g., modules over importObj)? The latter seems fine, actually, and no more limiting than providing a single namespace object.

[mtrofin]

@flagxor - this would be a helper API, correct?

I worry we would enter composition engine territory with this API. Questions of policy arise. Besides cycles, as @rossberg-chromium illustrates, there's also: how do you match imports and exports; what if there are 2 imports you want satisfied with the same export.

I would prefer we let third parties come up with such policies, and us just provide the basic building blocks, API-wise.

[jfbastien]

@mtrofin this is an offshoot of #991. The main goal is to allow multiple ongoing streams to make forward progress, reducing the time to play on a wasm site that uses dynamic linking.

You bring up important design points, but I don't think third parties can do what you propose. The JS API have to do this. Unless I'm wrong :)

So, let's consider what you say, but I think we have to do so in the JS API.

[mtrofin]

Let's also take a step back. What's the motivating user story for this API?

The proposal frames this in the context of dynamic linking. The motivations for that feature are, afaik:

• apps like photoshop that have lots of tools, the user doesn't need them all, so the app brings down needed tool on demand;
• the shared library case, where a common engine (like a gaming engine) is cached and then multiple unrelated apps instantiate it, instantiate their own module(s), and share same heap
• the hosted JIT language - the JIT-ed program is a module that's instantiated separately from (but likely sharing heap with) the module that's the language runtime

In all of these cases, bindings happen at different times - first we instantiate one module; later, and, importantly, because of some user event, we want to instantiate another one and relate imports/exports.

The instantiateGroup API doesn't accommodate for that "some user event".

Which places the API square in the field of composition (a'la dependency injection frameworks and composition engines). Why do we want (what motivates) that?

@jfbastien - I am not sure how #991 blocks a third party composition engine from working when using the response-based APIs. Specifically:

var components = [];
components.push(WebAssembly.compile(fetch(url1)));
components.push(WebAssembly.compile(fetch(url2)));

Promise.all(components)
  .then(compiled_modules => {
    // wire them here.
});

[lukewagner]

@mtrofin You're thinking about runtime dynamic linking. What we're talking about here is load-time dynamic linking. The use cases for that are:

• common engine, small variation between multiple applications
• separate frequently-changing parts of app from infrequently changing parts to minimize re-download+recompile

[mtrofin]

@lukewagner - I agree, this is about load-time linking. I think I went about my point in too convoluted of a way. Let me try this:

• I claim developers are not blocked in achieving the load time composition use cases, even when you factor in Adding Response based versions of validate, compile, and instantiate. #991. I offered a snippet for the latter, and I think we know how it'd work with the shipped APIs.

• because developers aren't blocked, and because instantiateGroup would have to make policy decisions (like how to match imports/exports, deal with duplicates, etc), I propose we not do this API: it will be expensive to spec, implement, and maintain, and it risks not being used. Instead, I propose we let the community bring in whatever load time composition engines they see fit, and should a winning set of policies come out, then reconsider providing this in the platform.