Feature request: a loadFile hook

[RReverser]

Right now, it's already possible to hook into the file loading via locateFile and provide URLs for JavaScript files, Workers, Wasm etc. After retrieving such a URL, Emscripten takes care of loading the actual resource in a hardcoded way depending on the environment (e.g. using XHR on the Web).

Problem is, many users use Emscripten output as part of larger Web apps using bundler tooling, which might offer optimisations such as code splitting, merging various resources together etc.

In this case, URL + XHR is no longer a sufficient combination for retrieving the contents back, as the only way to retrieve resources back is via custom APIs (e.g. import('some-loader!../path-to-resource') syntax in Webpack). It's possible to opt-out of those APIs and preserve files as separate, but then user loses out on any optimisations offered by such tooling.

I propose that, in addition to locateFile(URL) hook, Emscripten would provide a loadFile(URL) hook as well, which would allow to customise the way a file is loaded and integrate better with arbitrary tooling and environments.

[kripken]

Interesting, thanks, this is an area of web tooling I'm not familiar with...

I wonder if #12203 might be relevant? Instead of adding a new hook (which we try to avoid), we could "polyfill" the XHR API for such environments. But I'm not sure it's the right solution for this.

[RReverser]

That issue definitely looks relevant! That said, I think polyfilling the whole XHR is a more heavyweight solution than a hook actually.

Perhaps, more generally, we should try to extract a common Environment interface instead that would provide all necessary APIs for interacting with the outside world?

Then most of Emscripten would use a single common API for loading files, getting env variables and so on, while actual implementations would be kept in respective files like environment/node.js, environment/web.js, environment/autodetect.js, etc.

This would make codebase a bit simpler, output smaller, and at the same time allow providing custom implementations for 3rd-party environments like bundler-specific ones.

[RReverser]

Oh wait, on a second read I see that's exactly what you're proposing, aside from the XHR polyfilling. So yeah, overall agree, but instead of XHR polyfill I'd probably use something like subset of Fetch API instead.

[RReverser]

I'll close this issue in favour of the other one to keep discussion in one place.