ECMAScript Module Support

[astorm]

In order to detect when certain methods are called and provide timings for those method calls, the Node.js Agent uses a module named require-in-the-middle. This module "hooks" (monkey-patches, intercepts, etc.) the initial loading of Node.js CommonJS module and allows us to wrap the methods we want with our custom timing and span generation.

Node 15 (and it's LTS support partner, Node 16) mark the support of ECMAScript modules (also known as ES6 modules) as non-experimental. https://nodejs.org/docs/latest-v15.x/api/esm.html#esm_modules_ecmascript_modules

This means third part code can begin using ECMAScript modules (import) in place of CommonJS modules (require). The require-in-the-middle module does not hook these new import/ECMAScript modules. This means any normally instrumented module that uses ECMAScript modules will not be instrumented by the Node.js Agent.

We need to determine the best was to hook these new modules. (perhaps the experimental loaders, perhaps compilation/transpilation, perhaps something else).

This will be a long term effort. If folks have ideas on approaches we'd love to hear about them.

Work Around: In the meantime, users can fallback to importing modules via require. If there's an instrumented module that's providing ECMAScript only modules, or the agent is somehow not working with the new ECMAScript features, we'd like to hear about them.

Todo:

• investigate: loader hooks for ESM Modules (was: import-in-the-middle for ESM Module Projects) #2343

[trentm]

Relevant to Kibana here: elastic/kibana#106868

[nhe23]

It would be really great to have esm support as it broadly used nowadays. Modern fullstack tooling like vite depends heavily on esm and it is either not possible or really time-consuming to convert all the generated code to cjs again. In my case I'm trying to set up apm with Sveltekit which kind of works, but in many cases I need to start transactions myself because modules cannot be monkey-patched. So I'm really looking forward to this feature.

[hjhjdev]

+1 for node based backend with ts-node or others use .ts directly.

[cdavid15]

We encountered this problem yesterday. Will this be delivered in 8.8? I can see it was originally considered for 8.1, 8.2 and 8.3 previously so trying to assess whether we hang fire for 8.8 or come up with alternative solutions.

[trentm]

@cdavid15 No, unfortunately 8.8 won't provide a complete solution for this. Full ESM support is a big change. It will rely on Node.js loader hooks, which are currently experimental and expected to change/break. It will require significant changes to how this agent is doing its auto-instrumentation to most, if not all, supported modules.

We've been using this issue over multiple milestones to do investigatory work. Apologies that that ends up being a bit misleading when we have it on a particular milestone. We're keen to have a solution for instrumenting ESM modules, but don't have an expected or firm target timeline for it yet.