Calls to Math are several orders of magnitude slower than when run outside of Jest

[theblang]

It appears that at least some calls to Math (only tested abs and acos) are several orders of magnitude slower than when run outside of the Jest environment.

See this repo and follow the first list of instructions to reproduce the issue. Running Math.abs ten million times takes about 10ms outside of Jest, but increases to about 2000ms within a Jest environment.

OS: macOS High Sierra 10.13.2
Jest: 21.2.1
Node: 8.9.1
NPM: 5.5.1

[theblang]

It seems like it could be some issue of optimization at a lower level perhaps because a single call, or even ten calls, to Math.abs is about the same speed regardless of environment; but as soon as I make the call a hundred times I can tell a difference in run speed. It was suggested in the referenced ticket to try a VM context, which I did and saw no slowdown.

[SimenB]

This is really interesting... It went from 2000ms to 1000ms by just injecting Math from parent context, but it's still a lot slower (and we don't want to do that anyways).

I think we somehow hit some deoptimisation, but I'm not sure how to debug that.

Thanks for the focused reproduction!

[chucksellick]

My suspicion is that this is somehow to do with parallel virtualisation of tests, which uses workers.

[SimenB]

Using runInBand (which disabled parallelization) doesn't make a difference

[chucksellick]

Does jest affect babel configuration in any way or does it take whatever's in .babelrc?

[cpojer]

cc @fhinkel do you ave any insights as to why Math would behave so much differently inside of a vm context vs outside?

[fhinkel]

I don't know of an obvious reason why Math.abs should be slower in a vm context. But I think @mattblang said in comment that he doesn't observe the slowdown when comparing regular to vm runs?

[cpojer]

Yeah that makes it even more odd, we are not touching the Math object created using the VM module so I’m unclear what would cause it to be slower than normal.

[fhinkel]

@mattblang thanks so much for providing a focused repo and instructions so I didn't have to guess what to run and what to time 👍 .

I digged into this a little 🕵️‍♀️:

Unfortunately, it's the normal overhead that comes from using the vm module. The vm module uses interceptors for every global property and the interceptors are not cheap. If the code heavily uses global variables, the slowdown is larger. It's not specific to Math, you see the slowdown for any global object.

Jest uses the vm module. If the vm module doesn't find a property, e.g., Math, in the sandboxed context, it has to go on and search the global context. Here's the relevant line of code: if the property is not on the sandbox, go on and check the global context. I think that's why @SimenB sees the 50% speed up, because it's one instead of two look ups. Here's a bit of background information if you want to understand the vm implementation on a high level.

I'm assuming you opened this issue because that slowdown is a real world problem for Math heavy applications like pngjs? If needed, one could introduce a --math mode to Jest: If you know that the code you want to test heavily uses Math, Jest could wrap all content in a function that has Math as a local variable. With a dirty hack that brings the tests back to 20ms instead of 2000ms (1000ms if Math is injected in the sandboxed context).

I added one line to script_transformer.js:

try {

      content = '((Math)=> {\n' + content + '\n })(Math);';         // This line is new.

      if (willTransform) {
        const transformedSource = this.transformSource(
        filename,
        content,
        instrument,
        !!(options && options.mapCoverage));

[cpojer]

@fhinkel thank you so much for digging into this, this is really really useful.

[thymikee]

Looks like we can close this now :)

[theblang]

@thymikee Well, we understand the cause now but it is still a problem that renders stuff like jest-image-snapshot almost unusable. Should this issue be left open?