Latest release broke storybook

[ndelangen]

In the latest release (about 2 hours ago since the time of writing this), storybook broke if users upgrade to the latest version of this package.

It seems that the virtual-modules-plugin we use is not compatible with this change:
https://www.npmjs.com/package/webpack-virtual-modules

We see the following error happening in our CI:

ERR! => Failed to build the preview
ERR! Module not found: Error: Can't resolve '/tmp/storybook/sandbox/angular-cli-13-ts/storybook-config-entry.js' in '/tmp/storybook/sandbox/angular-cli-13-ts'
ERR! /tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/Compilation.js:2016
ERR! 					const notFoundError = new ModuleNotFoundError(
ERR! 					                      ^
ERR! 
ERR! ModuleNotFoundError: Module not found: Error: Can't resolve '/tmp/storybook/sandbox/angular-cli-13-ts/storybook-config-entry.js' in '/tmp/storybook/sandbox/angular-cli-13-ts'
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/Compilation.js:2016:28)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:798:13)
ERR!     at eval (eval at create (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/tapable/lib/HookCodeFactory.js:28:14), <anonymous>:10:1)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:270:22)
ERR!     at eval (eval at create (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/tapable/lib/HookCodeFactory.js:28:14), <anonymous>:9:1)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:434:22)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:116:11)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:670:25)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:855:8)
ERR!     at <anonymous> (/tmp/storybook/sandbox/angular-cli-13-ts/node_modules/@storybook/builder-webpack5/node_modules/webpack/lib/NormalModuleFactory.js:975:5)

link to CI

The file .../storybook-config-entry.js, is injected as a virtual module using the plugin described above.

When we version lock to 5.10.0 the problem goes away.

comparing 5.10.0 & 5.11.0:
v5.10.0...v5.11.0

It looks like this code was added:
{ throwIfNoEntry: false }
link to change

Looks like it was introduced in this PR:
#324

@markjm @sokra Do you have any recommendations for us?

[ndelangen]

In the unlikely event of upgrading only enhanced-resolve, but not graceful-fs or webpack, we can expect people to update graceful-fs to fix the bug they are running into.

This doesn't seem like something we can do.. we're not in control of which version enhanced-resolve the user uses.
We also do not have a dependency on graceful-fs.

And though we can force newer version of webpack to be used, this is mostly up to the end-user of storybook to decide.

Every new installation of storybook is broken, every user that has no lockfile, or updates their lockfile runs into this issue.

[yannbf]

Here are a couple more findings after some debugging.

I logged file, err, stat in:

// FileExistsPlugin.js
fs.stat(file, (err, stat) => {
  if(file.includes('storybook-config-entry')) {
    console.log({file, err, stat})
  }
  // ...
}

And did the same in 5.11.0.

Here's running Storybook with enhanced-resolve 5.10.0:

{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.mjs',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.mjs'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.mjs'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.js',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.js'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.js'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.jsx',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.jsx'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.jsx'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.ts',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.ts'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.ts'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.tsx',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.tsx'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.tsx'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.json',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.json'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.json'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.cjs',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.cjs'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.cjs'
  },
  stat: undefined
}
{
  file: 'storybook-config-entry.js',
  err: null,
  stat: VirtualStats {
    dev: 8675309,
    nlink: 0,
    uid: 1000,
    gid: 1000,
    rdev: 0,
    blksize: 4096,
    ino: 45000001,
    mode: 33188,
    size: 5591,
    blocks: 1,
    atime: 2022-11-22T17:56:59.907Z,
    mtime: 2022-11-22T17:56:59.907Z,
    ctime: 2022-11-22T17:56:59.907Z,
    birthtime: 2022-11-22T17:56:59.907Z
  }
}

So it seems like it tries to resolve storybook-config-entry.js in all possible extensions (cjs, mjs, etc...) and fails. Then tries again, and succeeds, and then you see the virtual module is found.

Now, for version 5.11.0, the output is the following. It just fails to find the file, retries, and fails 6 times:

{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.mjs',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.mjs'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.mjs'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.js',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.js'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.js'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.jsx',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.jsx'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.jsx'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.ts',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.ts'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.ts'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.tsx',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.tsx'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.tsx'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.json',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.json'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.json'
  },
  stat: undefined
}
{
  file: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.cjs',
  err: [Error: ENOENT: no such file or directory, stat 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.cjs'] {
    errno: -2,
    code: 'ENOENT',
    syscall: 'stat',
    path: 'Users/yannbraga/open-source/status-page/storybook-config-entry.js.cjs'
  },
  stat: undefined
}

When running Storybook with WVM_DEBUG=true, in both 5.10.0 and 5.11.1 I see these logs happening before fs.stat is first invoked: preview Write virtual module: /Users/yannbraga/open-source/status-page/storybook-config-entry.js, so supposedly the file is correctly there?

Could this be an incompatibility with webpack-virtual-modules?

[yannbf]

How to reproduce the issue

The fastest way to get a Storybook + Webpack project is by running the following command in the terminal:

npx giget "gh:storybookjs/repro-templates-temp/react-webpack/17-ts/after-storybook#next" webpack-repro
cd webpack-repro
yarn
yarn storybook

[larixer]

@yannbf Thanks for the detailed repro steps! So the change introduced in:
#324
has a consequence that additional options argument is passed, it does not matter what is passed inside options object, the mere fact is important to the logic at this line:
https://github.com/webpack/enhanced-resolve/blob/main/lib/CachedInputFileSystem.js#L163
It sends all the requests having options argument directly to the core file system module without consulting with the cache implemented in this class. This breaks the whole idea of webpack-virtual-modules which is based on injecting virtual modules into this cache. I can do nothing here to fix this from webpack-virtual-modules side. The only other idea to have virtual modules is non-webpack related - direct patching of core file system module, which I have tried to avoid because of the higher level of intrusion into low-level mechanisms.

[markjm]

Yep - just traced the same line of reasoning. By same implication, there is no caching done on any requests with options, which probably isnt what we want from a perf perspective either (aside from virtual modules).

True fix is to support cache for requests with options, but that may take a small while to get working (need to check if there are wacky options for certain fs calls that might complicate things). Probably best to revert so as not to rush figuring that all out

cc @sokra

[ndelangen]

would it be possible to roll this back?

[hongxuWei]

would it be possible to roll this back?

agree