[BUG] ^7.20.3 no longer resolves local package first on install (workspaces)

[rijkvanzanten]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

In a workspaces based environment, for example:

.
+-- package.json
`-- packages
   +-- a
   |   `-- package.json
   `-- b
       `-- package.json

Running npm install b --workspace a will no longer install the locally linked package correctly. Instead, it'll use the version hosted on npm, or throw a 404 if you use a custom (unique) package name.

Expected Behavior

In <7.20.x, NPM would resolve the local package, and correctly alter package.json and package-lock.json.

Steps To Reproduce

With node v16.6.1 and npm 7.20.3 installed:

1. Make a new directory

mkdir ./npm-install-debug

2. Add a package.json

cd ./npm-install-debug
npm init -y

3. Setup 2 workspace packages

npm init -w ./packages/a
npm init -w ./packages/b

(I used @rijk/a and @rijk/b as names, to avoid confusion with the existing packages a and b.)

5. Run npm install in the root, to give it a chance to npm link all the packages (not sure if needed)

npm install

6. Try installing @rijk/b as a dependency of @rijk/a

npm install @rijk/b -w @rijk/a

5. See error...

npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@rijk%2fb - Not found
npm ERR! 404
npm ERR! 404  '@rijk/b@*' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/rijkvanzanten/.npm/_logs/2021-08-10T20_07_25_648Z-debug.log

---

If you install 7.20.2 or below (npm i -g npm@7.20.2), the above flow still works as expected.

Environment

• OS: macOS 11.4, also seen in GitHub Actions
• Node: v16.6.1
• npm: 7.20.3 and up

[gribnoysup]

Running into the same issue without even trying to install specific packages inside the workspaces, just running npm install from the project root where some workspaces are not yet published / missing from the registry. If that's any help for triaging, here is a very verbose install log of a failed install

[ParadoxInfinite]

I could reproduce the result by using the steps in the issue.
After a bit of shallow debugging, I found that the error E404 is found even in v7.20.2, it is not thrown however. In the latest build v7.20.6, it is. So, not sure if this is intended or a bug.

Now, I did a bit more digging around and found there are a couple of interesting keys which differ between v7.20.6 and v7.20.2

In the parent node:

ArboristVirtualNode {
  name: 'a',
  packageName: '@paradox/a',
  version: '1.0.0',
  location: '',
  path: '/Users/paradoxinfinite/Documents/testing/packages/a',
  extraneous: true,
  dev: true,
  optional: true,
  peer: true,
  isProjectRoot: true,  // this field is not present in v7.20.2
  edgesOut: Map(1) { '@paradox/b' => { prod @paradox/b@^1.0.0 MISSING } },
  sourceReference: ArboristNode {
    name: 'a',
    packageName: '@paradox/a',
    version: '1.0.0',
    location: 'packages/a',
    path: '/Users/paradoxinfinite/Documents/testing/packages/a',
    isWorkspace: true,  // this field is not present in v7.20.2
    edgesOut: Map(1) {
      '@paradox/b' => { prod @paradox/b@^1.0.0 -> node_modules/@paradox/b }
    }
  }
}

I couldn't get any further than this, so if someone who knows the https://github.com/npm/arborist codebase better could help or point me out on where to look, that'd be great!

Finally, by the looks of it, this issue would be more relevant to https://github.com/npm/arborist, you could try raising an issue there!

[willnode]

I had a problem with Gatsby running with Workspaces.

npm i gatsby-cli -g
npm init -y
gatsby new a
gatsby new b

Then adding workspaces in package.json

Before dedupe

D:\m\a>npm list methods
gatsby-starter-default@0.1.0 D:\m\a
`-- gatsby@3.11.1
  `-- express@4.17.1
    `-- methods@1.1.2

After dedupe

D:\m\a>npm list methods
a@0.1.0 D:\m\a
`-- (empty)
D:\m\b>npm list methods
b@0.1.0 D:\m\b
`-- (empty)

Despite

D:\m>npm list methods
m@1.0.0 D:\m
`-- a@0.1.0 -> .\a
  `-- gatsby@3.11.1
    `-- express@4.17.1
      `-- methods@1.1.2

This broke my gatsby builds.

Unfortunately downgrading to 7.20.2 didn't work for me :( could be a separate issue.

My node is v14.17.3

[simllll]

Anything we can do to get this fixed? Super annoying bug for us, I need to downgrade npm every few days due to this issue.

[rijkvanzanten]

We temporarily updated our CI/CD pipeline to a locked outdated version of NPM to mitigate this:

      - name: Use Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '16.x'

      # See https://github.com/npm/cli/issues/3637
      - run: npm i -g npm@7.20.2

https://github.com/directus/directus/blob/607e32ea36c115a9d6cf820a26475ebbf9e6e9ed/.github/workflows/release.yml#L36-L42

For what it's worth, OSS - Sprint 36 says it's due September 07, so lets hope for the best 🤞🏻

[simllll]

Is there any status update to this issue? Will it be fixed? Is it not reproducable on your side? Or will this be the new intended behaviour? Thanks

all are actually in the same monorepo:

npm WARN
npm WARN Could not resolve dependency:
npm WARN @hokify/shared-components@"^1.2.9" from @hokify/pwa-core@1.2.10
npm WARN packages/pwa-core
npm WARN @hokify/pwa-core@1.2.6
npm WARN node_modules/@hokify/pwa-core
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@hokify%2fuser-app - Not found
npm ERR! 404
npm ERR! 404 '@hokify/user-app@^1.2.10' is not in this registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)

[rijkvanzanten]

I'm wondering if the regression that caused this might be related to #3881