[BUG] npm 7.x: updating git dependency branch and/or refetching branch not working.

[kalbert312]

After upgrading from 6.x to 7.x I'm having trouble updating git-based dependencies.

Steps To Reproduce:

Setup:

"dependencies": {
  "@our-scope/our-dep": "git+ssh://[email protected]:github-org/github-repo.git#v0.5.0"
}

The git dependency uses npm prepare to build the project on demand when installed. No dist files are git controlled.
I pushed changes I want to test to a new branch on remote.

Steps:

1. Update the git dependency version to a new branch to test changes:

"dependencies": {
  "@our-scope/our-dep": "git+ssh://[email protected]:github-org/github-repo.git#JIRA-123"
}

2. Run npm i @our-scope/our-dep

Current Behavior:

3. npm changes the line to

"dependencies": {
  "@our-scope/our-dep": "*"
}

and installs the wrong version

Expected Behavior:

3. npm force refetches the dependency (no cache because we ran npm i @our-scope/our-dep), updates the package-lock with the correct commit, and runs prepare on the latest version.

Environment:

• OS: macOS 11.2.2
• Node: 14.15.4
• npm: 7.6.0

[wraithgar]

May be related to #2934

[zimtsui]

Same problem with npm update, npm update will not fetch changed repo from git either.

And this issue doesn't come up with npm@6, but do only with npm@7

[wraithgar]

This is partially working in v7.12.0. Note here I have specifically ran npm install abbrev. If I just run npm install the tag in package.json is not used so that is still a bug.

~/D/n/bar $ grep abbrev package.json 
    "abbrev": "github:isaacs/abbrev-js#v1.1.0"
~/D/n/bar $ npm ls abbrev
[email protected] /Users/wraithgar/Development/npm/bar
└── [email protected] (git+ssh://[email protected]/isaacs/abbrev-js.git#7136d4d95449dc44115d4f78b80ec907724f64e0)

~/D/n/bar $ cp new.json package.json
overwrite package.json? (y/n [n]) y
~/D/n/bar $ grep abbrev package.json
    "abbrev": "github:isaacs/abbrev-js#v1.1.1"
~/D/n/bar $ npm i abbrev

changed 1 package, and audited 2 packages in 2s

found 0 vulnerabilities
~/D/n/bar $ npm ls abbrev
[email protected] /Users/wraithgar/Development/npm/bar
└── [email protected] (git+ssh://[email protected]/isaacs/abbrev-js.git#a9ee72ebc8fe3975f1b0c7aeb3a8f2a806a432eb)

[wraithgar]

This is likely going to stay as it is now. The issue has to do with the fact that git branches are not idempotent. There is also no reliable way for us to know if the content after the # here is a tag or a branch. Because of that, if we made the cli always consider branches to be stale, npm ci would break every time anyone pushed to a branch that was referred to in your package.json.

Consider if you had a dev-feature branch in that repo and put github:npm/abbrev-js#dev-feature in your package.json. When installed, the lockfile will resolve that spec into a full git url i.e. git+ssh://[email protected]/npm/abbrev-js.git#[some hash].

If someone were to then push to that branch, and you were to delete your package-lock and install again, the lockfile would resolve to a completely different git url i.e. git+ssh://[email protected]/npm/abbrev-js.git#[a brand new hash]

This would be fine if npm ci didn't exist. npm ci will refuse to work if your package.json and lockfile are in disagreement with one another. npm would resolve the branch name in the package.json to the new hash, see that it's different than what's currently resolved in the lockfile, and exit with an error.

The reason npm install abbrev works is because that is an install of a specific package. It short circuits all of the "is this dependency out of date" logic and goes straight to re-resolving it. This isn't something a bare npm install can do because the entry in package.json and the resolved field are always different. There's no way to know that the value in the package.json changed.

If you do need to change to a new version of a git ref, you are going to have to install that dep explicitly in order to have npm re-resolve it in the lockfile.

[eight04]

This is likely going to stay as it is now. The issue has to do with the fact that git branches are not idempotent. There is also no reliable way for us to know if the content after the # here is a tag or a branch. Because of that, if we made the cli always consider branches to be stale, npm ci would break every time anyone pushed to a branch that was referred to in your package.json.

Instead of treating all git dependencies as staled, can npm know a git dependency is stale when the tag in package.json doesn't match the lock file? Which should be enough to handle the use case in #2844 (comment)

Otherwise, npm should at least throw an error when npm install finds an unmatched git tag, avoiding confusion like openstyles/less-bundle#11 (comment). Currently it just silently returns, leaving unmatched tag in package.json and the lock file.