chore: use yarn berry #10188

SimenB · 2020-06-23T08:39:35Z

Summary

#9476 with node_modules. I've just ran yarn, then the e2e tests (so they lockfiles there are migrated as well), then changed our scripts since postinstall, prebuild etc no longer run. Not sure if it makes sense to cache node_modules or ditch the global yarn cache and cache .yarn/cache instead.

/cc @arcanis

Test plan

Green CI

arcanis · 2020-06-23T09:25:00Z

I think here, you need to add an environment variable to force PnP (since it's disabled repo-wide, so the E2E test doesn't find it): https://github.com/facebook/jest/blob/master/e2e/__tests__/pnp.test.ts#L18-L20

I'm not sure about the TS 3.8 problems though 🤔

SimenB · 2020-06-23T14:59:56Z

you need to add an environment variable to force PnP

what's the variable?

I'm not sure about the TS 3.8 problems though

I'll dig into them later, I think

arcanis · 2020-06-23T15:09:37Z

what's the variable?

Just YARN_NODE_LINKER=pnp should do the trick!

SimenB · 2020-06-24T08:13:40Z

@arcanis adding that doesn't fix the test (although it does change the error)

Any ideas for how to make it pass?

I would like to use pnp at some point, but landing v2 with node-modules should be simpler, so wanna do that first. It should make sure all scripts etc are correct.

arcanis · 2020-06-24T10:11:04Z

Pushed the fix on this PR (which targets your branch): https://github.com/SimenB/jest/pull/601/files

The source change for jest-pnp-resolver is here: arcanis/jest-pnp-resolver@1fbb5ea. I tried to keep it as light as possible to avoid potential regressions in the cases where it already worked (both functionally and perf-wise).

SimenB · 2020-06-24T10:21:55Z

Thanks @arcanis!

.azure-pipelines-steps.yml

SimenB · 2020-07-03T11:48:26Z

Updated to sources now, seems to have broken something?

No idea what those errors are 😛

EDIT: Ah, circle has more

Seems the builtin patches were updated. Weird - I ran yarn locally before pushing and there was no lockfile changes

arcanis · 2020-07-03T14:16:10Z

@SimenB There are two things in play here (tldr: YARN_CHECKSUM_BEHAVIOR=update yarn locally, then commit the changes):

we sometimes update the patch to include fixes or improved integrations. It's recommended to run yarn install after doing an upgrade, to make sure everything is in order. That explains why the patches are different (specifically, the generated archives now have a consistent entry ordering, which wasn't the case before).
when the archives change, their checksums change, and Yarn reports it. In general this isn't a problem (when it happens we also bump a "cache key" that causes Yarn to detect that the changes are expected and discard the error), but I forgot to do it in a previous diff. Using YARN_CHECKSUM_BEHAVIOR=update will cause Yarn to ignore the error and update the invalid checksums.

SimenB · 2020-07-03T14:17:23Z

running yarn locally before pushing didn't complain - some sort of cache?

I'll try YARN_CHECKSUM_BEHAVIOR=update yarn, thanks!

EDIT: Still no update...

$ YARN_CHECKSUM_BEHAVIOR=update yarn
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0.42s
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 1.28s
➤ YN0000: ┌ Link step
➤ YN0007: │ core-js@npm:3.6.5 must be built because it never did before or the last one failed
➤ YN0007: │ weak-napi@npm:2.0.1 must be built because it never did before or the last one failed
➤ YN0007: │ mongodb-memory-server@npm:6.5.2 must be built because it never did before or the last one failed
➤ YN0007: │ fsevents@patch:fsevents@npm%3A1.2.12#builtin<compat/fsevents>::version=1.2.12&hash=495457 must be built because it never did before or the last one failed
➤ YN0007: │ core-js@npm:2.6.11 must be built because it never did before or the last one failed
➤ YN0007: │ core-js-pure@npm:3.6.5 must be built because it never did before or the last one failed
➤ YN0007: │ gifsicle@npm:4.0.1 must be built because it never did before or the last one failed
➤ YN0007: │ jpegtran-bin@npm:4.0.0 must be built because it never did before or the last one failed
➤ YN0007: │ optipng-bin@npm:5.1.0 must be built because it never did before or the last one failed
➤ YN0000: └ Completed in 57.85s
➤ YN0000: Done in 59.82s
$ git status
On branch yarn-berry
Your branch is up to date with 'origin/yarn-berry'.

nothing to commit, working tree clean

arcanis · 2020-07-03T14:19:25Z

Hmm perhaps - you can try to run yarn cache clean --all before running the install. I'll think about how that could happen 🤔

For the record, here are the changes I have locally:

diff --git a/yarn.lock b/yarn.lock
index 552467801..ff8bd296c 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -8991,7 +8991,7 @@ fsevents@^1.2.7:
     nan: ^2.12.1
     node-gyp: latest
     node-pre-gyp: "*"
-  checksum: 181d9f88933413ce7653288b743abae38bb12d4327552698dcfa4663583933f79e63e8ce0ec7987ca087e0c49add8f05e735d17ada20bd406403fa603ded93c2
+  checksum: 07f95321e9908fc81115320b2e94bad9cd381c5a3f041ff46611f8170031da34700db3b19fd37861377d74fe92675c2c203e3537bfa561b4af5e4c7091cc2434
   languageName: node
   linkType: hard
 
@@ -9000,7 +9000,7 @@ fsevents@^1.2.7:
   resolution: "fsevents@patch:fsevents@npm%3A2.1.3#builtin<compat/fsevents>::version=2.1.3&hash=495457"
   dependencies:
     node-gyp: latest
-  checksum: b2c7c3576498b568fcc9e932d6ad10774abffb4668450de2d0afb64531f3aef35016bef0880a74f71b0fee99925301430634698d59c68781c337ba6e28ec038c
+  checksum: 0005677b72f38a129a3cbe8c3794bdc83081a2bec53dfc03b085c2e5e4ca7a33a861a779d623313652df89746d97f79d24e4fef3b101c11c39ce1ea8a9690e18
   languageName: node
   linkType: hard
 
@@ -16401,7 +16401,7 @@ fsevents@^1.2.7:
   resolution: "resolve@patch:resolve@npm%3A1.17.0#builtin<compat/resolve>::version=1.17.0&hash=3388aa"
   dependencies:
     path-parse: ^1.0.6
-  checksum: 99df12ab341fb8e96c6bce2ea521d3a1294c5c56f85524369808f7294fbf6f01d6f6c7f4408a1f6cd4fb5762b800a889629ff64abc08f12bcc8c3553e3564ab8
+  checksum: 4bcfb568860d0c361fd16c26b6fce429711138ff0de7dd353bdd73fcb5c7eede2f4602d40ccfa08ff45ec7ef9830845eab2021a46036af0a6e5b58bab1ff6399
   languageName: node
   linkType: hard
 
@@ -18534,7 +18534,7 @@ fsevents@^1.2.7:
   bin:
     tsc: bin/tsc
     tsserver: bin/tsserver
-  checksum: d7d9d86cfd4b8618f5becff8a7b77a8e5f616ba92de06e01de472cbc83edf59f98a82712d105aae9579c325233dd8207b97332a15159a74369779d3598c28cf4
+  checksum: d176f576eb66e90df93305925d31d9dfa8fe5789cd9ce59a8006cd1429ca644aa63ff6f87432937e232a8ba1b578d47b677ea10a8af4fc871a761357f7cd872f
   languageName: node
   linkType: hard

SimenB · 2020-07-03T14:25:36Z

Misclicked...

I tried this

$ cd `yarn config get cacheFolder`
$ rm fsevents-* resolve-* typescript-*
$ cd -
$ yarn
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0.45s
➤ YN0000: ┌ Fetch step
➤ YN0013: │ fsevents@npm:1.2.12 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ fsevents@npm:2.1.3 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ fsevents@patch:fsevents@npm%3A1.2.12#builtin<compat/fsevents>::version=1.2.12&hash=495457 can't be found in the cache and will be fetched from the disk
➤ YN0013: │ fsevents@patch:fsevents@npm%3A2.1.3#builtin<compat/fsevents>::version=2.1.3&hash=495457 can't be found in the cache and will be fetched from the disk
➤ YN0018: │ fsevents@patch:fsevents@npm%3A2.1.3#builtin<compat/fsevents>::version=2.1.3&hash=495457: The remote archive doesn't match the expected checksum
➤ YN0013: │ resolve-from@npm:3.0.0 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ resolve-from@npm:4.0.0 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ resolve-from@npm:5.0.0 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ resolve-url@npm:0.2.1 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ resolve@npm:1.17.0 can't be found in the cache and will be fetched from the remote registry
➤ YN0018: │ fsevents@patch:fsevents@npm%3A1.2.12#builtin<compat/fsevents>::version=1.2.12&hash=495457: The remote archive doesn't match the expected checksum
➤ YN0013: │ resolve@patch:resolve@npm%3A1.17.0#builtin<compat/resolve>::version=1.17.0&hash=3388aa can't be found in the cache and will be fetched from the disk
➤ YN0018: │ resolve@patch:resolve@npm%3A1.17.0#builtin<compat/resolve>::version=1.17.0&hash=3388aa: The remote archive doesn't match the expected checksum
➤ YN0013: │ typescript@npm:3.9.2 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ typescript@patch:typescript@npm%3A3.9.2#builtin<compat/typescript>::version=3.9.2&hash=64df9d can't be found in the cache and will be fetched from the disk
➤ YN0018: │ typescript@patch:typescript@npm%3A3.9.2#builtin<compat/typescript>::version=3.9.2&hash=64df9d: The remote archive doesn't match the expected checksum
➤ YN0000: └ Completed in 34.37s
➤ YN0000: Failed with errors in 34.82s
$ YARN_CHECKSUM_BEHAVIOR=update yarn
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0.46s
➤ YN0000: ┌ Fetch step
➤ YN0013: │ fsevents@patch:fsevents@npm%3A1.2.12#builtin<compat/fsevents>::version=1.2.12&hash=495457 can't be found in the cache and will be fetched from the disk
➤ YN0013: │ fsevents@patch:fsevents@npm%3A2.1.3#builtin<compat/fsevents>::version=2.1.3&hash=495457 can't be found in the cache and will be fetched from the disk
➤ YN0013: │ resolve@patch:resolve@npm%3A1.17.0#builtin<compat/resolve>::version=1.17.0&hash=3388aa can't be found in the cache and will be fetched from the disk
➤ YN0013: │ typescript@patch:typescript@npm%3A3.9.2#builtin<compat/typescript>::version=3.9.2&hash=64df9d can't be found in the cache and will be fetched from the disk
➤ YN0000: └ Completed in 24.77s
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed in 4.19s
➤ YN0000: Done in 29.69s

Deleting the entry from the lockfile didn't help, had to delete the cached file from the cache

SimenB · 2020-07-05T15:07:10Z

Thanks for the help @arcanis

github-actions · 2021-05-11T05:08:32Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

SimenB requested a review from jeysal June 23, 2020 08:39

SimenB force-pushed the yarn-berry branch from 5e5df87 to c898c74 Compare June 23, 2020 09:00

facebook-github-bot added the cla signed label Jun 23, 2020

SimenB force-pushed the yarn-berry branch from 576ec03 to 359f675 Compare June 24, 2020 08:12

SimenB commented Jun 24, 2020

View reviewed changes

.azure-pipelines-steps.yml Outdated Show resolved Hide resolved

SimenB force-pushed the yarn-berry branch from a1d6c8b to 2730c10 Compare June 24, 2020 12:40

SimenB mentioned this pull request Jun 24, 2020

chore: fix #9809 #10191

Closed

SimenB and others added 13 commits July 3, 2020 13:39

chore: use yarn berry

4cf42bb

fix command for getting yarn cache di

9d968db

env for pnp test

ee3d298

run yarn after rebase

57af0ea

set linker for pnp test when running yarn as well

e307c95

fix travis install command

4f3e038

drop removed script from azure pipelines

cdaa15b

Fixes the tests (#601)

8220543

lint

d90c521

cache is of course missing on travis

23c9c20

chore: attempt to fix caching on azure pipelines

e8856ae

no immutable (will break during publish)

ceff7d2

chore: yarn from sources

7e23ee9

SimenB force-pushed the yarn-berry branch from 2730c10 to 7e23ee9 Compare July 3, 2020 11:42

SimenB closed this Jul 3, 2020

SimenB reopened this Jul 3, 2020

SimenB added 5 commits July 3, 2020 16:26

chore: update hashes of yarn patches

e903d31

chore: update lockfiles of e2e tests as well

690499c

Merge branch 'master' into yarn-berry

7238a95

chore: add runYarn helper for e2e tests

d741f9e

chore: attempt to cache yarn cache on circle instead of node_modules

0f323db

merceyz mentioned this pull request Jul 5, 2020

fix(core): update cache version yarnpkg/berry#1544

Merged

3 tasks

SimenB added 2 commits July 5, 2020 14:07

cache folder is set in the environment

6d40b26

Merge branch 'master' into yarn-berry

4eccc00

This was referenced Jul 5, 2020

chore: add runYarn helper for e2e tests #10246

Merged

chore: update jest-pnp-resolver #10247

Merged

SimenB added 3 commits July 5, 2020 15:11

Merge branch 'master' into yarn-berry

39eeecb

chore: add interactive-tools plugin

d449bd0

chore: prettier

e3b62b8

SimenB merged commit 7946ba6 into jestjs:master Jul 5, 2020

SimenB deleted the yarn-berry branch July 5, 2020 15:06

SimenB mentioned this pull request Jul 5, 2020

Use Yarn PnP #9476

Draft

github-actions bot locked as resolved and limited conversation to collaborators May 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: use yarn berry #10188

chore: use yarn berry #10188

SimenB commented Jun 23, 2020

arcanis commented Jun 23, 2020

SimenB commented Jun 23, 2020

arcanis commented Jun 23, 2020

SimenB commented Jun 24, 2020 •

edited

Loading

arcanis commented Jun 24, 2020

SimenB commented Jun 24, 2020

SimenB commented Jul 3, 2020 •

edited

Loading

arcanis commented Jul 3, 2020 •

edited

Loading

SimenB commented Jul 3, 2020 •

edited

Loading

arcanis commented Jul 3, 2020

SimenB commented Jul 3, 2020

SimenB commented Jul 5, 2020

github-actions bot commented May 11, 2021

chore: use yarn berry #10188

chore: use yarn berry #10188

Conversation

SimenB commented Jun 23, 2020

Summary

Test plan

arcanis commented Jun 23, 2020

SimenB commented Jun 23, 2020

arcanis commented Jun 23, 2020

SimenB commented Jun 24, 2020 • edited Loading

arcanis commented Jun 24, 2020

SimenB commented Jun 24, 2020

SimenB commented Jul 3, 2020 • edited Loading

arcanis commented Jul 3, 2020 • edited Loading

SimenB commented Jul 3, 2020 • edited Loading

arcanis commented Jul 3, 2020

SimenB commented Jul 3, 2020

SimenB commented Jul 5, 2020

github-actions bot commented May 11, 2021

SimenB commented Jun 24, 2020 •

edited

Loading

SimenB commented Jul 3, 2020 •

edited

Loading

arcanis commented Jul 3, 2020 •

edited

Loading

SimenB commented Jul 3, 2020 •

edited

Loading