Fix #9815: fix caching for quick-jobs CI (XDG, cache keys) #9845
Conversation
|
With caching restored, these jobs become several minutes faster, e.g.
E.g. for "Meta checks" most of the time is slandered in the "ghcup" step (1m 30s). @ulysses4ever : How about skipping this step and just use the preinstalled ghc and cabal? @ulysses4ever : These jobs use a fixed cache key. This means the cache is never updated. At the same time, |
andreasabel
added
continuous-integration
re: xdg
Yes, sounds perfectly acceptable in the interest of quicker Quick Jobs.
It's probably a blunder. I'm not fluent in the cache action, so if you could update it, that would be great. Let's nuke the |
andreasabel
force-pushed
the
ci-xdg
branch
5 times, most recently
from
773602c to
c4edc3b
Compare
andreasabel
changed the title
|
When you start digging, as usual, you unearth all this bit-rot... Well, this PR got bigger than originally intended, see updated PR description on top. |
|
Before: After: |
This is the same as #9845 but for the changelogs.yml workflow.
| @@ -18,130 +18,135 @@ jobs: | |||
| meta: | |||
| name: Meta checks | |||
| runs-on: ubuntu-latest | |||
| env: | |||
| cabal_build: >- | |||
There was a problem hiding this comment.
Wow, that's the next level... I'm always worried about putting too much intelligence in the action code. Could this be a Makefile target too? (I see you made some updates to the Makefile).
I know, I'm stretching it, but some specification of what's supposed to happen inside the "quick jobs" workflow may help in future maintenance... Maybe a comment at the top of the file if not a whole section in CONTRIBUTING.md... OTOH it's always impossible to keep it up to date with the code.
There was a problem hiding this comment.
Yes, I am aware that I am breaking the abstraction that the Makefile should offer, but I didn't see a way around it.
What I need here is a cache-key for some specific makefile goal(s), like e.g spdx templates. Putting this computation into the Makefile e.g. as spdx-templates-cache-hash would also be a bit awkward, and, I would need to invoke hashfiles from the Makefile (although this should be solvable).
I think we should document which goals of the Makefiles are in which workflows, so that these are changed in sync. E.g. the lexer goal was removed from the Makefile in #8980 but not from the workflow.
There was a problem hiding this comment.
Agreed. Future work then... Maybe a ticket.
This is the same as #9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner.
mergify
bot
added
the
merge delay passed
This is the same as #9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner.
The `lexer` target was removed in #8980
Not sure why this was not up to date on master and still CI passed. Maybe the content of this file is dependent on the GHC version we are using to build the `get-cabal-macros` tool?
This is the same as #9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner.
This is the same as #9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner.
This is the same as #9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner.
|
@andreasabel do you know if any of these fixes (or ones from #9849) can also be applied to the main workflow (validate) and worth it? |
This is the same as haskell#9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner.
This is the same as haskell#9845 but for the changelogs.yml workflow. `changelog-d` currently has restrictive bound `base < 4.19` which we need to ignore if we want to build on the latest GHC as shipped by the GHA runner. (cherry picked from commit af5d606)
|
CI-related backport |
|
@mergify backport 3.12 |
✅ Backports have been created
|
|
Just a note for future us: it'd be helpful to keep the title of an issue in sync with its content. I just lost a day in CI hell because I wouldn't guess that the title "fix caching" is attached to a PR fixing actual CI failures like this one: https://github.com/haskell/cabal/pull/9845/files#diff-1d32b38ff76b89f854ace50e412f5b4bf3cadca31080b2d3104ef6937f7cccb7R5-R10 |
…9845) (#10041) * Fix #9815: switch quick-jobs CI to XDG Fix #9815: - Cache `~/.local/state/cabal` instead of `~/.cabal/store` - `~/.local/bin` is used instead of `~/.cabal/bin` and is already in the PATH (verify this by calling `alex` after installing it) As I am passing by: - bump cache action to v4 - double-quote `$USER` to keep actionlint happy - move `if` from shell-level to job-level - allow newest `alex` (cherry picked from commit e916cb5) * CI quick-jobs: use preinstalled GHC and Cabal (cherry picked from commit c209a82) * Makefile: remove dead target 'lexer', use '.PHONY' systematically The `lexer` target was removed in #8980 (cherry picked from commit e600087) * CI "Meta checks": correct cache key (cherry picked from commit 56426e4) * CI "Meta checks": print Haskell versions (cherry picked from commit 9a311bd) * CI "Doctest Cabal": daily refresh of cache (cherry picked from commit ba6f6ff) * CI "Check Field Syntax Reference": correct cache key (cherry picked from commit 5949e3f) * Update generated Cabal/src/Distribution/Simple/Build/Macros/Z.hs Not sure why this was not up to date on master and still CI passed. Maybe the content of this file is dependent on the GHC version we are using to build the `get-cabal-macros` tool? (cherry picked from commit 947860a) * CI quick-jobs: entirely wipe ghcup directory rights workaround (cherry picked from commit 5aa8afd) * !fixup --------- Co-authored-by: Andreas Abel <[email protected]> Co-authored-by: Artem Pelenitsyn <[email protected]>
Fix #9815:
~/.local/state/cabalinstead of~/.cabal/store~/.local/binis used instead of~/.cabal/binand is already in the PATH(verify this by calling
alexafter installing it)As I am passing by:
$USERto keep actionlint happyiffrom shell-level to job-levelalexUPDATE: This PR has been extended to
lexergoal in theMakefile(was removed in Add alex to build-tool-depends remove separate code gen #8980).PHONYsystematically inMakefile