Build release artifacts in CI

[ezyang]

In #3755, it was mentioned that one barrier to more frequent releases was the difficulty of getting binaries for all platforms. I've started experimenting with having our CIs generate binaries for us. Here is one binary from AppVeyor (cut off the 2.0-staging branch):

• https://ci.appveyor.com/api/buildjobs/rqo8q29y9ivh9c6l/artifacts/cabal-install%2Fdist%2Fbuild%2Fcabal%2Fcabal.exe

[23Skidoo]

Nice. Note that for Windows and Linux we normally produce both a 32-bit and a 64-bit version.

[ezyang]

How do these 32-bit Windows binaries get built? 32-bit machine? "Cross" compile?

[23Skidoo]

Using the 32-bit HP on a Windows x64 machine. On Linux I used chroot + distro-bundled GHC.

[maoe]

It would be great if binaries were released on GitHub Releases on every tag or something like that.

[ezyang]

I had pretty good success for Linux builds, see: https://github.com/ezyang/cabal-release

[maoe]

Great!

Once this is set up for every major platform, I'd like to use it from the threadscope project. Currently we're using hvr's PPA for Linux, this build for macOS and stack for Windows. It would be great if we could use the same cabal-install for all the platforms.

[23Skidoo]

GHC is standardising on Jenkins for its CI setup, maybe we can piggyback on that effort?

[phadej]

@23Skidoo we should. That could solve other CI problems.

[23Skidoo]

I asked @bgamari about this today at HIW, and he said that it may be possible, conditional on the available server capacity. So if Cabal CI will increase the load on GHC Jenkins infrastructure too much, we may need to look for a source of additional funding to pay for that.

[andreabedini]

This should be easy with GitHub Actions, I am assigning myself (I know I will regret this).

[jneira]

@andreabedini i think there are plans to build artifacts in gitlab instead (see https://github.com/haskell/cabal/blob/master/.gitlab-ci.yml) but not sure about, maybe @Mikolaj could confirm

[Mikolaj]

We do build release artifacts in gitlab and I think there are reasons for this, e.g., that's the same setup that is used for GHC.

However, we don't build the artifacts for each PR, which is a shame. We should either set up additional CI on the gitlab CI or at least try to mimic as closely as possible the artifact building pipeline in GHA. Let's discuss, e.g., on Matrix/IRC or the call this Thursday.

[phadej]

FWIW, we did, but someone decided to not replicate how we built artifacts already but instead wrote a gitlab pipeline from scratch.

You should try to escape the vicious CI refactoring cycle.

[Mikolaj]

Do you think it may be better to builds the artifacts on GHA? If so, we can reopen that discussion (if there was one, I don't remember).

[phadej]

That is irrelevant. If https://github.com/haskell/cabal/blob/master/.gitlab/ci.sh script part which actually builds and packages weren't tied to GitLab, then using it also on GitHub would been trivial.

That why I made release.py back then, so it can be used anywhere, GitHub/GitLab/locally/... and was angry that GitLab CI setup doesn't use it, but reinvented the wheel.

[jneira]

i've just was finding out the release.py: https://github.com/haskell/cabal/blob/cabal-install-3.4.0.0/release.py
And tbf it had the advantage it was ci agnostic.

[jneira]

However, we don't build the artifacts for each PR, which is a shame. We should either set up additional CI on the gitlab CI or at least try to mimic as closely as possible the artifact building pipeline in GHA. Let's discuss

Even more, the artifacts to be released are not tested in ci so although the code is the same (modulo chosen commit errors) and the probabilities to introduce bugs in the specific build for gitlab is low, theys exist

[Mikolaj]

I see, so if we used an updated release.py, we could run it on gitlab at release time to match GHC architectures and on GHA on every Ci run?

@emilypi, @hasufell: do you remember why we haven't used release.py? No python on the gitlab machines? Anything else?

Perhaps somebody will remember during the Thursday devs call.

[hasufell]

do you remember why we haven't used release.py?

I don't understand why we need a 300 LOC python script to run cabal build --project-file=cabal.project.release.

If anyone wants to share code between github actions and gitlab CI, just create a shell script. I don't see any need for this, though.

[Mikolaj]

And what is the code that creates the artifacts on gitlab? Could you post a link?

[jneira]

https://github.com/haskell/cabal/blob/master/.gitlab/ci.sh and next ones, inputs are in https://github.com/haskell/cabal/blob/master/.gitlab-ci.yml

[Mikolaj]

Thank you.

@hasufell: I must have misinterpreted what you wrote. I expected to compare a single line cabal build --project-file=cabal.project.release vs 300 LOC, but what I see is (sets of) scripts of comparable size.

When you say "I don't see any need for this, though.", do you mean that you don't see a need to run tests on artifacts before packaging them, or that you don't see a need to run tests on binaries created in exactly the same way as artifacts in PR CI runs? Or both?

[hasufell]

must have misinterpreted what you wrote. I expected to compare a single line cabal build --project-file=cabal.project.release vs 300 LOC, but what I see is (sets of) scripts of comparable size.

I'm not sure which scripts you were looking at, but https://github.com/haskell/cabal/blob/master/.gitlab/ci.sh minus GHC installation is about 30 LOC.

When you say "I don't see any need for this, though.", do you mean that you don't see a need to run tests on artifacts before packaging them, or that you don't see a need to run tests on binaries created in exactly the same way as artifacts in PR CI runs? Or both?

I can't really follow. I don't see that the python script runs tests on the binary. It may certainly make sense to add tests to gitlab CI.

[andreabedini]

I will un-assign me for the moment. It doesn't look like we are on the same page, so I'd rather not keep this on my plate for now.

[jneira]

related with #6616
ok, actual state is we are using gitlab ci to build release artifacts so the original issue request is fullfilled, time to move forward and maybe open a new issue to track what changes are needed in gitlab for the next release