Compile HLS from source

[tfausak]

An alternative to #14 that actually works, unlike #15. Fixes #12. Also fixes #13.

---

• I manually tested the code locally to make sure that it works.
• If there is documentation, I made sure it's accurate and up to date.
• If possible, I wrote automated tests for the new behavior.

[tfausak]

Weirdly everything except GHC 9.4.3 succeeded.

[tfausak]

Aha! I was interpreting exit status 9 as an out of memory error, but actually it's a package that's failing to compile: https://github.com/acilearning/docker-haskell/actions/runs/3448340819/jobs/5755277198#step:3:2175

Error: cabal: Failed to build ghc-exactprint-1.6.0 (which is required by
exe:haskell-language-server-wrapper from haskell-language-server-1.8.0.0 and
exe:haskell-language-server from haskell-language-server-1.8.0.0). See the
build log above for details.

Indeed, ghc-exactprint-1.6.0 does not support GHC 9.4.3. But version 1.6.1 does! alanz/ghc-exactprint#117

I'm not sure how to convince the build to use 1.6.1 instead of 1.6.0. I figured that it would use the latest allow version anyway.

[tfausak]

It's trying to use an older version of ghc-exactprint because HLS pins the index state: https://github.com/haskell/haskell-language-server/blob/385dd1b57af75ccd286e104b7e00f2b0cb626588/cabal.project#L53

[tfausak]

This branch produces an image that is about twice as big as the one from the main branch:

• main: 632 MB public.ecr.aws/acilearning/haskell:9.2.4-08f0de6f7cf16cb1707e97ce43370b0649faf1fd
• This branch: 1.11 GB public.ecr.aws/acilearning/haskell:9.2.5-f9048e692cd00062f033f56346da4c9b0ae3371e

It's possible that this could be reduced, but we may run into problems like we did before: #8 (comment)

[tfausak]

This doesn't work if you mount a volume at /cabal-store. That's because HLS needs shared libraries from there:

/home/haskell/.ghcup/bin/haskell-language-server-9.2.5: error while loading shared libraries: libHShaskell-language-server-1.8.0.0-f210c2f18ddcb783f54b60d2d25ac31984ac496b9bc2f3d7db9fda0c7f35d266-ghc9.2.5.so: cannot open shared object file: No such file or directory

$ docker run \
  --interactive \
  --rm \
  --tty \
  --volume $PWD/cabal-store:/cabal-store \
  public.ecr.aws/acilearning/haskell:9.2.5-f9048e692cd00062f033f56346da4c9b0ae3371e \
  sh -c 'ldd /home/haskell/.ghcup/bin/haskell-language-server-9.2.5 | grep "not found" | wc -l'
226

[tfausak]

I just tested with a simple project and everything seems to work!

The latest image (public.ecr.aws/acilearning/haskell:9.2.5-9e6f4e3e4bc082c549f3493c4e29d2d0c8afe035) is 883 MB. I'm not sure why that's so much bigger than using the pre-built HLS executable. But I'm just happy it works.

[codygman]

This doesn't work if you mount a volume at /cabal-store. That's because HLS needs shared libraries from there:

/home/haskell/.ghcup/bin/haskell-language-server-9.2.5: error while loading shared libraries: libHShaskell-language-server-1.8.0.0-f210c2f18ddcb783f54b60d2d25ac31984ac496b9bc2f3d7db9fda0c7f35d266-ghc9.2.5.so: cannot open shared object file: No such file or directory

$ docker run \
  --interactive \
  --rm \
  --tty \
  --volume $PWD/cabal-store:/cabal-store \
  public.ecr.aws/acilearning/haskell:9.2.5-f9048e692cd00062f033f56346da4c9b0ae3371e \
  sh -c 'ldd /home/haskell/.ghcup/bin/haskell-language-server-9.2.5 | grep "not found" | wc -l'
226

I can think of two solutions for this:

1. Don't use a docker volume for cabal store
2. build the cabal store ghcup compile hls generates for HLS only

The second options I'm slightly suspicious of, but think it would be fine.

I'm not sure if you consider 2 an option @tfausak, thoughts?

As for 1, I'm not sure if it's possible.

I'll ask in haskell/haskell-language-server#3345 (comment).

[codygman]

I don't believe that 2 is an option since the cabal store is supposed to be a singular thing.

conclusion: HLS with template Haskell is only compatible with "fat" docker images and not "thin" docker images which mount an external volume for the cabal store

[codygman]

It was recommended to create our own custom bindist like hls does but for our docker container.

I've never created a bindist, but here are relevant links on how hls does that:

https://github.com/haskell/haskell-language-server/blob/master/GNUmakefile
https://github.com/haskell/haskell-language-server/tree/master/bindist
https://github.com/haskell/haskell-language-server/blob/master/bindist/ghcs

[tfausak]

I'm not necessarily opposed to making our own bindist, but it seems like potentially a lot of work.

[codygman]

Yeah, I don't really want to do it because it seems like a lot of work and kind of complex to maintain.

However the only alternatives I see to resolve this situation are:

1. Move back to "fat" images with cabal libraries included
2. Don't support HLS

We've discussed 2 a few times and it's not a popular suggestion. I lean towards 1 maybe mostly because of familiarity, but I remember you mentioning how 1 was a pain to maintain and that you like thin images more.

[tfausak]

It would be especially hard to support fat images given that we provide images for multiple versions of GHC.

Another option is to install HLS with a separate Cabal store directory. That's probably worse than making a custom bindist, but it's also easier. I think it would be something like cabal --store-dir /cabal-store-hls instal ....

We could also stick with versions of GHC that HLS has official support for. Currently that would be 9.2.4 and 9.4.2. (It's not clear to me why HLS doesn't release more frequently. haskell/haskell-language-server#3355)

[codygman]

We could also stick with versions of GHC that HLS has official support for. Currently that would be 9.2.4 and 9.4.2. (It's not clear to me why HLS doesn't release more frequently. haskell/haskell-language-server#3355)

I can't remember if 9.2.4 actually worked or had the same problems.

I'll go back to 9.2.4 and see if things work.