GHC 8.8.4 fails sanity checks during install due to Cabal-3.0.1.0 being hidden

[spinnylights]

General summary/comments

The LTS resolvers for GHC 8.8.4 (currently 16.12–16.17) fail to install GHC 8.8.4 in my environment because GHC complains about Cabal 3.0.1.0 being hidden when trying to build the sanity checks. This is the output I get when running stack setup in a freshly-created Stack project:

$ stack setup
Preparing to install GHC (tinfo6) to an isolated location.
This will not interfere with any system-level installation.
Downloaded ghc-tinfo6-8.8.4.
Installed GHC.
The GHC located at /home/zoe/.stack/programs/x86_64-linux/ghc-tinfo6-8.8.4/bin/ghc-8.8.4 failed to compile a sanity check. Please see:

    http://docs.haskellstack.org/en/stable/install_and_upgrade/

for more information. Exception was:
Received ExitFailure 1 when running
Raw command: /home/zoe/.stack/programs/x86_64-linux/ghc-tinfo6-8.8.4/bin/ghc-8.8.4 /tmp/stack-sanity-check-28c750373cb78094/Main.hs -no-user-package-db
Run from: /tmp/stack-sanity-check-28c750373cb78094/
Standard output:

[1 of 1] Compiling Main             ( /tmp/stack-sanity-check-28c750373cb78094/Main.hs, /tmp/stack-sanity-check-28c750373cb78094/Main.o )
Standard error:

Loaded package environment from /home/zoe/.ghc/x86_64-linux-8.8.4/environments/default

/tmp/stack-sanity-check-28c750373cb78094/Main.hs:1:1: error:
    Could not load module âDistribution.Simpleâ
    It is a member of the hidden package âCabal-3.0.1.0â.
    You can run â:set -package Cabalâ to expose it.
    (Note: this unloads all the modules in the current scope.)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
  |
1 | import Distribution.Simple
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^

This is in an up-to-date Arch Linux environment. Neither the nightly resolver nor the latest LTS for 8.8.3 (16.11) turn up this issue when I run stack setup. Cabal-3.0.1.0 is present in ~/.stack/programs/x86_64-linux/ghc-tinfo6-8.8.4/lib/ghc-8.8.4.

Steps to reproduce

1. Create a new Stack project via stack new.
2. cd into the project directory.
3. Run stack setup.

Expected

I expected to see Installed GHC followed by the stack will use a sandboxed GHC it installed etc. message.

Actual

See here for output from stack setup --verbose.

Stack version

$ stack --version
Version 2.3.3, Git revision cb44d51bed48b723a5deb08c3348c0b3ccfc437e x86_64 hpack-0.33.0

Method of installation

• Official binary, downloaded from get.haskellstack.org to ~/.local/bin

[hseg]

Trying things out a bit, found that the presence of ~/.ghc/$build/environments/default is responsible for the sanity check failure: despite the -B"$topdir" flag in bin/ghc-$version, the environment set there is picked up. If it has a package-db directive, it clears the one given by the script.
This can be checked by running it under bash -x.
Curiously, stack exec -- ghc /tmp/Main.hs -no-user-package-db works.
(side note: I don't see -B documented anywhere, what does it do?)

[hseg]

Luckily, it doesn't break if package-db is set by GHC_PACKAGE_PATH. so a workaround is to delete ~/.ghc/*/environments/default and set any package-db directives by environment variable instead.

[qrilka]

@hseg ~/.ghc doesn't look like a directory controlled by Stack and for example I don't have any files like that locally. Maybe your question should go to a tracker of some other tool? Maybe Cabal or ghcup?

[hseg]

You misunderstand me. Sure, `~/.ghc` isn't _controlled_ by stack. But it still influences stack's operation where it shouldn't. Unless stack's portability guarantees are only in the absence of such configuration? Because it was my understanding that stack ignores the local configuration, preferring to eg install its own ghc whose settings it manages. On March 18, 2021 4:08:48 PM GMT+02:00, Kirill Zaborsky ***@***.***> wrote: ***@***.*** `~/.ghc` doesn't look like a directory controlled by Stack and for example I don't have any files like that locally. Maybe your question should go to a tracker of some other tool? Maybe Cabal or ghcup?

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #5412 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[qrilka]

Sorry, I'm misinterpreted you. And this is a feature of GHC - https://downloads.haskell.org/ghc/8.8.4/docs/html/users_guide/packages.html#package-environments
Stack a workaround for this -

stack/src/Stack/Setup.hs

Line 322 in 60fee51

Map.insert "GHC_ENVIRONMENT" "-"

and it was present in 2.3.3 already so maybe there is some code path no unsetting these environments

[spinnylights]

Unfortunately the environment in which I encountered this issue is now gone (I've switched back to Gentoo), but looking in the backup of my home folder, this is the contents of ~/.ghc/x86_64-linux-8.8.4/environments/default:

clear-package-db
global-package-db
package-db /home/zoe/.cabal/store/ghc-8.8.4/package.db
package-id ghc-8.8.4
package-id bytestring-0.10.10.1
package-id unix-2.7.2.2
package-id base-4.13.0.0
package-id time-1.9.3
package-id hpc-0.6.0.3
package-id filepath-1.4.2.1
package-id process-1.6.9.0
package-id array-0.5.4.0
package-id integer-gmp-1.0.2.0
package-id containers-0.6.2.1
package-id ghc-boot-8.8.4
package-id binary-0.8.7.0
package-id ghc-prim-0.5.3
package-id ghci-8.8.4
package-id rts
package-id terminfo-0.4.1.4
package-id transformers-0.5.6.2
package-id deepseq-1.4.4.0
package-id ghc-boot-th-8.8.4
package-id pretty-1.1.3.6
package-id template-haskell-2.15.0.0
package-id directory-1.3.6.0
package-id text-1.2.4.0
package-id random-1.2.0-28a5c0c3758e837f482d16b6e3f147535abec91165664e57cecc40c547f76afe
package-id split-0.2.3.4-5a5e27ee035c1ff3330632a356f1c8da25930d22ce1a49c9942543470bdfe35f

I didn't write that file myself and am not entirely sure how it got there. There seem to be a number of bug reports floating around about issues caused by it (cabal bug #5559 appears to be the most central one). The only other file in ~/.ghc (that isn't a directory) is ~/.ghc/ghci_history which I imagine is benign in this scenario.

I can't find any obvious way that GHC_ENVIRONMENT would have been set contrary to what Stack is doing. There was nothing sourced in my shell from my home directory or /etc that involved that environment variable, as far as I can see. I wasn't doing anything unusual—at one point I was using ghcup (and then all I was doing was sourcing ~/.ghcup/env in my shell on login) but I later switched to using only the official Arch packages. I'm not sure at what point I hit this bug relative to that.

[m4lvin]

I ran into this error. The file ~/.ghc/x86_64-linux-8.10.7/environments/default got created when I did cabal install --lib matrix. The content was:

clear-package-db
global-package-db
package-db /home/malvin/.cabal/store/ghc-8.10.7/package.db
package-id ghc-8.10.7
package-id bytestring-0.10.12.0
package-id unix-2.7.2.2
package-id base-4.14.3.0
package-id time-1.9.3
package-id hpc-0.6.1.0
package-id filepath-1.4.2.1
package-id process-1.6.13.2
package-id array-0.5.4.0
package-id integer-gmp-1.0.3.0
package-id containers-0.6.5.1
package-id ghc-boot-8.10.7
package-id binary-0.8.8.0
package-id ghc-prim-0.6.1
package-id ghci-8.10.7
package-id rts
package-id terminfo-0.4.1.4
package-id transformers-0.5.6.2
package-id deepseq-1.4.4.0
package-id ghc-boot-th-8.10.7
package-id pretty-1.1.3.6
package-id template-haskell-2.16.0.0
package-id directory-1.3.6.0
package-id text-1.2.4.1
package-id matrix-0.3.6.1-22742c7b72af847e0183b006f77cbb29e8bc1198d189c727687cdd659c4b1b8b

Deleting the file (or the whole folder .ghc) made stack setup work again 🥳

But I would agree with @qrilka that this is a bug in stack: stack setup is depending on configuration files it does not control and that intuitively are not related to it at all. I only found this solution by googling "stack setup sanity check Distribution.Simple" and landing here.

[lylek]

This has bitten me several times lately. It turns up when using HLS in VS Code because it calls stack setup --silent in order to try to find the GHC version of the Stack project. This is very confusing abstraction-breaking and would be especially troublesome for beginners.

Yes, deleting .ghc works, but it will only work until you do another cabal install - then it will recreate it. And by the time that happens, you've forgotten what caused the problem and what the solution was. So it's not a very good workaround.

Another workaround is to add a package-id line to that environments/default file with the currently installed version of Cabal. But I don't know how long that would last, either.

I also agree with @qrilka that the preferred solution is for stack not to rely on configuration files it does not control.

[mpilgrem]

If I follow the above correctly, the problem is that when Stack does its sanity check on the GHC it has fetched:

-- | Perform a basic sanity check of GHC
sanityCheck :: (HasProcessContext env, HasLogFunc env)
            => Path Abs File -> RIO env ()
sanityCheck ghc = withSystemTempDir "stack-sanity-check" $ \dir -> do
  let fp = toFilePath $ dir </> relFileMainHs
  liftIO $ S.writeFile fp $ T.encodeUtf8 $ T.pack $ unlines
    [ "import Distribution.Simple" -- ensure Cabal library is present
    , "main = putStrLn \"Hello World\""
    ]
  logDebug $ "Performing a sanity check on: " <> fromString (toFilePath ghc)
  eres <- withWorkingDir (toFilePath dir) $ proc (toFilePath ghc)
    [ fp
    , "-no-user-package-db"
    ] $ try . readProcess_
  case eres of
    Left e -> prettyThrowIO $ GHCSanityCheckCompileFailed e ghc
    Right _ -> pure () -- TODO check that the output of running the command is
                       -- correct

it does not pass the GHC flag -hide-all-packages and so GHC looks for a package environment in $HOME/.ghc/arch-os-version/environments/default, if it exists.

If -hide-all-packages is passed, then all the packages required need to be specified with GHC -package options. I think those packages are Cabal and base.

[mpilgrem]

I think pull request #6137 will fix this, but I would appreciate it if somebody who has experienced the problem could test that it is a cure.

[lylek]

I think pull request #6137 will fix this, but I would appreciate it if somebody who has experienced the problem could test that it is a cure.

I tried your pull request, and it worked. See the PR for more details. Thanks for the fix!