Make paths of cabal, ghc available to tests?

[jrp2014]

I have a test suite that runs compiles up some tests the results of which are processed by tasty-golden.

To invoke the compiler, I trying to use cabal exec.

   63 -- | Use `cabal exec` to run the compilation, so that the smuggler plugin is
   64 -- picked up from the local database.  GHC alone would use the global one.
   65 compile :: FilePath -> Options -> IO ()
   66 compile testcase opts = do
   67     -- env <- getEnvironment
   68    --  print env
   69     cabalPath <- lookupEnv "CABAL" -- find, eg, @/opt/ghc/bin/cabal@ or @cabal -vnormal+nowrap@
   70     let cabalCmd = words $ fromMaybe "cabal" cabalPath -- default to "cabal" if @CABAL@ is not set
   71     let cabalConfig = setEnvInherit $ proc (head cabalCmd) (tail cabalCmd ++ cabalArgs) :: ProcessConfig () () ()
   72     -- print cabalConfig
   73     runProcess_ cabalConfig
   74   where
   75     cabalArgs :: [String]
   76     cabalArgs =
   77       -- - not sure why it is necessary to mention the smuggler package explicitly,
   78       --   but it appears to be hidden otherwise.
   79       -- - This puts the .imports files that smuggler generates somewhere they
   80       --   can easily be found
   81       ["exec", ghc, "--", "-package smuggler", "-v0", "-dumpdir=" ++ testDir, "-fno-code", "-fplugin=Smuggler.Plugin"]
   82         ++ map
   83           ("-fplugin-opt=Smuggler.Plugin:" ++)
   84           ( let ia = importAction opts
   85                 ea = exportAction opts
   86                 p = [show ia, show ea]
   87              in case newExtension opts of
   88                   Nothing -> mkExt ia ea : p -- provide a default extension
   89                   Just e -> e : p
   90           )
   91         ++ [testcase]

The only way I have found of getting hold of cabal is to amend the .travis.yml to export the CABAL and do a bit of processing.

Is there a reason that it is not exported in the haskell-ci-generated original? Is there a better way to pick up the cabal that is being used, or indeed to compile the test cases otherwise?

I have tried to use GHC.paths to try to get hold of ghc (for line 81), but that doesn't seem to work. I just get cabal: The program 'ghc' version >=7.0.1 is required but it could not be found., but perhaps that is just cabal complaining that it can't find a ghc to use for its own purposes (which could be fixed by a -w flag, if I could get hold of the location of ghc from within this test harness). Suggestions for how best to get hold of the rightghc within the test harness welcome!

PS:
Near the start of the job, I get a, probably harmless, diagnostic:

found cache
adding /home/travis/.cabal/packages to cache
creating directory /home/travis/.cabal/packages
adding /home/travis/.cabal/store to cache
creating directory /home/travis/.cabal/store
adding /home/travis/.hlint to cache
creating directory /home/travis/.hlint
$ ghc-8.8.3 --version
ghc-8.8.3: command not found                                                  <<<<<<<<<<<<<<<<<<<<<<<<<
before_install.
$ HC=$(echo "/opt/$CC/bin/ghc" | sed 's/-/\//')

[kindaro]

I have a similar issue with haskell-ci version 0.16.6. I have written a test suite that needs to call the compiled executable, but neither cabal nor ghc are found on the path. This means that my test suite does not work.

A possible workaround would be to list Cabal and GHC in build-tool-depends but this would blow up the time needed to run the integration script.

I see that the integration script generated by haskell-ci calls these executables as $CABAL and $GHC. I can in principle tweak my test suite to access these environment variables, but this would needlessly complicate the code — these parameters will need to be passed all over.

I looked through the many command line flags haskell-ci --help advertises, but I cannot see anything that would help me.

[kindaro]

This problem can be solved by adding these lines:

       - name: Set PATH and environment variables
         run: |
           echo "$HOME/.cabal/bin" >> $GITHUB_PATH
+          echo "$HOME/.ghcup/bin" >> $GITHUB_PATH
+          "$HOME/.ghcup/bin/ghcup" set ghc "$HCVER"
           echo "LANG=C.UTF-8" >> "$GITHUB_ENV"
           echo "CABAL_DIR=$HOME/.cabal" >> "$GITHUB_ENV"

— Into the generated integration script. Maybe we can have a command line option for this?

[phadej]

You probably want to depend on https://hackage.haskell.org/package/ghc-paths with all the drawbacks it brings.

It's a bad practice to depend on compiler being called ghc and being in PATH.

(EDIT: or make a build-type: Custom script, like cabal-doctest to capture the build environment. And that is also bad approach, which I wouldn't recommend).

[kindaro]

ghc-paths can give me the name of the compiler. But I also need the cabal executable — the same that was used to build the package, ideally.

It's a bad practice to depend on compiler being called ghc and being in PATH.

Why? Where is this written? I do depend on this in my everyday work.

[kindaro]

You have not answered my question: maybe we can have a command line option for this? It does not seem like a particularly hard thing to implement. I guess I can draft a patch. It seems you are vaguely displeased but being vaguely displeased is not a constructive way to solve problems and I do not think we have exhausted the conversation yet.

[phadej]

it does not seem like a particularly hard thing to implement

That is irrelevant. Somethings are worth doing even they are hard, some easy things are not worth it. haskell-ci have gone a long way of not having ghc in the PATH (caughting bugs and accidental dependencies elsewhere), and I must be persuaded more strongly than just that "it's easy to do".

I don't know what you are doing, but needing to call ghc in a test-suite doesn't feel right. And even so, you really should use ghc-paths, to be sure to call the same GHC you are compiling with. If you don't care about breaking cabal build -w ghc-x.y.z workflows, it's fine; but I do, and haskell-ci too.

[kindaro]

Alright, so not having ghc on the path is a specific design decision. On top of that, there is a way to get it with ghc-paths.

But what about Cabal?

The specific case I have is that I want to run the built executable so that I can test it. The way I find the executable is by calling cabal list-bin. If there is no cabal on the path, then I am stuck.

Cabal sometimes wants to have ghc on the path. (I am not sure why, since the executable is supposedly already built.) Maybe I can ask ghc-paths where ghc is located and inform Cabal via the with-compiler command line option.

So maybe I only need Cabal.

[phadej]

The specific case I have is that I want to run the built executable so that I can test it.

That's cabal-install issue: haskell/cabal#7577.

[kindaro]

Alright, let me offer you a use case. Imagine you want to add automated checks to cabal-fmt that format some example package descriptions. You want to ensure that these example packages do in fact build after having been formatted. For this purpose, your automated checks will be calling cabal build inside the example folders, and cabal build will be calling ghc. How do you make this work with haskell-ci?

You can add Cabal to build-tool-depends and Cabal will make Cabal available. I tried this and it seems to work. However, it is a waste of time and electricity — you already have Cabal somewhere in the runner machine, and now Cabal is building another Cabal from scratch. Does this seem to you like elegant design?

[sgraf812]

happy's CI is affected by a similar issue as well. In order to run its testsuite, we need the happy executable, which in turn depends on seeing the correct happy_datadir environment variable to know where to look up the HappyTemplate.hs data-file.

At the moment, we work around it by applying the following patch to the generated GHA workflow:

https://github.com/haskell/happy/blob/635cc81b8722270bbae802844262807ab83c4ce0/.github/haskell-ci.patch

I had hoped that we could somehow get rid of this patch. From what I gather, that would only work if either

1. cabal test sets the correct happy_datadir path variables, or
2. instead of calling happy from the Makefile of the testsuite, call cabal exec -v0 happy instead (which sets the proper paths), or
3. the generated GHA workflow installs any executable components prior to running the testsuite of the sdist and makes it available on the $PATH.

It appears (1) is unlikely to be fixed: haskell/cabal#10072

(2) is not easily possible for lack of a reliable way to call cabal from the GHA workflow. It could easily fixed by exporting the $CABAL variable as in the linked patch. I would think that it a rather principled and clean solution.

(3) could be implemented by a flag offered by haskell-ci.

[phadej]

My stance is unchanged, make cabal test do the needed stuff, then haskell-ci setup (and all others) will work. There are multiple issues open to cabal-install about that, maybe eventually something will be implemented there.

The run-tool-depends is one option. (To run test-suite you need happy), but AFAIK build-tool-depends works in the meantime.

[sgraf812]

It appears I commented a few hour too early. I managed to derive the happy CI workflow with vanilla haskell-ci regenerate in haskell/happy#284. All that was needed was getting rid of the cleverness in figuring out where happy lives. It appears that build-tool-depends takes care of the rest, as advertised.