Dependency issues with cabal repl and test-suites

[sol]

cabal repl still does not work properly for test suites. If the test suite depends on the library component (which is pretty common!), we run into dependency issues.

Steps to reproduce

Create the following files:

$ tree
.
├── foo.cabal
├── src
│   └── Foo.hs
└── test
    └── Spec.hs

2 directories, 3 files

-- foo.cabal
name:           foo
version:        0.0.0
build-type:     Simple
cabal-version:  >= 1.8

library
  hs-source-dirs: src
  exposed-modules: Foo
  build-depends: base, process

test-suite spec
  type: exitcode-stdio-1.0
  hs-source-dirs: test
  main-is: Spec.hs
  build-depends: base, foo, hspec

-- src/Foo.hs
module Foo where

import System.Process

foo :: Int
foo = 42

-- test/Spec.hs
module Main (main) where
import           Test.Hspec
import           Foo

main :: IO ()
main = hspec $ do
  describe "foo" $ do
    it "it is 42" $ do
      foo `shouldBe` 42

Running the specs with cabal test works, but running cabal repl spec fails:

$ cabal repl spec
src/Foo.hs:3:8:
    Could not find module ‘System.Process’
    It is a member of the hidden package ‘process-1.2.0.0’.
    Perhaps you need to add ‘process’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
Failed, modules loaded: none.
Prelude> 

[sol]

Trying this again, I can not reproduce it anymore. I would close this and wait if we get more reports like this.

[MaxGabriel]

I just ran into this doing cabal repl test for this project. Haven't investigated much yet.

Edit: I couldn't repro using @sol's repro steps though, with or without a sandbox

Version info

• GHC:

> ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.8.3

• Cabal:

> cabal --version
cabal-install version 1.22.0.0
using version 1.22.0.0 of the Cabal library

• Using a sandbox

[jsl]

Hi @MaxGabriel - we can re-open this or create a new issue if this problem still occurs, however I was able to get cabal repl test to work on that project after a couple of minor changes that I pushed to a fork of your project.

I don't have the whole set of output when I was testing that locally since I cleared my buffer, but essentially the problems I fixed were:

1. Delete the cabal.config since the base constraint didn't match what I had on my system - I just let cabal create a new one automatically
2. Then I saw a bunch of errors cabal repl test was trying to access packages that were hidden (they were probably included by other deps in the test suite, but not accessible because not explicitly listed in the test section). This was fixed by copying the complete set of dependencies from the library section of the cabal file to the test section of the cabal file.

Do the changes I made in my repo make sense to you to fix this issue, or is there still something wrong? I'd appreciate if you could paste all of your build output, perhaps in the form of a new ticket if there is a different problem you're still seeing.

[MaxGabriel]

@jsl Oh, I thought from @sol's issue that it wasn't supposed to be necessary to add the packages from the main project to the build-depends section of the test project to use cabal repl test. If it's expected that you need to add the packages to build-depends for the test project then there's no issue.

[MaxGabriel]

I made the complete output of cabal repl test available as a gist; good idea @jsl.

The output of cabal test is just:

Max@maximilians-mbp ~/D/N/t/hackvote-yesod> cabal test
Preprocessing library HackVote-0.0.0...
In-place registering HackVote-0.0.0...
Preprocessing test suite 'test' for HackVote-0.0.0...
Running 1 test suites...
Test suite test: RUNNING...
Test suite test: PASS
Test suite logged to: dist/test/HackVote-0.0.0-test.log
1 of 1 test suites (1 of 1 test cases) passed.

I can reproduce this issue in the scaffolding for yesod as well; there may be an issue with how that project is setup.

[jsl]

@MaxGabriel thanks for providing the complete output.

It may be that it's possible to avoid copying the dependencies, but do you have any links to where that feature is described? It's very possible I've missed something in a recent cabal, but I've always copied the dependencies in my projects between the libs and test sections.

If we determine that you should be able to get away without copying dependencies between test and lib sections in the cabal file, having cabal test repl not respect that does seem like a bug to me, and let's re-open this ticket.

[jonsterling]

At PivotCloud, we have reproduced this with an executable target that depends on the library defined in the same cabal file.

At this point, we have had to switch to using ghci with some various options, since cabal repl is sadly pretty unpredictable at this point.

Has anyone else experienced this for executable targets? It might be that this bug, when it occurs, affects all non-library components which depend on the main library defined in the package.

[ezyang]

Can anyone associated with this ticket give up-to-date instructions on how to reproduce this, with the latest version of Cabal? That would be very helpful!

[Blaisorblade]

I think the original issue is still relevant with the latest version of Cabal. Namely, to use test libraries with cabal repl for the testsuite, one needs to add the test libraries as dependencies for the main library. I'll confess I've last experienced it with stack ghci though and haven't debugged this; I could try investigating.

[ezyang]

Please do! The most useful way you can test is by actually creating a test case in one of the Cabal test suites; but clear repro instructions are also good enough (and I can make the test.)

[Blaisorblade]

OK, the original issue appears fixed, but I suspect I know why it still doesn't help in the emacs modes: they don't realize they need to load a separate REPL for the testsuite. I mention this because that's why I taught cabal repl had a bug. (I last tested this with stack's cabal-mode, sorry, should try rechecking with the original emacs mode). OTOH, a separate REPL does sound awful in terms of memory consumption.

Another issue in this ticket (EDIT: see #2032 (comment)) was about copying library dependencies to the testsuite. But as far as I can tell, this does not work currently (my testcase below). Is there we have a feature request for that? If not, below's a testcase for what I'd like to work. That should be genuinely useful for Cabal.

I'm not sure about @jonsterling's issue, but that seems yet a separate ticket.

However, I should probably reproduce this more cleanly in a cabal sandbox. I was impatient and had hspec already built by stack (I confess), so I hacked a cabal.config to reuse stack's package db (here be dragons, see last section).

Test libraries not available in cabal repl

Both cabal repl spec and cabal repl test/Spec.hs work, but :load test/Spec.hs in cabal repl does not work, and that's the kind of thing that the emacs modes do. But that's probably bad.

$ cabal repl spec
Preprocessing library foo-0.0.0...
In-place registering foo-0.0.0...
Preprocessing test suite 'spec' for foo-0.0.0...
GHCi, version 7.10.3: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Main             ( test/Spec.hs, interpreted )
Ok, modules loaded: Main.
$ cabal repl test/Spec.hs
Warning: Ignoring 'test/Spec.hs. The whole test suite 'spec' will be built.
(Support for module and file targets has not been implemented yet.)
Preprocessing library foo-0.0.0...
In-place registering foo-0.0.0...
Preprocessing test suite 'spec' for foo-0.0.0...
GHCi, version 7.10.3: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Main             ( test/Spec.hs, interpreted )
Ok, modules loaded: Main.
$ cabal repl
Preprocessing library foo-0.0.0...
GHCi, version 7.10.3: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Foo              ( src/Foo.hs, interpreted )
Ok, modules loaded: Foo.
*Foo> :load test/Spec.hs

test/Spec.hs:2:18:
    Could not find module ‘Test.Hspec’
    It is a member of the hidden package ‘hspec-2.2.3@hspec_CYWZadvwAraHdF11VslegI’.
    Perhaps you need to add ‘hspec’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
Failed, modules loaded: none.

Testcase for needing to copy libraries

This testcase doesn't build, because somehow transitive dependencies of the library aren't available in the testsuite.

$ cabal test
./foo.cabal has been changed. Re-configuring with most recently used options.
If this fails, please run configure manually.
Resolving dependencies...
Configuring foo-0.0.0...
Preprocessing library foo-0.0.0...
In-place registering foo-0.0.0...
Preprocessing test suite 'spec' for foo-0.0.0...

test/Spec.hs:4:8:
    Could not find module ‘System.Process’
    It is a member of the hidden package ‘process-1.2.3.0@proce_52AgREEfSrnJLlkGV9YZZJ’.
    Perhaps you need to add ‘process’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.

foo.cabal:

name:           foo
version:        0.0.0
build-type:     Simple
cabal-version:  >= 1.8

library
  hs-source-dirs: src
  exposed-modules: Foo
  build-depends: base, process

test-suite spec
  type: exitcode-stdio-1.0
  hs-source-dirs: test
  main-is: Spec.hs
  --build-depends: base, process, foo, hspec # works
  build-depends: foo, hspec #Feel free to readd base (O(1) deps), but not process (O(N) deps).

test/Spec.hs

module Main (main) where
import           Test.Hspec
import           Foo
import System.Process

main :: IO ()
main = hspec $ do
    describe "foo" $ do
          it "it is 42" $ do
                  foo `shouldBe` 42

The rest is copied from the OP's testcase, so:

-- src/Foo.hs
module Foo where

import System.Process

foo :: Int
foo = 42

$ tree
.
├── foo.cabal
├── src
│   └── Foo.hs
└── test
    └── Spec.hs

2 directories, 3 files

Reusing stack's DB as I did in my testing

That's unsupported, and one shouldn't even install in this config. But here's what I did.

$ stack exec bash
$ echo $GHC_PACKAGE_PATH
/Users/pgiarrusso/Documents/Research/Sorgenti/cabal-test/.stack-work/install/x86_64-osx/lts-6.6/7.10.3/pkgdb:/Users/pgiarrusso/.stack/snapshots/x86_64-osx/lts-6.6/7.10.3/pkgdb:/Users/pgiarrusso/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d
$ unset GHC_PACKAGE_PATH # cabal-install is unhappy otherwise
$ cat cabal.config
user-install: False
package-db: /Users/pgiarrusso/.stack/snapshots/x86_64-osx/lts-6.6/7.10.3/pkgdb
package-db: /Users/pgiarrusso/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d

[ezyang]

Thank you for the detailed report. Let me address these in reverse order.

Testcase for needing to copy libraries

This is what you'd expect: the dependencies of a test suite are independent of the dependencies of the library. This behavior is justified by analogy to how library dependencies work: if a library p depends on process, just adding build-depends: p to another package doesn't bring process's modules into scope.

Of course, the code in a test suite is often closely related to the code for the library, so having some way to define a common stanzas between the two would be useful (#2832) (and empirically, probably no one would lose sleep if test suites just inherited the dependencies of the library a priori). Unfortunately, that feature request has languished due to a lack of someone to helm it. My understanding is also that hpack was at least partially motivated in making this case nicer for users (and it supports older Cabal versions, which is not something any fix to #2832 can do).

There is one other thing to know about here: there is a common design pattern, especially for executable only packages, for the test suite to contain a "copy" of the library/executable code. This SO question and in fact cabal-install itself are both examples of this pattern. When you look at the file system there is only one copy of the source file, true, but when you build the actual executable and test the module is compiled twice, and those two compilations really are different. This can lead to puzzling errors sometimes (though I don't have a link ready.)

Both cabal repl spec and cabal repl test/Spec.hs work, but :load test/Spec.hs in cabal repl does not work, and that's the kind of thing that the emacs modes do. But that's probably bad.

OK. I'll also add that if you cabal repl spec, then :load test/Spec.hs works.

This behavior is "by design" but I'm inclined to think the design is wrong in some non-obvious way. Here's what's going on: the design of Cabal's REPL is that it always operates in the context of some particular component. This is because GHCi doesn't know how to handle loading files from multiple packages (https://ghc.haskell.org/trac/ghc/ticket/10827). As a consequence of this, whichever component you pick, only the build-depends of THAT component is brought into scope for the REPL (see my answer above). So for any component you pick, you can only interactively work with files from THAT component.

So what it seems to me is that people want a REPL mode that, at the very least, unions together the build dependencies of ALL configured components, with the idea that if you load any module, all of its dependencies will be in scope. Obviously there are some corner cases where this will not work (e.g., if the library and the test suite have differing dependencies that bring into scope conflicting module names) but by and large it should work. In effect, we are making a fake, "uber-package" which contains all of the source files for all of the components, to work around GHCi's lack of support for handling multi-package things. It will break whenever your code actually needed to have things put in separate packages but it might still be useful.

Perhaps there is some recommendation we can give the haskell-modes so that they know which repl to start up? One heuristic is to look at the hs-source-dirs of components to determine what component any given file is likely to be in, and load the repl for that component. And yes you would have to have multiple repls open.

[ulysses4ever]

Here's a summary of issues mentioned in this ticket.

OP is fixed: checked by #2032 (comment) and myself. I suggest we should close the ticket and open new ones if there's a need. The candidates for new ones are listed in #2032 (comment), and I'll be quoting it

Test libraries not available in cabal repl

Both cabal repl spec and cabal repl test/Spec.hs work, but :load test/Spec.hs in cabal repl does not work, and that's the kind of thing that the emacs modes do. But that's probably bad.

The author admits this is bad, so let's not pursue it.

[ulysses4ever]

Sorry, sent the previous message accidentally.

Testcase for needing to copy libraries

This testcase doesn't build, because somehow transitive dependencies of the library aren't available in the testsuite.

This is no surprise and works as expected. In the example provided, the test component depends on process but doesn't declare it in build-depends, hence fails. So, no-bug.

Reusing stack's DB as I did in my testing

That's unsupported, and one shouldn't even install in this config.

This doesn't sound like a proposal for a ticket/feature-request, but if anyone keen on it, they could open a new ticket about it, I guess.

---

Bottom line: I think we can close this ticket.

[Blaisorblade]

FWIW I'm not the original author (since you're replying to #2032 (comment)), but I haven't checked on this bug in year so I can't stand in the way.

Some points from @ezyang 's response (#2032 (comment)) might deserve a ticket, but somebody (not me) would have to take a look. In particular:

This behavior is "by design" but I'm inclined to think the design is wrong in some non-obvious way. [...] So what it seems to me is that people want a REPL mode that, at the very least, unions together the build dependencies of ALL configured components, with the idea that if you load any module, all of its dependencies will be in scope.

Anyway, tentatively closing. (Feel free to reopen, but I should best unsubscribe either way).