Unable to build with backpack modules from unpacked-containers package

[harendra-kumar]

Describe the bug

I have the following cabal file:

cabal-version:       2.0
name:                unpacked-example
version:             0.1.0.0
build-type:          Simple

library example
  default-language: Haskell2010
  hs-source-dirs: example
  exposed-modules: Int
  build-depends: base

executable unpacked-set-example
  default-language: Haskell2010
  main-is: example/Main.hs
  build-depends: base, unpacked-containers, example
  mixins: unpacked-containers (Set as Int.Set) requires (Key as Int)

And the following source files:

$ cat example/Int.hs 
module Int where

type Key = Int

$ cat example/Main.hs 
module Main where

import Int.Set

ten :: Set
ten = fromList [1..10]

main :: IO ()
main = print ten

I can't build it and cannot figure out what is wrong. Here is the cabal build message:

$ cabal v2-build
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - unpacked-containers-0 (lib with Key=unpacked-example-0.1.0.0-inplace-example:Int) (requires build)
 - unpacked-example-0.1.0.0 (exe:unpacked-set-example) (first run)
Starting     unpacked-containers-0 (lib)

Failed to build unpacked-containers-0. The failure occurred during the
configure step.
Build log (
/Users/harendra/.cabal/logs/ghc-8.8.3/npckd-cntnrs-0-8f491d38+B2PfsuvwtufFrJw0XqUoXM.log
):
Configuring library instantiated with
  Key = unpacked-example-0.1.0.0-inplace-example:Int
for unpacked-containers-0..
Warning: The package has an extraneous version range for a dependency on an
internal library: unpacked-containers -any && ==0. This version range includes
the current package but isn't needed as the current package's library will
always be used.
Error:
    The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
planned package unpacked-containers-0 is broken due to missing package unpacked-example-0.1.0.0-inplace-example

cabal: Failed to build unpacked-containers-0 (which is required by
exe:unpacked-set-example from unpacked-example-0.1.0.0). See the build log
above for details.

To Reproduce
Steps to reproduce the behavior:

Run cabal v2-build with above configuration.

Expected behavior

cabal build should succeed or print a message which can clearly pinpoint what is wrong. I tried going through the docs as well, but could not figure out what is wrong here. Did many trial and errors as well, but no luck with compiling this example.

System information

• Operating system: Mac OSX
• cabal version: 3.2
• ghc version: 8.8.3

[phadej]

There is #6005 which mentions problems with unpacked-containers, but it's marked as fixed. cc @ezyang

[harendra-kumar]

This has stalled my work, I would like to know if there is a workaround, how can I use unpacked-containers in my project?

[ezyang]

I agree this looks pretty similar to #6317 but given the cabal and ghc versions quoted in this ticket you should have this fix already.

[ezyang]

@harendra-kumar do you think you would have time to do a bisect for this problem?

[harendra-kumar]

@ezyang I can try, what do you suggest? Maybe, I can try running cabal at the commit right after that fix.

[harendra-kumar]

I built cabal at the commit in that PR and it still fails with the same error. I tried with ghc-8.6.5 as well, but no luck.

The example I am building is exactly copied from the unpacked-containers package. Surprisingly, when I build unpacked-containers package itself which contains this example, it builds perfectly well. But this example which uses unpacked-containers as a dependency does not build.

Based on the above, I guessed that if I add unpacked-containers to my cabal.project it might work and it worked! The problem seems to occur only when it is used as an external dependency.

$ cat cabal.project 
packages:., ../unpacked-containers/unpacked-containers

[ezyang]

Yeah, sounds like I didn't manage to completely nail it in that PR then. Please use this workaround for now, I'll take a look this weekend.

[Mikolaj]

@harendra-kumar: thank you again for reporting this; just in case: does the bug persist? [edit: and with which cabal version?]

[danidiaz]

I've encountered the same problem with my moo-nad package. To reproduce:

git clone --branch cabal_bug_6835 https://github.com/danidiaz/moo-nad.git moo-nad-x
cd moo-nad-x/
cabal test

The code that emits the error is in Cabal/Distribution/Backpack/Configure.hs, the function toComponentLocalBuildInfos. Which seems to create a Graph (Either InstalledPackageInfo ReadyComponent) which according to the comments is:

-- The database of transitively reachable installed packages that the
-- external components the package (as a whole) depends on.

So it seems that the graph is made of already compiled and installed external packages (from the store?) or ReadyComponents that can be compiled (should the products of compilation go in the store as well?)

In my case, the "keys" in the graph, along with their dependencies, are:

For node UnitId "base-4.15.0.0" the deps are [UnitId "ghc-bignum-1.0",UnitId "ghc-prim-0.7.0",UnitId "rts"]
For node UnitId "dep-t-0.4.4.0-91bfa4959ff60e0ce747ecfe782ff084cc3879da25a778ac219d4d5ff0aa0684" the deps are [UnitId "base-4.15.0.0",UnitId "mtl-2.2.2",UnitId "transformers-0.5.6.2",UnitId "unliftio-core-0.2.0.1-eddc7cf14ccd70f3513c556621fa675e3ed0579117262a3d9447d71c1c729978"]
For node UnitId "ghc-bignum-1.0" the deps are [UnitId "ghc-prim-0.7.0"]
For node UnitId "ghc-prim-0.7.0" the deps are [UnitId "rts"]
For node UnitId "moo-nad-0.1.0.2-136596e5d30785f22eda9db329422a145f8a7b99b9390088a3356b95680ad70e" the deps are [UnitId "base-4.15.0.0",UnitId "dep-t-0.4.4.0-91bfa4959ff60e0ce747ecfe782ff084cc3879da25a778ac219d4d5ff0aa0684",UnitId "mtl-2.2.2"]
For node (this is the only ReadyComponent, the other are InstalledPackageInfo) UnitId "moo-nad-0.1.0.2-136596e5d30785f22eda9db329422a145f8a7b99b9390088a3356b95680ad70e+L8yXyvV74rTHJagYNPv9JN" the deps are [UnitId "moo-nad-0.1.0.2-136596e5d30785f22eda9db329422a145f8a7b99b9390088a3356b95680ad70e",UnitId "base-4.15.0.0",UnitId "dep-t-0.4.4.0-91bfa4959ff60e0ce747ecfe782ff084cc3879da25a778ac219d4d5ff0aa0684",UnitId "mtl-2.2.2",UnitId "moo-oops-0.1.0.2-inplace-example-impl-x"]
For node UnitId "mtl-2.2.2" the deps are [UnitId "base-4.15.0.0",UnitId "transformers-0.5.6.2"]
For node UnitId "rts" the deps are []
For node UnitId "transformers-0.5.6.2" the deps are [UnitId "base-4.15.0.0"]
For node UnitId "unliftio-core-0.2.0.1-eddc7cf14ccd70f3513c556621fa675e3ed0579117262a3d9447d71c1c729978" the deps are [UnitId "base-4.15.0.0",UnitId "transformers-0.5.6.2"]

In particular, the "fully instantiated" ReadyComponent moo-nad-0.1.0.2-136596e5d30785f22eda9db329422a145f8a7b99b9390088a3356b95680ad70e+L8yXyvV74rTHJagYNPv9JN depends on the local compilation unit moo-oops-0.1.0.2-inplace-example-impl-x, but that unit doesn't have its own node in the graph, and that triggers the error.

Also, in the full entry for moo-nad-0.1.0.2-136596e5d30785f22eda9db329422a145f8a7b99b9390088a3356b95680ad70e+L8yXyvV74rTHJagYNPv9JN, if we look at the field instc_insts_deps that contains detailed descriptions about the dependencies, we find

instc_insts_deps = [(UnitId "moo-oops-0.1.0.2-inplace-example-impl-x",MungedPackageId {mungedName = MungedPackageName (PackageName "nonexistent-package-this-is-a-cabal-bug")

So it seems the bug happens earlier. Perhaps the graph should have an entry for moo-oops-0.1.0.2-inplace-example-impl-x, and it should be "munged" correctly. However, it's an inplace unit. Does it really belong to this graph, which seems to be for "external" dependencies?

[Mikolaj]

@danidiaz: thank you for the report. I see GHC is 9.0. Is cabal 3.4? Do you get the same error with current master branch of cabal?

Could anybody reproduce, please?

[danidiaz]

I'm using GHC 9.0.1 and cabal 3.4. I'm having trouble compiling the current master branch, so I didn't try with it.

[Mikolaj]

Thank you for the details. Master branch should compile fine, so feel free to open an issue if it doesn't. Or if I can help otherwise, please elaborate here.

[harendra-kumar]

@Mikolaj the original example I gave above is pretty simple and quick to reproduce. I tried it again with cabal built from the master branch but it failed with the same message.

$ ~/.cabal/bin/cabal --version
cabal-install version 3.5
compiled using version 3.5.0.0 of the Cabal library

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.8.3

$ ~/.cabal/bin/cabal build
...
Configuring library instantiated with
  Key = unpacked-example-0.1.0.0-inplace-example:Int
for unpacked-containers-0..
Warning: The package has an extraneous version range for a dependency on an
internal library: unpacked-containers:utils >=0 && ==0,
unpacked-containers:{unpacked-containers, example} (>=0 && >=0) && ==0. This
version range includes the current package but isn't needed as the current
package's library will always be used.
Error:
    The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
planned package unpacked-containers-0 is broken due to missing package unpacked-example-0.1.0.0-inplace-example

cabal: Failed to build unpacked-containers-0 (which is required by
exe:unpacked-set-example from unpacked-example-0.1.0.0). The failure occurred
during the configure step.