ghc-pkg: cannot create: ../dist-newstyle/packagedb/ghc-7.10.3 already exists

[edsko]

I have a cabal project with one main project and two sub-cabal packages in separate directories; i.e., my cabal.project looks like

packages: ./main-package.cabal
        , ./deps/sub-package-A
        , ./deps/sub-package-B

package main-package
  tests: True

When I try to build this after wiping the dist-newstyle directory completely, I get:

[7.10.3 ~/path/to/main-package]
# cabal new-build -j
Resolving dependencies...
In order, the following will be built (use -v for more details):
sub-package-A-0.1.1.0
sub-package-B-0.1.0.0
main-package-0.1.0.0
ghc-pkg: cannot create: /home/edsko/path/to/main-package/dist-newstyle/packagedb/ghc-7.10.3 already exists

[7.10.3 ~/path/to/main-package]
# Configuring sub-package-A-0.1.1.0...

I can work around the issue by manually installing one package at a time. I suspect it's something to do with concurrency (notice the interleaving of the final message from cabal and the bash prompt above).

@ezyang note: need to pass -j to trigger parallelism.

[23Skidoo]

IIRC @ezyang fixed a couple of similar issues for 1.24.

[dcoutts]

@edsko master, 1.24? Is this reliably reproducible?

[edsko]

1.24; I get it every time I call new-build after wiping the dist-newstyle directory in this project.

[ezyang]

I can repro on HEAD if I pass -j. Test-case in https://github.com/ezyang/cabal/tree/T3460

[dcoutts]

So I cannot actually reproduce the packagedb already exists behaviour with the T3460 test case (though I have seen this in the wild before too) but T3460 reveals a different issue for me in 1.24, which I actually fixed already in 1acc00f, so that's nice.

Nevertheless, it's pretty clear that it's a concurrency problem and I've made PR #3509 to avoid doing that action concurrently.