Unexpected reinstall with sandboxes #1274

UnkindPartition · 2013-04-19T06:32:58Z

I'm hitting a strange issue with sandboxes.

Here's what I did:

Created a sandbox for one of my projects
add-source'd a bunch of other local repositories
Ran cabal install -j --enable-tests. Tests were run and all the deps installed.

Ran cabal test. Suddenly, cabal wants to rebuild some dependencies.

Using a sandbox located at /home/feuerbach/prog/haskell-names/.cabal-sandbox
Reading available packages...
Reading installed packages...
("/opt/ghc762rc/bin/ghc-pkg",["dump","--global","-v0"])
("/opt/ghc762rc/bin/ghc-pkg",["dump","--package-db=/home/feuerbach/prog/haskell-names/.cabal-sandbox/ghc-7.6.1.20121207-packages.conf.d","-v0"])
("/opt/ghc762rc/bin/ghc",["--print-libdir"])
Reading available packages...
Choosing modular solver.
Resolving dependencies...
[__0] trying: haskell-names-0.1 (user goal)
[__1] next goal: base (dependency of haskell-names-0.1)
[__1] rejecting: base-3.0.3.2, 3.0.3.1 (global constraint requires installed instance)
[__1] trying: base-4.6.0.1/installed-495...
[__2] trying: rts-1.0/installedbuil... (dependency of base-4.6.0.1/installed-495...)
[__3] trying: integer-gmp-0.5.0.0/installed-b90... (dependency of base-4.6.0.1/installed-495...)
[__4] trying: ghc-prim-0.3.0.0/installed-00f... (dependency of base-4.6.0.1/installed-495...)
[__5] trying: haskell-names-0.1:!test
[__6] trying: type-eq-0.2/installed-668... (dependency of haskell-names-0.1)
[__7] trying: semigroupoids-3.0.2/installed-da5... (dependency of type-eq-0.2/installed-668...)
[__8] trying: transformers-0.3.0.0/installed-788... (dependency of semigroupoids-3.0.2/installed-da5...)
[__9] trying: semigroups-0.9/installed-9a5... (dependency of semigroupoids-3.0.2/installed-da5...)
[_10] trying: nats-0.1/installed-25f... (dependency of semigroups-0.9/installed-9a5...)
[_11] trying: contravariant-0.4/installed-28d... (dependency of semigroupoids-3.0.2/installed-da5...)
[_12] trying: transformers-compat-0.1.1.1/installed-19c... (dependency of contravariant-0.4/installed-28d...)
[_13] trying: comonad-3.0.2/installed-c5d... (dependency of semigroupoids-3.0.2/installed-da5...)
[_14] trying: groupoids-3.0.1.1/installed-e38... (dependency of type-eq-0.2/installed-668...)
[_15] trying: traverse-with-class-0.1/installed-217... (dependency of haskell-names-0.1)
[_16] trying: template-haskell-2.8.0.0/installed-b77... (dependency of traverse-with-class-0.1/installed-217...)
[_17] trying: pretty-1.1.1.0/installed-80a... (dependency of template-haskell-2.8.0.0/installed-b77...)
[_18] trying: tagged-0.6/installed-aab... (dependency of haskell-names-0.1)
[_19] trying: data-lens-template-2.1.7/installed-327... (dependency of haskell-names-0.1)
[_20] trying: data-lens-2.10.2/installed-cb0... (dependency of haskell-names-0.1)
[_21] trying: comonad-transformers-3.0.1/installed-7c4... (dependency of data-lens-2.10.2/installed-cb0...)
[_22] trying: distributive-0.3.1/installed-b86... (dependency of comonad-transformers-3.0.1/installed-7c4...)
[_23] trying: bytestring-0.10.0.2/installed-d44... (dependency of haskell-names-0.1)
[_24] trying: deepseq-1.3.0.1/installed-6ad... (dependency of bytestring-0.10.0.2/installed-d44...)
[_25] trying: array-0.4.0.1/installed-7b0... (dependency of deepseq-1.3.0.1/installed-6ad...)
[_26] trying: aeson-0.6.1.0/installed-8c8... (dependency of haskell-names-0.1)
[_27] trying: vector-0.10.0.1/installed-4be... (dependency of aeson-0.6.1.0/installed-8c8...)
[_28] trying: primitive-0.5.0.1/installed-4fa... (dependency of vector-0.10.0.1/installed-4be...)
[_29] trying: unordered-containers-0.2.3.0/installed-024... (dependency of aeson-0.6.1.0/installed-8c8...)
[_30] trying: time-1.4.0.1/installed-3d3... (dependency of aeson-0.6.1.0/installed-8c8...)
[_31] trying: text-0.11.2.3/installed-bd8... (dependency of aeson-0.6.1.0/installed-8c8...)
[_32] trying: syb-0.4.0/installed-ea3... (dependency of aeson-0.6.1.0/installed-8c8...)
[_33] trying: old-locale-1.0.0.5/installed-616... (dependency of aeson-0.6.1.0/installed-8c8...)
[_34] trying: hashable-1.2.0.5/installed-92f... (dependency of aeson-0.6.1.0/installed-8c8...)
[_35] trying: dlist-0.5/installed-ad6... (dependency of aeson-0.6.1.0/installed-8c8...)
[_36] trying: blaze-builder-0.3.1.1/installed-f05... (dependency of aeson-0.6.1.0/installed-8c8...)
[_37] trying: attoparsec-0.10.4.0/installed-518... (dependency of aeson-0.6.1.0/installed-8c8...)
[_38] trying: uniplate-1.6.10/installed-bb1... (dependency of haskell-names-0.1)
[_39] trying: containers-0.5.0.0/installed-565... (dependency of haskell-names-0.1)
[_40] trying: filepath-1.3.0.1/installed-dd9... (dependency of haskell-names-0.1)
[_41] trying: mtl-2.1.2/installed-e4d... (dependency of haskell-names-0.1)
[_42] trying: haskell-src-exts-1.13.5/installed-a08... (dependency of haskell-names-0.1)
[_43] trying: cpphs-1.16/installed-9a9... (dependency of haskell-src-exts-1.13.5/installed-a08...)
[_44] trying: old-time-1.1.0.1/installed-110... (dependency of cpphs-1.16/installed-9a9...)
[_45] trying: directory-1.2.0.1/installed-660... (dependency of cpphs-1.16/installed-9a9...)
[_46] trying: unix-2.6.0.1/installed-8f7... (dependency of directory-1.2.0.1/installed-660...)
[_47] trying: Cabal-1.17.1 (user goal)
[_48] trying: Cabal-1.17.1:!test
[_49] trying: process-1.1.0.2/installed-bee... (dependency of Cabal-1.17.1)
[_50] trying: haskell-packages-0.1 (user goal)
[_51] trying: EitherT-0.2.0/installed-c6a... (dependency of haskell-packages-0.1)
[_52] trying: transformers-base-0.4.1/installed-500... (dependency of EitherT-0.2.0/installed-c6a...)
[_53] trying: monad-control-0.3.2/installed-341... (dependency of EitherT-0.2.0/installed-c6a...)
[_54] trying: base-unicode-symbols-0.2.2.4/installed-f67... (dependency of monad-control-0.3.2/installed-341...)
[_55] trying: optparse-applicative-0.4.3/installed-d5a... (dependency of haskell-packages-0.1)
[_56] trying: hse-cpp-0.1 (user goal)
[_57] trying: test-framework-0.8.1 (user goal)
[_58] trying: test-framework-0.8.1:-tests
[_59] trying: test-framework-0.8.1:+splitbase
[_60] trying: random-1.0.1.1/installed-970... (dependency of test-framework-0.8.1:+splitbase)
[_61] trying: hostname-1.0/installed-d99... (dependency of test-framework-0.8.1)
[_62] trying: xml-1.3.13/installed-cec... (dependency of test-framework-0.8.1)
[_63] trying: extensible-exceptions-0.1.1.4/installed-cdb... (dependency of test-framework-0.8.1)
[_64] trying: regex-posix-0.95.2/installed-bff... (dependency of test-framework-0.8.1)
[_65] trying: regex-base-0.93.2/installed-53d... (dependency of regex-posix-0.95.2/installed-bff...)
[_66] trying: ansi-wl-pprint-0.6.6/installed-6dc... (dependency of test-framework-0.8.1)
[_67] trying: ansi-terminal-0.6/installed-a84... (dependency of test-framework-0.8.1)
[_68] next goal: test-framework-golden (user goal)
[_68] trying: test-framework-golden-1.1.2
[_69] done
In order, the following would be installed:
Cabal-1.17.1 (reinstall)
hse-cpp-0.1 (reinstall)
haskell-packages-0.1 (reinstall)
test-framework-0.8.1 (reinstall)
test-framework-golden-1.1.2 (reinstall)
cabal: The following packages are likely to be broken by the reinstalls:
haskell-names-0.1
Use --force-reinstalls if you want to install anyway.

It's totally unclear why reinstalls are happening. I don't see any "rejecting ..." messages from the solver. Besides, nothing has changed between cabal install --enable-tests and cabal test, so the installed package set should be sufficient.

The text was updated successfully, but these errors were encountered:

UnkindPartition · 2013-04-19T06:35:38Z

Here is the set of the packages in the sandbox:

/home/feuerbach/prog/haskell-names/.cabal-sandbox/ghc-7.6.1.20121207-packages.conf.d
   Cabal-1.17.1
   EitherT-0.2.0
   aeson-0.6.1.0
   ansi-terminal-0.6
   ansi-wl-pprint-0.6.6
   attoparsec-0.10.4.0
   base-unicode-symbols-0.2.2.4
   blaze-builder-0.3.1.1
   comonad-3.0.2
   comonad-transformers-3.0.1
   contravariant-0.4
   cpphs-1.16
   data-lens-2.10.2
   data-lens-template-2.1.7
   distributive-0.3.1
   dlist-0.5
   extensible-exceptions-0.1.1.4
   filemanip-0.3.6.2
   groupoids-3.0.1.1
   hashable-1.2.0.5
   haskell-lexer-1.0
   haskell-names-0.1
   haskell-packages-0.1
   haskell-src-exts-1.13.5
   hostname-1.0
   hse-cpp-0.1
   monad-control-0.3.2
   mtl-2.1.2
   nats-0.1
   optparse-applicative-0.4.3
   pretty-show-1.5
   primitive-0.5.0.1
   random-1.0.1.1
   regex-base-0.93.2
   regex-posix-0.95.2
   semigroupoids-3.0.2
   semigroups-0.9
   syb-0.4.0
   tagged-0.6
   test-framework-0.8.1
   test-framework-golden-1.1.2
   text-0.11.2.3
   transformers-0.3.0.0
   transformers-base-0.4.1
   transformers-compat-0.1.1.1
   traverse-with-class-0.1
   type-eq-0.2
   uniplate-1.6.10
   unix-compat-0.4.1.1
   unordered-containers-0.2.3.0
   utf8-string-0.3.7
   vector-0.10.0.1
   xml-1.3.13

tibbe · 2013-04-19T14:09:32Z

These are exactly the kind of issues we'd like to find before the release. Thanks for the report. Any way you could publish the packages needed to repro this somewhere online (doesn't have to be Hackage).

23Skidoo · 2013-04-19T15:09:24Z

This is caused by this fragment in testAction:

  -- If we're in a sandbox, (re)install all add-source dependencies.
  useSandbox <- maybeInstallAddSourceDeps verbosity globalFlags

  reconfigure verbosity distPref addConfigFlags [] globalFlags checkFlags
  maybeWithSandboxDirOnSearchPath useSandbox $
    build verbosity distPref mempty []

The problem is that we're running build unconditionally each time, so we have to reinstall add-source deps (since we can't tell whether they've been changed). We could run build only if reconfigure returned True, but that would miss changes done to the source files.

This is a kind of problem add-source --snapshot would alleviate (by providing cabal-dev add-source behaviour).

UnkindPartition · 2013-04-19T15:32:07Z

Ah, that makes sense. But perhaps cabal should unregister these packages (or something along those lines) before re-installing them to avoid this effect?

tibbe · 2013-04-19T15:35:29Z

Is there a cheap check we could make to see if rebuild is not required? (I guess not.)

23Skidoo · 2013-04-19T15:46:29Z

@tibbe The only thing I can think of is to check the modification date of all modules and compare that with the modification date of exe/library.

@feuerbach Yes, unregistering add-source packages before reinstalling makes sense.

23Skidoo · 2013-04-19T15:57:44Z

Opened issue #1275 about unregistering add-source deps.

For minimal rebuilds of add-source deps there is already #1121.

23Skidoo · 2013-04-20T02:36:43Z

@tibbe I'm considering closing this issue in favor of #1275 and #1121. Is there something else we want to do here?

23Skidoo · 2013-04-22T05:04:11Z

@tibbe

On the topic of a smarter rebuild check, do you think that we could get away with just comparing the modification dates of the add-source deps' sources with the modification date of dist/setup-config (since reinstalling a modified library requires reconfigure)? Or, even better, we could add a special file .cabal-sandbox/sandbox-timestamp (since dist/setup-config can be modified by running runhaskell Setup.hs configure).

That'd also allow us to implement add-source by copying as @feuerbach suggested in #1281.

One thing I'll have to investigate is whether we can get a list of all sources just by parsing a .cabal file, without running sdist.

23Skidoo · 2013-04-22T06:15:17Z

One thing I'll have to investigate is whether we can get a list of all sources just by parsing a .cabal file, without running sdist.

Looking at D.S.SrcDist, it seems to be possible.

tibbe · 2013-04-22T23:46:04Z

Closing in favor of the other tickets seems sensible.

23Skidoo · 2013-04-23T04:33:16Z

Closing in favour of #1121 and #1275.

23Skidoo mentioned this issue Apr 19, 2013

add-source dependencies should be unregistered before reinstalling #1275

Closed

23Skidoo mentioned this issue Apr 22, 2013

haddock: can't find a package database at <sandbox> #1281

Closed

23Skidoo closed this as completed Apr 23, 2013

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Unexpected reinstall with sandboxes #1274

Unexpected reinstall with sandboxes #1274

UnkindPartition commented Apr 19, 2013

UnkindPartition commented Apr 19, 2013

Uh oh!

tibbe commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 19, 2013

Uh oh!

UnkindPartition commented Apr 19, 2013

Uh oh!

tibbe commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 20, 2013

Uh oh!

23Skidoo commented Apr 22, 2013

Uh oh!

23Skidoo commented Apr 22, 2013

Uh oh!

tibbe commented Apr 22, 2013

Uh oh!

23Skidoo commented Apr 23, 2013

Uh oh!

Unexpected reinstall with sandboxes #1274

Unexpected reinstall with sandboxes #1274

Comments

UnkindPartition commented Apr 19, 2013

UnkindPartition commented Apr 19, 2013

Uh oh!

tibbe commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 19, 2013

Uh oh!

UnkindPartition commented Apr 19, 2013

Uh oh!

tibbe commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 19, 2013

Uh oh!

23Skidoo commented Apr 20, 2013

Uh oh!

23Skidoo commented Apr 22, 2013

Uh oh!

23Skidoo commented Apr 22, 2013

Uh oh!

tibbe commented Apr 22, 2013

Uh oh!

23Skidoo commented Apr 23, 2013

Uh oh!