Skip to content

new-build failure with custom-setup when setup-depends don't contain Cabal #4288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
phadej opened this issue Feb 1, 2017 · 11 comments
Closed

Comments

@phadej
Copy link
Collaborator

phadej commented Feb 1, 2017

custom-issue.cabal:

name:                custom-issue
version:             0.1.0.0
license:             BSD3
license-file:        LICENSE
author:              Oleg Grenrus
maintainer:          [email protected]
build-type:          Custom
extra-source-files:  ChangeLog.md
cabal-version:       >=1.10

custom-setup
  setup-depends:
    base, cabal-doctest

library
  exposed-modules:     CustomIssue
  build-depends:       base >=4.9 && <4.10
  default-language:    Haskell2010

Setup.hs:

import Distribution.Extra.Doctest
main = defaultMainWithDoctests "doctests"

Steps to reproduce

cabal new-build

Output

In order, the following will be built (use -v for more details):
 - custom-issue-0.1.0.0 (lib:custom-issue) (first run)
Building custom-issue-0.1.0.0...
Preprocessing library custom-issue-0.1.0.0...
[1 of 1] Compiling CustomIssue      ( CustomIssue.hs, /home/ogre/mess/custom-issue/dist-newstyle/build/x86_64-linux/ghc-8.0.1/custom-issue-0.1.0.0/build/CustomIssue.o )
Creating package registration file:
/home/ogre/mess/custom-issue/dist-newstyle/tmp/package-registration--16283/pkgConf
cabal: Failed to build custom-issue-0.1.0.0. The failure occurred during the
final install step. The exception was:
user error ('/opt/ghc/8.0.1/bin/ghc-pkg' exited with an error:
custom-issue-0.1.0.0: Warning: Unrecognized field indefinite on line 9
custom-issue-0.1.0.0: Warning: haddock-interfaces:
/home/ogre/mess/custom-issue/dist-newstyle/build/x86_64-linux/ghc-8.0.1/custom-issue-0.1.0.0/doc/html/custom-issue/custom-issue.haddock
doesn't exist or isn't a file
custom-issue-0.1.0.0: Warning: haddock-html:
/home/ogre/mess/custom-issue/dist-newstyle/build/x86_64-linux/ghc-8.0.1/custom-issue-0.1.0.0/doc/html/custom-issue
doesn't exist or isn't a directory
custom-issue-0.1.0.0: installed package info from too old version of Cabal
(key field does not match id field)
)

Workaround

  • Add Cabal to setup-depends

Original issue

Reproducible with ghc-8.0.1 and ghc-8.0.2

distributive-0.5.2-e10f1d2339ae8fc2ff95360c5e91f0ace4f1e27add106f7939389ad5ec9ca0aa.
The failure occurred during the final install step. The exception was:
user error ('/opt/ghc/8.0.1/bin/ghc-pkg' exited with an error:
distributive-0.5.2: Warning: Unrecognized field indefinite on line 16
distributive-0.5.2: Warning: haddock-interfaces:
/home/ogre/.cabal/store/ghc-8.0.1/distributive-0.5.2-e10f1d2339ae8fc2ff95360c5e91f0ace4f1e27add106f7939389ad5ec9ca0aa/share/doc/html/distributive.haddock
doesn't exist or isn't a file
distributive-0.5.2: Warning: haddock-html:
/home/ogre/.cabal/store/ghc-8.0.1/distributive-0.5.2-e10f1d2339ae8fc2ff95360c5e91f0ace4f1e27add106f7939389ad5ec9ca0aa/share/doc/html
doesn't exist or isn't a directory
distributive-0.5.2: installed package info from too old version of Cabal (key
field does not match id field)

Steps to reproduce

cabal get comonod-5
cd comonad-5
cabal new-build --disable-tests -j1

/note/ -j1 isn't necessary

Additional info

part of new-build log building the distributive

% /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup register --verbose=2 --builddir=/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2 --inplace --gen-pkg-config=/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--12792/pkgConf
/opt/ghc/8.0.1/bin/ghc --abi-hash -fbuilding-cabal-package -O -outputdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -odir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -hidir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -stubdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -i -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -isrc -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -optP-include -optP/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen/cabal_macros.h -this-unit-id distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw -hide-all-packages -no-user-package-db -package-db /home/ogre/.cabal/store/ghc-8.0.1/package.db -package-db /home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1 -package-id base-4.9.0.0 -package-id base-orphans-0.5.4-221596738ac95aba5b36b0b0362492907162ea1af94cf8f335da4c27a8bef6e9 -package-id tagged-0.8.5-6a76d83406e0466a505c9569c11ade18b2bb910a6a09df4ba0d355bf7922badf -package-id transformers-0.5.2.0 -package-id transformers-compat-0.5.1.4-a6f22a40e821e7b9a582fac258c4122bb958675f217a786f78176ce395f17184 -XHaskell98 Data.Distributive Data.Distributive.Generic -Wall
Creating package registration file:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--12792/pkgConf
/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--12792/:
openBinaryTempFileWithDefaultPermissions: does not exist (No such file or
directory)

That directory doesn't seem to exist.

@phadej
Copy link
Collaborator Author

phadej commented Feb 1, 2017

does any one remember was there a bug in 1.24?

The GHC-7.10.3 seems successfully register stuff:

/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-7.10.3/distributive-0.5.2/setup/setup
register --verbose=2
--builddir=/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-7.10.3/distributive-0.5.2
--inplace

But otoh, distributive travis did pass yesterday fine, EDIT: because we didn't use new-build there!

@phadej
Copy link
Collaborator Author

phadej commented Feb 1, 2017

/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup
/opt/ghc/8.0.1/bin/ghc --make -fbuilding-cabal-package -odir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup -hidir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup -i -i/home/ogre/mess/distributive-0.5.2/. -optP-include -optP/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup_macros.h -hide-all-packages -no-user-package-db -package-db /home/ogre/.cabal/store/ghc-8.0.1/package.db -package-db /home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1 -package-id base-4.9.0.0 -package-id cabal-doctest-1-6a5408abc48f41d4246a205b4e0e096398ba6620b428170fc35098b9b2458faa /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup.hs -o /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup -threaded
/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/setup/setup
register --verbose=2
--builddir=/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2
--inplace
--gen-pkg-config=/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--28640/pkgConf
/opt/ghc/8.0.1/bin/ghc --abi-hash -fbuilding-cabal-package -O -outputdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -odir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -hidir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -stubdir /home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -i -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -isrc -i/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen -I/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build -optP-include -optP/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build/autogen/cabal_macros.h -this-unit-id distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw -hide-all-packages -no-user-package-db -package-db /home/ogre/.cabal/store/ghc-8.0.1/package.db -package-db /home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1 -package-id base-4.9.0.0 -package-id base-orphans-0.5.4-221596738ac95aba5b36b0b0362492907162ea1af94cf8f335da4c27a8bef6e9 -package-id tagged-0.8.5-6a76d83406e0466a505c9569c11ade18b2bb910a6a09df4ba0d355bf7922badf -package-id transformers-0.5.2.0 -package-id transformers-compat-0.5.1.4-a6f22a40e821e7b9a582fac258c4122bb958675f217a786f78176ce395f17184 -XHaskell98 Data.Distributive Data.Distributive.Generic -Wall
Creating package registration file:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/tmp/package-registration--28640/pkgConf
"register"
InstalledPackageInfo {sourcePackageId = PackageIdentifier {pkgName = PackageName "distributive", pkgVersion = mkVersion [0,5,2]}, installedUnitId = UnitId "distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw", installedComponentId_ = ComponentId "", instantiatedWith = [], compatPackageKey = "distributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw", license = BSD3, copyright = "Copyright (C) 2011-2016 Edward A. Kmett", maintainer = "Edward A. Kmett <[email protected]>", author = "Edward A. Kmett", stability = "provisional", homepage = "http://github.com/ekmett/distributive/", pkgUrl = "", synopsis = "Distributive functors -- Dual to Traversable", description = "Distributive functors -- Dual to Traversable", category = "Data Structures", abiHash = AbiHash "103f62b0c9b3d483d1d59f3559a62b22", indefinite = False, exposed = True, exposedModules = [ExposedModule {exposedName = ModuleName ["Data","Distributive"], exposedReexport = Nothing},ExposedModule {exposedName = ModuleName ["Data","Distributive","Generic"], exposedReexport = Nothing}], hiddenModules = [], trusted = False, importDirs = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build"], libraryDirs = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/build"], libraryDynDirs = [], dataDir = "/home/ogre/mess/distributive-0.5.2", hsLibraries = ["HSdistributive-0.5.2-3Qb1gotTB50JIJFSZJyPVw"], extraLibraries = [], extraGHCiLibraries = [], includeDirs = [], includes = [], depends = [UnitId "base-4.9.0.0",UnitId "base-orphans-0.5.4-221596738ac95aba5b36b0b0362492907162ea1af94cf8f335da4c27a8bef6e9",UnitId "tagged-0.8.5-6a76d83406e0466a505c9569c11ade18b2bb910a6a09df4ba0d355bf7922badf",UnitId "transformers-0.5.2.0",UnitId "transformers-compat-0.5.1.4-a6f22a40e821e7b9a582fac258c4122bb958675f217a786f78176ce395f17184"], abiDepends = [], ccOptions = [], ldOptions = [], frameworkDirs = [], frameworks = [], haddockInterfaces = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive/distributive.haddock"], haddockHTMLs = ["/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive"], pkgRoot = Nothing}
/opt/ghc/8.0.1/bin/ghc-pkg update - --global --no-user-package-db '--package-db=/home/ogre/.cabal/store/ghc-8.0.1/package.db' '--package-db=/home/ogre/mess/distributive-0.5.2/dist-newstyle/packagedb/ghc-8.0.1'
cabal: Failed to build distributive-0.5.2-inplace. The failure occurred during
the final install step. The exception was:
user error ('/opt/ghc/8.0.1/bin/ghc-pkg' exited with an error:
distributive-0.5.2: Warning: Unrecognized field indefinite on line 16
distributive-0.5.2: Warning: haddock-interfaces:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive/distributive.haddock
doesn't exist or isn't a file
distributive-0.5.2: Warning: haddock-html:
/home/ogre/mess/distributive-0.5.2/dist-newstyle/build/x86_64-linux/ghc-8.0.1/distributive-0.5.2/doc/html/distributive
doesn't exist or isn't a directory
distributive-0.5.2: installed package info from too old version of Cabal (key
field does not match id field)

I see there is comment in ProjectBuilding:

                -- We register ourselves rather than via Setup.hs. We need to
                -- grab and modify the InstalledPackageInfo. We decide what
                -- the installed package id is, not the build system.

but why there is both .../setup register and ghc-pkg update ?

ping @dcoutts

EDIT this is Cabal with

diff --git a/cabal-install/Distribution/Client/ProjectBuilding.hs b/cabal-install/Distribution/Client/ProjectBuilding.hs
index 8733dbe..2aa1f69 100644
--- a/cabal-install/Distribution/Client/ProjectBuilding.hs
+++ b/cabal-install/Distribution/Client/ProjectBuilding.hs
@@ -1086,6 +1086,8 @@ buildInplaceUnpackedPackage verbosity
           mipkg <- if elabRequiresRegistration pkg
             then do
                 ipkg0 <- generateInstalledPackageInfo
+                print "register"
+                print ipkg0

not sure where from the first register is even called.

@phadej
Copy link
Collaborator Author

phadej commented Feb 1, 2017

hmm..

Interestingly, if i cabal get distributive-0.5.2 and add Cabal, filepath, directory to setup-depends, problem seems go away. But those packages aren't directly used in Setup.lhs

EDIT verifying currently with travis build.

@23Skidoo
Copy link
Member

23Skidoo commented Feb 1, 2017

Just bumped into this as well.

@23Skidoo
Copy link
Member

23Skidoo commented Feb 1, 2017

I wonder why the setup script compiled without Cabal in build-depends of the custom-setup section, since it seems to be importing stuff from Distribution.*.

@23Skidoo 23Skidoo added this to the 2.2 milestone Feb 1, 2017
@phadej phadej changed the title new-build failure new-build failure with custom-setup when setup-depends don't contain Cabal Feb 1, 2017
@phadej
Copy link
Collaborator Author

phadej commented Feb 1, 2017

@23Skidoo edited the original issue with minimal example. if cabal-install knows how to handle setup-depends, Setup.hs doesn't need Cabal, if you check the CPP magic there properly.

@23Skidoo
Copy link
Member

23Skidoo commented Feb 1, 2017

@phadej Thanks, that clears it.

@ezyang
Copy link
Contributor

ezyang commented Feb 2, 2017

At a guess, this is probably because cabal-install is applying an upper bound to the choice of Cabal library, even when an older version of Cabal wouldn't work with a new GHC? Though I'm a bit confused because 1.24 should be a permitted version in any case.

@mgsloan
Copy link
Collaborator

mgsloan commented Nov 13, 2017

Recently encountered something similar in stack - commercialhaskell/stack#3560 (comment) . Should there always be an implicit dependency on Cabal for Setup.hs?

I don't think there should be. What if there is a modified version of Cabal that has a different name? Now the implicit dep on Cabal could cause module ambiguity.

@grayjay
Copy link
Collaborator

grayjay commented Nov 19, 2017

@mgsloan I wasn't able to reproduce this issue, but I don't think cabal-install should add an implicit dependency on Cabal in this case. I think there are only two places where it adds implicit dependencies:

  1. mkDefaultSetupDeps :: UnresolvedSourcePackage -> Maybe [Dependency]
    This is a workaround for Package with custom Setup.hs "Encountered missing dependencies" #3199. It only applies to old build in certain cases where the .cabal file contains buildable: False.
  2. new-build adds a set of default setup dependencies to packages with build-type: Custom but no custom-setup section. I don't think new-build is ever supposed to modify an existing custom-setup.

@grayjay
Copy link
Collaborator

grayjay commented Nov 19, 2017

I made a mistake before, and I can actually reproduce this bug with custom-issue.cabal. I used cabal from master and GHC 8.0.1, and I compared the result of new-build with and without the explicit dependency on Cabal. I think that the explicit Cabal dependency affected the choice of cabal spec version, not the version of the Cabal dependency. The log showed that cabal chose the same instance of Cabal in both cases, the installed Cabal-1.24.0.0. I also tried printing the spec version in D.C.ProjectPlanning.packageSetupScriptSpecVersion:

packageSetupScriptSpecVersion _ pkg deps =
case find ((cabalPkgname ==) . packageName) (CD.setupDeps deps) of
Just dep -> packageVersion dep
Nothing -> PD.specVersion pkg

With an explicit Cabal dependency, cabal chose that dependency's version (1.24.0.0) at line 2896. Without the direct dependency, cabal chose the package's cabal spec version at line 2897, which was 1.10. I think that packageSetupScriptSpecVersion needs to also check for indirect setup dependencies on Cabal.

grayjay added a commit to grayjay/cabal that referenced this issue Feb 26, 2018
…c version.

Fixes haskell#4288.

Previously, new-build only looked for a direct dependency on Cabal when choosing
the spec version to use for running the setup script.  When the setup script
only had a transitive dependency on Cabal, cabal used the package's
cabal-version field, which could easily differ from the actual Cabal
dependency's version.  Then cabal could pass flags to the setup script that
weren't recognized by the Cabal dependency.

This commit considers any setup dependency on Cabal when choosing the Cabal spec
version.
grayjay added a commit to grayjay/cabal that referenced this issue Feb 26, 2018
grayjay added a commit to grayjay/cabal that referenced this issue Mar 8, 2018
…c version.

Fixes haskell#4288.

Previously, new-build only looked for a direct dependency on Cabal when choosing
the spec version to use for running the setup script.  When the setup script
only had a transitive dependency on Cabal, cabal used the package's
cabal-version field, which could easily differ from the actual Cabal
dependency's version.  Then cabal could pass flags to the setup script that
weren't recognized by the Cabal dependency.

This commit considers any setup dependency on Cabal when choosing the Cabal spec
version.

(cherry picked from commit ea01b9d)
grayjay added a commit to grayjay/cabal that referenced this issue Mar 8, 2018
Sonna added a commit to Sonna/toy-robot-haskell that referenced this issue Mar 29, 2018
Fix the setup of this Haskell project within CircleCI and when setting
up on a new machine or within Docker. This required that `Haq` not be
considered a Library (without an executable), but instead an exposed
module that was built and included within both the base project, as well
as its tests.

This should resolve problems that where occurring when running
`cabal install`

== Notes:
- Change the order of `cabal update` with `cabal sandbox init`, since
  update will only run it there is a sandbox available, which is not
  present in the Haskell image container

- Add `--enable-tests` and `--enable-documentation` to cabal install to
  slience other warnings

== References:
- [How to write a Haskell program \- HaskellWiki]
  (https://wiki.haskell.org/How_to_write_a_Haskell_program#Add_some_tests)

- [cabal \- Set up Haskell Project and run tests \- Stack Overflow]
  (https://stackoverflow.com/questions/38928802/set-up-haskell-project-and-run-tests)

- [Cabal\-Install \- HaskellWiki]
  (https://wiki.haskell.org/Cabal-Install)

- [Invoking Haddock — Haddock 1\.0 documentation]
  (http://haskell-haddock.readthedocs.io/en/latest/invoking.html)

- [new\-build failure with custom\-setup when setup\-depends don't
  contain Cabal · Issue \#4288 · haskell/cabal]
  (haskell/cabal#4288)

- [2\.1\. Configuration — Cabal <release> User's Guide]
  (https://www.haskell.org/cabal/users-guide/installing-packages.html)
ysangkok referenced this issue in commercialhaskell/stackage Mar 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants