--extra-include-dirs and --extra-lib-dirs not respected when building Setup itself

[IreneKnapp]

As you may be aware, the first thing that Cabal does upon configure is, if there is a Setup.hs script, attempt to compile it. This compilation does not appear to respect the usual options that apply to every other compilation, including the two named above: --extra-include-dirs and --extra-lib-dirs.

This is a problem, because some systems do not have global /usr and /lib directories, but rather put these things elsewhere. In particular, it has always been the case on the Mac really that one is supposed to be using paths within the SDK directory rather than global paths, to allow for compilation to target other versions of OS X than the one currently running. Now in this year's 10.NDA release, the global paths are removed entirely.

This problem appears to apply regardless of whether the paths are specified in ~/.cabal/config, or on the command line.

Things have gotten to this point without totally breaking because not every package uses a custom Setup.hs, so this build step doesn't always need to be done.

Tested on:
cabal-install version 1.16.0.2
using version 1.16.0 of the Cabal library

I'll have a pull request shortly for everyone to ignore. You're welcome.

[IreneKnapp]

I see that the setup wrapper, as it's called, exists only under cabal-install and not as part of the Cabal library. That means some unfortunate abstraction violation to make this work, which is probably why it hasn't happened yet.

That's kinda too bad.

Since I have no expectation that this patch will be accepted, I'm just going to violate that abstraction. If you want code submissions to comply to your standards, you need to review them before they bitrot, not make people wait for literally years without even looking at it.

[IreneKnapp]

The errors look like this, for example:

irene@Estradiol:~/Projects/Adopted/cabal/Cabal$ ghc --make Setup
[ 1 of 68] Compiling Distribution.PackageDescription.Utils ( Distribution/PackageDescription/Utils.hs, Distribution/PackageDescription/Utils.o ) [flags changed]
[ 2 of 68] Compiling Distribution.Simple.CCompiler ( Distribution/Simple/CCompiler.hs, Distribution/Simple/CCompiler.o ) [flags changed]
[ 3 of 68] Compiling Distribution.Compat.Exception ( Distribution/Compat/Exception.hs, Distribution/Compat/Exception.o ) [flags changed]
[ 4 of 68] Compiling Distribution.Compat.TempFile ( Distribution/Compat/TempFile.hs, Distribution/Compat/TempFile.o ) [flags changed]
[ 5 of 68] Compiling Distribution.Compat.CopyFile ( Distribution/Compat/CopyFile.hs, Distribution/Compat/CopyFile.o ) [flags changed]
[ 6 of 68] Compiling Distribution.GetOpt ( Distribution/GetOpt.hs, Distribution/GetOpt.o ) [flags changed]
[ 7 of 68] Compiling Distribution.Compat.ReadP ( Distribution/Compat/ReadP.hs, Distribution/Compat/ReadP.o ) [flags changed]
[ 8 of 68] Compiling Distribution.Text ( Distribution/Text.hs, Distribution/Text.o ) [flags changed]
[ 9 of 68] Compiling Distribution.Version ( Distribution/Version.hs, Distribution/Version.o ) [flags changed]
[10 of 68] Compiling Language.Haskell.Extension ( Language/Haskell/Extension.hs, Language/Haskell/Extension.o ) [flags changed]
[11 of 68] Compiling Distribution.TestSuite ( Distribution/TestSuite.hs, Distribution/TestSuite.o ) [flags changed]
[12 of 68] Compiling Distribution.System ( Distribution/System.hs, Distribution/System.o ) [flags changed]
[13 of 68] Compiling Distribution.Simple.PreProcess.Unlit ( Distribution/Simple/PreProcess/Unlit.hs, Distribution/Simple/PreProcess/Unlit.o ) [flags changed]
[14 of 68] Compiling Distribution.ReadE ( Distribution/ReadE.hs, Distribution/ReadE.o ) [flags changed]
[15 of 68] Compiling Distribution.Verbosity ( Distribution/Verbosity.hs, Distribution/Verbosity.o ) [flags changed]
[16 of 68] Compiling Distribution.Package ( Distribution/Package.hs, Distribution/Package.o ) [flags changed]
[17 of 68] Compiling Distribution.ModuleName ( Distribution/ModuleName.hs, Distribution/ModuleName.o ) [flags changed]
[18 of 68] Compiling Distribution.Simple.Utils ( Distribution/Simple/Utils.hs, Distribution/Simple/Utils.o ) [flags changed]
[19 of 68] Compiling Distribution.Simple.Program.Types ( Distribution/Simple/Program/Types.hs, Distribution/Simple/Program/Types.o ) [flags changed]
[20 of 68] Compiling Distribution.Simple.Program.Builtin ( Distribution/Simple/Program/Builtin.hs, Distribution/Simple/Program/Builtin.o ) [flags changed]
[21 of 68] Compiling Distribution.Simple.Program.Db ( Distribution/Simple/Program/Db.hs, Distribution/Simple/Program/Db.o ) [flags changed]
[22 of 68] Compiling Distribution.Simple.Program.Run ( Distribution/Simple/Program/Run.hs, Distribution/Simple/Program/Run.o ) [flags changed]
[23 of 68] Compiling Distribution.Simple.Program ( Distribution/Simple/Program.hs, Distribution/Simple/Program.o ) [flags changed]
[24 of 68] Compiling Distribution.Simple.Program.Ar ( Distribution/Simple/Program/Ar.hs, Distribution/Simple/Program/Ar.o ) [flags changed]
[25 of 68] Compiling Distribution.Simple.Program.Hpc ( Distribution/Simple/Program/Hpc.hs, Distribution/Simple/Program/Hpc.o ) [flags changed]
[26 of 68] Compiling Distribution.Simple.Program.Ld ( Distribution/Simple/Program/Ld.hs, Distribution/Simple/Program/Ld.o ) [flags changed]
[27 of 68] Compiling Distribution.Simple.Program.Script ( Distribution/Simple/Program/Script.hs, Distribution/Simple/Program/Script.o ) [flags changed]
[28 of 68] Compiling Distribution.License ( Distribution/License.hs, Distribution/License.o ) [flags changed]
[29 of 68] Compiling Distribution.Compiler ( Distribution/Compiler.hs, Distribution/Compiler.o ) [flags changed]
[30 of 68] Compiling Distribution.PackageDescription ( Distribution/PackageDescription.hs, Distribution/PackageDescription.o ) [flags changed]
[31 of 68] Compiling Distribution.PackageDescription.Configuration ( Distribution/PackageDescription/Configuration.hs, Distribution/PackageDescription/Configuration.o ) [flags changed]
[32 of 68] Compiling Distribution.PackageDescription.Check ( Distribution/PackageDescription/Check.hs, Distribution/PackageDescription/Check.o ) [flags changed]
[33 of 68] Compiling Distribution.ParseUtils ( Distribution/ParseUtils.hs, Distribution/ParseUtils.o ) [flags changed]
[34 of 68] Compiling Distribution.InstalledPackageInfo ( Distribution/InstalledPackageInfo.hs, Distribution/InstalledPackageInfo.o ) [flags changed]
[35 of 68] Compiling Distribution.Simple.PackageIndex ( Distribution/Simple/PackageIndex.hs, Distribution/Simple/PackageIndex.o ) [flags changed]
[36 of 68] Compiling Distribution.Simple.GHC.IPI642 ( Distribution/Simple/GHC/IPI642.hs, Distribution/Simple/GHC/IPI642.o ) [flags changed]
[37 of 68] Compiling Distribution.Simple.GHC.IPI641 ( Distribution/Simple/GHC/IPI641.hs, Distribution/Simple/GHC/IPI641.o ) [flags changed]
[38 of 68] Compiling Distribution.PackageDescription.Parse ( Distribution/PackageDescription/Parse.hs, Distribution/PackageDescription/Parse.o ) [flags changed]
[39 of 68] Compiling Distribution.Simple.Command ( Distribution/Simple/Command.hs, Distribution/Simple/Command.o ) [flags changed]
[40 of 68] Compiling Distribution.Simple.Compiler ( Distribution/Simple/Compiler.hs, Distribution/Simple/Compiler.o ) [flags changed]
[41 of 68] Compiling Distribution.Simple.Program.HcPkg ( Distribution/Simple/Program/HcPkg.hs, Distribution/Simple/Program/HcPkg.o ) [flags changed]
[42 of 68] Compiling Distribution.Simple.InstallDirs ( Distribution/Simple/InstallDirs.hs, Distribution/Simple/InstallDirs.o ) [flags changed]
[43 of 68] Compiling Distribution.Simple.Setup ( Distribution/Simple/Setup.hs, Distribution/Simple/Setup.o ) [flags changed]
[44 of 68] Compiling Distribution.Simple.LocalBuildInfo ( Distribution/Simple/LocalBuildInfo.hs, Distribution/Simple/LocalBuildInfo.o ) [flags changed]
[45 of 68] Compiling Distribution.Simple.Build.Macros ( Distribution/Simple/Build/Macros.hs, Distribution/Simple/Build/Macros.o ) [flags changed]
[46 of 68] Compiling Distribution.Simple.BuildPaths ( Distribution/Simple/BuildPaths.hs, Distribution/Simple/BuildPaths.o ) [flags changed]
[47 of 68] Compiling Distribution.Simple.Build.PathsModule ( Distribution/Simple/Build/PathsModule.hs, Distribution/Simple/Build/PathsModule.o ) [flags changed]
[48 of 68] Compiling Distribution.Simple.LHC ( Distribution/Simple/LHC.hs, Distribution/Simple/LHC.o ) [flags changed]
[49 of 68] Compiling Distribution.Simple.Hpc ( Distribution/Simple/Hpc.hs, Distribution/Simple/Hpc.o ) [flags changed]
[50 of 68] Compiling Distribution.Simple.JHC ( Distribution/Simple/JHC.hs, Distribution/Simple/JHC.o ) [flags changed]
[51 of 68] Compiling Distribution.Simple.NHC ( Distribution/Simple/NHC.hs, Distribution/Simple/NHC.o ) [flags changed]
[52 of 68] Compiling Distribution.Simple.UHC ( Distribution/Simple/UHC.hs, Distribution/Simple/UHC.o ) [flags changed]
[53 of 68] Compiling Distribution.Simple.BuildTarget ( Distribution/Simple/BuildTarget.hs, Distribution/Simple/BuildTarget.o ) [flags changed]
[54 of 68] Compiling Distribution.Simple.Program.GHC ( Distribution/Simple/Program/GHC.hs, Distribution/Simple/Program/GHC.o ) [flags changed]
[55 of 68] Compiling Distribution.Simple.GHC ( Distribution/Simple/GHC.hs, Distribution/Simple/GHC.o ) [flags changed]
[56 of 68] Compiling Distribution.Simple.Test ( Distribution/Simple/Test.hs, Distribution/Simple/Test.o ) [flags changed]
[57 of 68] Compiling Distribution.Simple.PreProcess ( Distribution/Simple/PreProcess.hs, Distribution/Simple/PreProcess.o ) [flags changed]
[58 of 68] Compiling Distribution.Simple.Hugs ( Distribution/Simple/Hugs.hs, Distribution/Simple/Hugs.o ) [flags changed]
[59 of 68] Compiling Distribution.Simple.Configure ( Distribution/Simple/Configure.hs, Distribution/Simple/Configure.o ) [flags changed]
[60 of 68] Compiling Distribution.Simple.Install ( Distribution/Simple/Install.hs, Distribution/Simple/Install.o ) [flags changed]
[61 of 68] Compiling Distribution.Simple.Register ( Distribution/Simple/Register.hs, Distribution/Simple/Register.o ) [flags changed]
[62 of 68] Compiling Distribution.Simple.Build ( Distribution/Simple/Build.hs, Distribution/Simple/Build.o ) [flags changed]
[63 of 68] Compiling Distribution.Simple.Haddock ( Distribution/Simple/Haddock.hs, Distribution/Simple/Haddock.o ) [flags changed]
[64 of 68] Compiling Distribution.Simple.SrcDist ( Distribution/Simple/SrcDist.hs, Distribution/Simple/SrcDist.o ) [flags changed]
[65 of 68] Compiling Distribution.Simple.UserHooks ( Distribution/Simple/UserHooks.hs, Distribution/Simple/UserHooks.o ) [flags changed]
[66 of 68] Compiling Distribution.Simple.Bench ( Distribution/Simple/Bench.hs, Distribution/Simple/Bench.o ) [flags changed]
[67 of 68] Compiling Distribution.Simple ( Distribution/Simple.hs, Distribution/Simple.o ) [flags changed]
[68 of 68] Compiling Main             ( Setup.hs, Setup.o ) [flags changed]
Linking Setup ...
In file included from /Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:30,

                 from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Stg.h:65:18:
     error: math.h: No such file or directory
In file included from /Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:32,

                 from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/HsFFI.h:30:20:
     error: stdint.h: No such file or directory

In file included from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:190:20:
     error: stdlib.h: No such file or directory
In file included from /Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:227,

                 from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/rts/Flags.h:17:19:
     error: stdio.h: No such file or directory
In file included from /Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:227,

                 from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/rts/Flags.h:22:0:
     error: expected specifier-qualifier-list before ‘FILE’

/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/rts/Flags.h:188:0:
     error: expected specifier-qualifier-list before ‘FILE’
In file included from /Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:234,

                 from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/rts/Threads.h:19:23:
     error: sys/types.h: No such file or directory
In file included from /Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/Rts.h:234,

                 from /var/folders/_k/81thrwmd0g59zw2l42vh2m5r0000gn/T/ghc7931_0/ghc7931_0.c:1:0:


/Library/Frameworks/GHC.framework/Versions/7.6.3-x86_64/usr/lib/ghc-7.6.3/include/rts/Threads.h:49:0:
     error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘forkProcess’

[IreneKnapp]

By the way, the dance to get things to build if you happen to be unfortunate enough to be testing this on Mac OS X 10.9 is:

$ cd ~/blah/blah/blah/cabal/Cabal/
$ runhaskell Setup.hs install --extra-include-dirs /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/ --extra-lib-dirs /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib/
$ cd ../cabal-install
$ runhaskell Setup.hs install --extra-include-dirs /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/ --extra-lib-dirs /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib/

The usual mechanisms that make everything in this repo use itself as the build manager, rather than any installed Cabal, don't appear to work for this particular scenario, and I don't really feel like investigating why not.

[IreneKnapp]

At this point, I've got the patch written, but I'm having trouble verifying it, because of an unrelated problem keeping GHC from building executables at all (though it has no problem with libraries) on 10.9 beta 2. This looks fairly superficial - something to do with a wrong list of runtime and system libraries that need to be statically vs. dynamically linked, or something - but I'm not sure how to bootstrap my way out of it. So I'm not submitting the pull request yet, because I can't yet verify that the code actually does what I intend it to.

[cartazio]

Any way I can help with this and some of the other apple / mac compat future proofing, i would like to do so

[benjaminfjones]

Over a year later, I'm still getting bitten by this (or something closely related). I have a cabal package that depends on two C libraries installed in a local prefix. cabal install --extra-lib-dirs=<prefix>/lib works fine, but then cabal repl barfs. It is especially confusing when this is the last bit of the error message:

setup: Missing dependencies on foreign libraries:
* Missing C libraries: foo, bar
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.

and cabal repl doesn't accept those command line flags.

[tomsmeding]

This issue still exists, and turns up if you try to build Haskell code on a system with a libgmp.so that is not in the usual location (in my case because the system doesn't have libgmp.so installed at all, so I built a local one). Adding the directory containing libgmp.so to extra-lib-dirs works for most things, except for packages with custom Setup: linking the setup executable fails because -lgmp is not found, despite me providing its location in extra-lib-dirs.