Building a dynamic binary with ghc-HEAD leaves redundant RUNPATHs in a binary #4046

trofi · 2016-10-27T09:12:19Z

The setup is ghc-HEAD/Cabal-HEAD.
It's a fallout of #4025 (insecure RUNPATHs)

This bug is about redundant RUNPATHs:

$ mkdir /tmp/foo
$ cd /tmp/foo
$ cabal sandbox init
$ cabal install idiii --reinstall --enable-executable-dynamic

All hackage packages I have installed are globally installed packages.
For globally installed packages I would expect 2 RUNPATHs here:

sandbox (/tmp/foo/.cabal-sandbox/lib/x86_64-linux-ghc-8.1.20161026)
and system (/usr/lib64/x86_64-linux-ghc-8.1.20161026).

We have the as the first and the last. But also a lot of intermediates:

$ readelf -a ./.cabal-sandbox/bin/read-idiii | grep -i x86
  Machine:                           Advanced Micro Devices X86-64
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
 0x000000000000001d (RUNPATH)            Library runpath: [/tmp/foo/.cabal-sandbox/lib/x86_64-linux-ghc-8.1.20161026:/usr/lib64/hunit-1.3.1.2/ghc-8.1.20161026:/usr/lib64/missingh-1.4.0.1/ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/array-0.5.1.1:/usr/lib64/ghc-8.1.20161026/base-4.9.0.0:/usr/lib64/ghc-8.1.20161026/binary-0.8.3.0:/usr/lib64/ghc-8.1.20161026/bytestring-0.10.8.1:/usr/lib64/ghc-8.1.20161026/containers-0.5.7.1:/usr/lib64/data-accessor-0.2.2.7/ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/deepseq-1.4.2.0:/usr/lib64/ghc-8.1.20161026/directory-1.2.6.2:/usr/lib64/ghc-8.1.20161026/filepath-1.4.1.0:/usr/lib64/ghc-8.1.20161026/ghc-prim-0.5.0.0:/usr/lib64/hslogger-1.2.10/ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/integer-gmp-1.0.0.1:/usr/lib64/mtl-2.2.1/ghc-8.1.20161026:/usr/lib64/network-2.6.3.1/ghc-8.1.20161026:/usr/lib64/old-locale-1.0.0.7/ghc-8.1.20161026:/usr/lib64/old-time-1.1.0.3/ghc-8.1.20161026:/usr/lib64/parsec-3.1.11/ghc-8.1.20161026:/usr/lib64/polyparse-1.12/ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/process-1.4.2.0:/usr/lib64/random-1.1/ghc-8.1.20161026:/usr/lib64/regex-base-0.93.2/ghc-8.1.20161026:/usr/lib64/regex-compat-0.95.1/ghc-8.1.20161026:/usr/lib64/regex-posix-0.95.2/ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/rts:/usr/lib64/text-1.2.2.1/ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/time-1.6.0.1:/usr/lib64/ghc-8.1.20161026/transformers-0.5.2.0:/usr/lib64/ghc-8.1.20161026/unix-2.7.2.0:/usr/lib64/utf8-string-1.0.1.1/ghc-8.1.20161026:/usr/lib64/x86_64-linux-ghc-8.1.20161026]

MissingH example:

$ ls /usr/lib64/x86_64-linux-ghc-8.1.20161026 | wc -l
270
$ ls /usr/lib64/missingh-1.4.0.1/ghc-8.1.20161026
Control  Data  Network  System  Text  libHSMissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT.a
$ ls /usr/lib64/x86_64-linux-ghc-8.1.20161026/libHSMissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT-ghc8.1.20161026.so 
/usr/lib64/x86_64-linux-ghc-8.1.20161026/libHSMissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT-ghc8.1.20161026.so

$ cat /usr/lib64/ghc-8.1.20161026/package.conf.d/MissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT.conf 
name: MissingH
version: 1.4.0.1
id: MissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT
key: MissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT
license: BSD3
copyright: Copyright (c) 2004-2016 John Goerzen
maintainer: John Goerzen <[email protected]>
stability: Beta
homepage: http://software.complete.org/missingh
synopsis: Large utility library
description:
    MissingH is a library of all sorts of utility functions for
    Haskell programmers.  It is written in pure Haskell and thus should
    be extremely portable and easy to use.
category: Unclassified
author: John Goerzen
exposed: True
indefinite: False
exposed-modules:
    Control.Concurrent.Thread.Utils Data.BinPacking Data.Bits.Utils
    Data.CSV Data.Compression.Inflate Data.Either.Utils
    Data.Hash.CRC32.GZip Data.Hash.MD5 Data.Hash.MD5.Zord64_HARD
    Data.List.Utils Data.MIME.Types Data.Map.Utils Data.Maybe.Utils
    Data.Progress.Meter Data.Progress.Tracker Data.Quantity
    Data.String.Utils Data.Tuple.Utils Network.Email.Mailbox
    Network.Email.Sendmail Network.SocketServer Network.Utils
    System.Cmd.Utils System.Console.GetOpt.Utils System.Daemon
    System.Debian System.Debian.ControlParser System.FileArchive.GZip
    System.IO.Binary System.IO.HVFS System.IO.HVFS.Combinators
    System.IO.HVFS.InstanceHelpers System.IO.HVFS.Utils System.IO.HVIO
    System.IO.PlafCompat System.IO.StatCompat System.IO.Utils
    System.IO.WindowsCompat System.Path System.Path.Glob
    System.Path.NameManip System.Path.WildMatch System.Posix.Consts
    System.Time.Utils Text.ParserCombinators.Parsec.Utils
abi: 0569cadb3f88446d9b96862fca9e2f27
trusted: False
import-dirs: /usr/lib64/missingh-1.4.0.1/ghc-8.1.20161026
library-dirs: /usr/lib64/missingh-1.4.0.1/ghc-8.1.20161026
              /usr/lib64/x86_64-linux-ghc-8.1.20161026
data-dir: /usr/share/missingh-1.4.0.1/ghc-8.1.20161026
hs-libraries: HSMissingH-1.4.0.1-KZEaH0SMnibJN6ydQLBYpT
depends:
    HUnit-1.3.1.2-YaRHoxLBElHOIkVfiC5Gn array-0.5.1.1 base-4.9.0.0
    containers-0.5.7.1 directory-1.2.6.2 filepath-1.4.1.0
    hslogger-1.2.10-CCSTKxSqeBD3c286Ydn8Pi
    mtl-2.2.1-6qsR1PHUy5lL47Hpoa4jCM
    network-2.6.3.1-G4Up1CPKbp7DeFsnywOnGG
    old-locale-1.0.0.7-6glXNhHF891B41ZfuI8hU8
    old-time-1.1.0.3-IcvdkJUsE9M8t3io8peAEp
    parsec-3.1.11-IbsJQ6XKIEqCZ9ZfUUeVyy process-1.4.2.0
    random-1.1-54KmMHXjttlERYcr1mvsAe
    regex-compat-0.95.1-BTBAorPgudcKbaemB2vl9s time-1.6.0.1
    unix-2.7.2.0
haddock-interfaces: /usr/share/doc/missingh-1.4.0.1/html/MissingH.haddock
haddock-html: /usr/share/doc/missingh-1.4.0.1/html

Seems Cabal does not populate dynamic-library-dirs here.

I built MissingH as:

./setup configure --ghc --prefix=/usr --with-compiler=/usr/bin/ghc --with-hc-pkg=/usr/bin/ghc-pkg --prefix=/usr --libdir=/usr/lib64 --libsubdir=missingh-1.4.0.1/ghc-8.1.20161026 --datadir=/usr/share/ --datasubdir=missingh-1.4.0.1/ghc-8.1.20161026 --disable-tests --ghc-options=-j8 +RTS -A256M -qb0 -RTS --ghc-option=-Wall --ghc-option=-O2 --ghc-option=-optc-O2 --ghc-option=-optc-pipe --ghc-option=-optc-fdiagnostics-show-option --ghc-option=-optc-frecord-gcc-switches --ghc-option=-optl-Wl,-O1 --ghc-option=-optl-Wl,--as-needed --ghc-option=-optl-Wl,--hash-style=gnu --disable-executable-stripping --docdir=/usr/share/doc/missingh-1.4.0.1 --verbose --enable-shared --enable-executable-dynamic --sysconfdir=/etc --disable-library-stripping

The text was updated successfully, but these errors were encountered:

ezyang · 2016-10-27T09:19:46Z

The configure command didn't have a dynlibdir set. Maybe this is just a bug where sandbox isn't configuring the dynlibdir?

ezyang · 2016-10-27T09:25:25Z

Actually, should be much simpler than that; we just have to uncomment this:

libraryDynDirSupported :: Compiler -> Bool
libraryDynDirSupported comp = case compilerFlavor comp of
  GHC ->
    -- GHC patch still hasn't landed yet. When it does, update
    -- this date and uncomment
    {-
      -- Not just v >= mkVersion [8,0,1,20161021], as there
      -- are many GHC 8.1 nightlies which don't support this.
    ((v >= mkVersion [8,0,1,20161021] && v < mkVersion [8,1]) ||
      v >= mkVersion [8,1,20161021])
      -} False
  _   -> False

christiaanb · 2016-10-27T10:20:21Z

GHC(-HEAD) does not use a shared directory for dynamic libraries of the base packages, there's nothing that Cabal can do about this. So even with the dynlibdir support, until GHC does something about the layout of its base packages you will always end up with:

1 RPATH for the user db/sandbox db packages
1 RPATH for the packages you installed in the global package database
1 RPATH per dependency on a package in the global package database that are bundled with GHC, which are:
- Cabal
- array
- base
- binary
- bytestring
- containers
- deepseq
- directory
- filepath
- ghc
- ghc-boot
- ghc-boot-th
- ghc-prim
- ghci
- haskeline
- hoopl
- hpc
- integer-gmp
- pretty
- process
- rts
- template-haskell
- terminfo
- time
- transformers
- unix
- xhtml

So, if you just want 2 RPATHs (global db + user/sandbox db), then file a bug report/feature request with GHC, so that the bundled dynamic libraries end up in a single shared directory.

ezyang · 2016-10-27T10:22:27Z

I just pushed a commit to master to turn version testing back on. Should help.

christiaanb · 2016-10-27T10:22:38Z

However, as @ezyang noted, Cabal-HEAD should uncomment the libraryDynDirSupported code.

trofi · 2016-10-27T20:50:36Z

After @ezyang 's f48e17b commit MissingH
now has proper libraries in .conf file:

...
library-dirs: /usr/lib64/missingh-1.4.0.1/ghc-8.1.20161026
dynamic-library-dirs: /usr/lib64/x86_64-linux-ghc-8.1.20161026
...

And RUNPATH is shortened for read-idiii:

$ readelf -a ./.cabal-sandbox/bin/read-idiii | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [/tmp/foo/.cabal-sandbox/lib/x86_64-linux-ghc-8.1.20161026:/usr/lib64/ghc-8.1.20161026/array-0.5.1.1:/usr/lib64/ghc-8.1.20161026/base-4.9.0.0:/usr/lib64/ghc-8.1.20161026/binary-0.8.3.0:/usr/lib64/ghc-8.1.20161026/bytestring-0.10.8.1:/usr/lib64/ghc-8.1.20161026/containers-0.5.7.1:/usr/lib64/ghc-8.1.20161026/deepseq-1.4.2.0:/usr/lib64/ghc-8.1.20161026/directory-1.2.6.2:/usr/lib64/ghc-8.1.20161026/filepath-1.4.1.0:/usr/lib64/ghc-8.1.20161026/ghc-prim-0.5.0.0:/usr/lib64/ghc-8.1.20161026/integer-gmp-1.0.0.1:/usr/lib64/ghc-8.1.20161026/process-1.4.2.0:/usr/lib64/ghc-8.1.20161026/rts:/usr/lib64/ghc-8.1.20161026/time-1.6.0.1:/usr/lib64/ghc-8.1.20161026/transformers-0.5.2.0:/usr/lib64/ghc-8.1.20161026/unix-2.7.2.0:/usr/lib64/x86_64-linux-ghc-8.1.20161026]

31 -> 17 reduction! 15 of them are GHC-bundled libs. I'll report the rest to GHC.

Thank you!

trofi mentioned this issue Oct 27, 2016

Don't use installedPkgs for internal library library dirs. #4033

Merged

ezyang closed this as completed Oct 29, 2016

ezyang added this to the 2.0 milestone Oct 29, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Building a dynamic binary with ghc-HEAD leaves redundant RUNPATHs in a binary #4046

Building a dynamic binary with ghc-HEAD leaves redundant RUNPATHs in a binary #4046

trofi commented Oct 27, 2016

ezyang commented Oct 27, 2016

Uh oh!

ezyang commented Oct 27, 2016

Uh oh!

christiaanb commented Oct 27, 2016

Uh oh!

ezyang commented Oct 27, 2016

Uh oh!

christiaanb commented Oct 27, 2016

Uh oh!

trofi commented Oct 27, 2016

Uh oh!

Building a dynamic binary with ghc-HEAD leaves redundant RUNPATHs in a binary #4046

Building a dynamic binary with ghc-HEAD leaves redundant RUNPATHs in a binary #4046

Comments

trofi commented Oct 27, 2016

ezyang commented Oct 27, 2016

Uh oh!

ezyang commented Oct 27, 2016

Uh oh!

christiaanb commented Oct 27, 2016

Uh oh!

ezyang commented Oct 27, 2016

Uh oh!

christiaanb commented Oct 27, 2016

Uh oh!

trofi commented Oct 27, 2016

Uh oh!