Building time with musl fails / boot packages in `plan.nix`?

[nomeata]

After my success of cross-building for Windows with this setup, I enthusiastically tackled the next task: Building with musl in order to get a fully static Linux executable out.

My current attempt simply passes crossSystem = localLib.systems.examples.musl64 (entropia/tip-toi-reveng@aff912d) and it actually builds a cross-compiling GHC, and some packages (e.g. transformers) build.

But then it tries to build is time, and that fails:

~/projekte/tip-toi-reveng $ nix-build nix -A static-exe
these derivations will be built:
  /nix/store/87ky4yb3x12245rrha081jppi99ds8vq-x86_64-unknown-linux-musl-ghc-8.4.4-with-unordered-containers.drv
  /nix/store/0b3jsm71qn5f35ny952s44y2360qzi71-unordered-containers-0.2.9.0-lib-unordered-containers-x86_64-unknown-linux-musl.drv
  /nix/store/7xaib1dax0f9mkazr3i5bgm361r7fbs2-time-1.8.0.2-lib-time-x86_64-unknown-linux-musl.drv
…
  /nix/store/1phb4ifd1biq108249z0s29k8n2vkdl1-x86_64-unknown-linux-musl-ghc-8.4.4-with-tttool.drv
  /nix/store/9yvh40psh1v9xkic821kw3r7jaxqx8bz-tttool-1.8-exe-tttool-x86_64-unknown-linux-musl.drv
building '/nix/store/7xaib1dax0f9mkazr3i5bgm361r7fbs2-time-1.8.0.2-lib-time-x86_64-unknown-linux-musl.drv'...
unpacking sources
unpacking source archive /nix/store/sn2g8zld91yx9zni0fqwv65dw6vq3afi-time-1.8.0.2.tar.gz
source root is time-1.8.0.2
setting SOURCE_DATE_EPOCH to timestamp 1494705576 of file time-1.8.0.2/changelog.md
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
Configure flags:
--prefix=/nix/store/h42hvhrlghb0ri8k64nblrd78fibs26l-time-1.8.0.2-lib-time-x86_64-unknown-linux-musl lib:time --package-db=clear --package-db=/nix/store/8yx0534zb14f5mbl52yk0fjargq76gja-time-1.8.0.2-lib-time-config-x86_64-unknown-linux-musl/package.conf.d --with-ghc=x86_64-unknown-linux-musl-ghc --with-ghc-pkg=x86_64-unknown-linux-musl-ghc-pkg --with-hsc2hs=x86_64-unknown-linux-musl-hsc2hs --with-gcc=x86_64-unknown-linux-musl-cc --with-ld=x86_64-unknown-linux-musl-ld --with-ar=x86_64-unknown-linux-musl-ar --with-strip=x86_64-unknown-linux-musl-strip --enable-executable-stripping --enable-library-stripping --enable-split-sections --enable-static --hsc2hs-option=--cross-compile --hsc2hs-option=--via-asm --configure-option=--host=x86_64-unknown-linux-musl
Configuring library for time-1.8.0.2..
Loaded package environment from /nix/store/8yx0534zb14f5mbl52yk0fjargq76gja-time-1.8.0.2-lib-time-config-x86_64-unknown-linux-musl/ghc-environment
checking for x86_64-unknown-linux-musl-gcc... /nix/store/6mn2kz9109afg48wpm7f121fdfqkwxmn-x86_64-unknown-linux-musl-stage-final-gcc-debug-wrapper-7.3.0/bin/x86_64-unknown-linux-musl-cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /nix/store/6mn2kz9109afg48wpm7f121fdfqkwxmn-x86_64-unknown-linux-musl-stage-final-gcc-debug-wrapper-7.3.0/bin/x86_64-unknown-linux-musl-cc accepts -g... yes
checking for /nix/store/6mn2kz9109afg48wpm7f121fdfqkwxmn-x86_64-unknown-linux-musl-stage-final-gcc-debug-wrapper-7.3.0/bin/x86_64-unknown-linux-musl-cc option to accept ISO C89... none needed
checking how to run the C preprocessor... /nix/store/6mn2kz9109afg48wpm7f121fdfqkwxmn-x86_64-unknown-linux-musl-stage-final-gcc-debug-wrapper-7.3.0/bin/x86_64-unknown-linux-musl-cc -E
checking for grep that handles long lines and -e... /nix/store/fr6apsdp46f9m1290gxgfmhfc4rpmcjw-gnugrep-3.1/bin/grep
checking for egrep... /nix/store/fr6apsdp46f9m1290gxgfmhfc4rpmcjw-gnugrep-3.1/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking for gmtime_r... yes
checking for localtime_r... yes
checking for clock_gettime... yes
checking whether struct tm is in sys/time.h or time.h... time.h
checking for struct tm.tm_zone... yes
checking whether time.h and sys/time.h may both be included... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking whether altzone is declared... no
configure: creating ./config.status
config.status: creating lib/include/HsTimeConfig.h
building
Preprocessing library for time-1.8.0.2..
hsc2hs: Failed to extract integer
CallStack (from HasCallStack):
  error, called at utils/hsc2hs/CrossCodegen.hs:606:27 in main:CrossCodegen
builder for '/nix/store/7xaib1dax0f9mkazr3i5bgm361r7fbs2-time-1.8.0.2-lib-time-x86_64-unknown-linux-musl.drv' failed with exit code 1
cannot build derivation '/nix/store/yvaaakfwx94knb1b09npcj7l3v760lai-tttool-1.8-exe-tttool-config-x86_64-unknown-linux-musl.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9yvh40psh1v9xkic821kw3r7jaxqx8bz-tttool-1.8-exe-tttool-x86_64-unknown-linux-musl.drv': 1 dependencies couldn't be built
error: build of '/nix/store/9yvh40psh1v9xkic821kw3r7jaxqx8bz-tttool-1.8-exe-tttool-x86_64-unknown-linux-musl.drv' failed

Not sure what the error message means. But I also wonder why it builds time at all, as this is already contained in the ghc package:

~/projekte/tip-toi-reveng $ find /nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/ -name \*time\*
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/package.conf.d/time-1.8.0.2.conf
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/time-1.8.0.2
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/time-1.8.0.2/libHStime-1.8.0.2_p.a
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/time-1.8.0.2/libHStime-1.8.0.2.a
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/time-1.8.0.2/HStime-1.8.0.2.o
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/directory-1.3.1.5/System/Directory/Internal/C_utimensat.p_hi
/nix/store/6q8kwqnihrz3mcdaa08465fvwqw319pf-x86_64-unknown-linux-musl-ghc-8.4.4/lib/x86_64-unknown-linux-musl-ghc-8.4.4/directory-1.3.1.5/System/Directory/Internal/C_utimensat.hi

I tried to compare it with building Haskell via nixpkgs, so see if this really is an issue with haskell.nix (compared to nixpkgs), but over there no time package is exposed:

~/projekte/tip-toi-reveng/nix-static $ nix-build -A pkgsCross.musl64.haskellPackages.time nixpkgs
error: expression does not evaluate to a derivation (or a set or list of those)

(the time field is present, but null).

So maybe the problem is that time shouldn’t actually be built here?

[nomeata]

What also doesn’t work: Removing time (and other GHC-provided packages) from plan.nix: aeson seems to depend on it.

[nomeata]

Maybe https://ghc.haskell.org/trac/ghc/ticket/14889, which has a patch against hsc2hs?
Maybe this (or some other patch) applied when building within ghc, but it is not used when building time exernally?

[nomeata]

I am experimenting with

diff --git a/builder/comp-builder.nix b/builder/comp-builder.nix
index 87cd54f..7cd0c2a 100644
--- a/builder/comp-builder.nix
+++ b/builder/comp-builder.nix
@@ -169,7 +169,8 @@ let
       ++ lib.optional (deadCodeElimination && stdenv.hostPlatform.isLinux) "--enable-split-sections"
       ++ lib.optional (static) "--enable-static"
       ++ lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) (
-        map (arg: "--hsc2hs-option=" + arg) ["--cross-compile" "--via-asm"]
+        [ "--hsc2hs-option=--cross-compile" ]
+        ++ lib.optional (stdenv.hostPlatform.isWindows) "--hsc2hs-option=--via-asm"
         ++ lib.optional (package.buildType == "Configure") "--configure-option=--host=${stdenv.hostPlatform.config}" )
       ++ component.configureFlags
   );

under the somewhat random assumption that musl cross-compilation isn’t cross-compilingishly enough to need --via-asm, and it seems I made it past building time.

[nomeata]

This looks good. With this change, and the following hack(?) in the output of cabal-to-nix:

diff --git a/nix/tttool.nix b/nix/tttool.nix
index 678f1c9..9856027 100644
--- a/nix/tttool.nix
+++ b/nix/tttool.nix
@@ -31,6 +31,13 @@
     components = {
       exes = {
         "tttool" = {
+          configureFlags = pkgs.lib.optionals (pkgs.hostPlatform.isMusl) [
+             "--ghc-option=-static"
+             "--ghc-option=-optl=-static"
+             "--extra-lib-dirs=${pkgs.gmp6.override { withStatic = true; }}/lib"
+             "--extra-lib-dirs=${pkgs.zlib.static}/lib"
+          ];
           depends = (([
             (hsPkgs.base)
             (hsPkgs.binary)

(to simulate --enable-executable-dynamic which was added to cabal only recently at haskell/cabal#5446 by nh2) it seems I can almost link it. I am now stuck because nixpkgs does not seem to provide a static version of libtinfo, which I have reported already at
nh2/static-haskell-nix#14

[nomeata]

Success!

After

~/projekte/tip-toi-reveng $ git diff nix/plan.nix 
diff --git a/nix/plan.nix b/nix/plan.nix
index 4b81166..147ce08 100644
--- a/nix/plan.nix
+++ b/nix/plan.nix
@@ -8,6 +8,7 @@ hackage:
       "polyparse".revision = (((hackage."polyparse")."1.12").revisions).default;
       "old-time".revision = (((hackage."old-time")."1.1.0.3").revisions).default;
       "haskeline".revision = (((hackage."haskeline")."0.7.4.2").revisions).default;
+      "haskeline".flags.terminfo = false;
       "split".revision = (((hackage."split")."0.2.3.3").revisions).default;
       "stm".revision = (((hackage."stm")."2.4.5.1").revisions).default;
       "HTF".revision = (((hackage."HTF")."0.13.2.5").revisions).default;

I can build a static version of the program:

~/projekte/tip-toi-reveng $ file $(nix-build -A static-exe nix)/bin/tttool
/nix/store/sn0p4q19l3gs88bb2vd2bcdc3jm4j27y-tttool-1.8-exe-tttool-x86_64-unknown-linux-musl/bin/tttool: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped

and my default.nix ends in this pleasing derivation:

in
 { linux-exe = tttool-exe {};
   windows-exe = tttool-exe { crossSystem = localLib.systems.examples.mingwW64;};
   static-exe = tttool-exe { crossSystem = localLib.systems.examples.musl64;};
 }

I’ll leave this open for you to assess the fix that applied to builder/comp-builder.nix.

[angerman]

@nomeata you can put all those customizations into your pkgs.nix at

https://github.com/entropia/tip-toi-reveng/blob/3183b6ff9be55701de78e71482ba5d6820300019/nix/pkgs.nix#L32

just add the following module:

{
   packages.haskeline.flags.terminfo = false;
   packages.tttool.configureFlags = pkgs.lib.optionals (pkgs.hostPlatform.isMusl) [
             "--ghc-option=-static"
             "--ghc-option=-optl=-static"
             "--extra-lib-dirs=${pkgs.gmp6.override { withStatic = true; }}/lib"
             "--extra-lib-dirs=${pkgs.zlib.static}/lib"
          ];
}

regarding the --via-asm; I thought that was fixed in a recent hsc2hs, but the change would work. I'll add something to input-output-hk/haskell.nix. Thanks!