Dhall-haskell nix build appears to be broken

[KirinDave]

I'd expect that if I checked out master and typed in nix-build, that I would produce the dhall results. However, on multiple builds with both nix-1.11 and nix-2* I get an identical error, suggesting a misconfigured build in the dependencies:

building path(s) ‘/nix/store/pjg437r1gglsns3slj3r55za50d04hz2-interactive-dhall-1.14.0-environment’
unpacking sources
variable $src or $srcs should point to the source
builder for ‘/nix/store/djg55rm4s6gz7qpbcb9kmx1y812l0lb3-interactive-dhall-1.14.0-environment.drv’ failed with exit code 1

Since dhall-text in nixpkgs-unstable is currently demanding dhall-1.14 and that can't be satisfied, it's difficult to deploy the latest version of dhall or related tooling.

[Gabriella439]

@KirinDave: nix-build --attr dhall should work

The reason for the issue is that default.nix also bundles the shell environment under the shell attribute which is only valid for nix-shell and can't be built using nix-build

[Gabriella439]

@KirinDave: Also, regarding the Nixpkgs issue, both dhall and dhall-text are on Stackage and up-to-date, meaning that the latest version of each package builds with each other and all the other packages on Stackage. That means that the Nixpkgs build failure is probably a Nixpkgs-specific issue

[KirinDave]

Perhaps there are larger issues in nix's unstable branch right now, @Gabriel439, because building that attribute does seem to fail on freshly installed nix systems. On a freshly installed VM I find:

username@hostname:~/Projects/staging/dhall-haskell$ nix-build --attr dhall
these derivations will be built:
  /nix/store/p4p8kmy2jl74h8jwsfpdffyb7sabbpn7-dhall-1.14.0.drv
building '/nix/store/p4p8kmy2jl74h8jwsfpdffyb7sabbpn7-dhall-1.14.0.drv'...
setupCompilerEnvironmentPhase
Build with /nix/store/mmhdqindrpvg91a2vnrr76vxcn5a4n0x-ghc-8.2.2.
ignoring (possibly broken) abi-depends field for packages
unpacking sources
unpacking source archive /nix/store/ya4x4mq0f7qygqgi7azq09cm2a6blfrg-dhall-haskell
source root is dhall-haskell
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/tmp/nix-build-dhall-1.14.0.drv-0/package.conf.d -j4 -threaded
<command line>: cannot satisfy -package-id basement-0.0.7-d0185eeb1592a454451853306f684f8b455b75663162ae9e1f76ddecaa952753
    (use -v for more information)
builder for '/nix/store/p4p8kmy2jl74h8jwsfpdffyb7sabbpn7-dhall-1.14.0.drv' failed with exit code 1
error: build of '/nix/store/p4p8kmy2jl74h8jwsfpdffyb7sabbpn7-dhall-1.14.0.drv' failed

[Gabriella439]

@KirinDave: The dhall derivation built by default.nix is pinned to a specific version of nixpkgs using the trick described here:

https://nixos.wiki/wiki/How_to_fetch_Nixpkgs_with_an_empty_NIX_PATH

See:

dhall-haskell/default.nix

Lines 4 to 10 in d91905d

	nixpkgs = fetchNixpkgs {
	rev = "804060ff9a79ceb0925fe9ef79ddbf564a225d47";
	sha256 = "01pb6p07xawi60kshsxxq1bzn8a0y4s5jjqvhkwps4f5xjmmwav3";
	outputSha256 = "0ga345hgw6v2kzyhvf5kw96hf60mx5pbd9c4qj5q4nan4lr7nkxn";
	};

So the revision of Nixpkgs shouldn't be a reason for the different behavior. That pinned revision also corresponds to the 18.03 branch of Nixpkgs, not the unstable branch.

Also, the dhall derivation is built by CI and blocks merge into master:

https://hydra.dhall-lang.org/job/dhall-haskell/master/dhall

... so I'm still inclined to think that this issue may be due to an impurity specific to your system. I know that you said that you tested this on a fresh VM, but I still want to double-check the following

• Have you built dhall using cabal before using nix-build? More specifically, has cabal left behind a dist/ sub-directory in your checkout of the dhall repository?
• Have you installed other Haskell packages installed globally (i.e. using cabal install outside of Nix)? More specifically, do you have a non-empty ~/.ghc directory?
• Are you building the latest master of dhall for a fresh checkout with no modifications?

[ocharles]

I feel like #661 might go towards fixing this too, as it at least consolidates all Nix expressions. @KirinDave - did you make any progress on your side?

[masaeedu]

Would it be possible to use cachix and some kind of CI to ensure a build of dhall-haskell (along with its transitive dependencies) is available in the binary cache? Very new to both Nix in general and cachix in particular, so apologies if the question doesn't make any sense.

I just cloned master into a clean machine and ran nix-build, which is taking quite a long time.

[ocharles]

This is actually already possible due to the use of Hydra. Simply add https://hydra.dhall-lang.org to your binary cache list. I actually prefer to just switch to root and run nix-store -r /nix/store/a-known-dhall-build --option extra-substituters https://hydra.dhall-lang.org so I don't always use the Dhall cache and add too much extra load. You can get the store path of a known build by going to the details tab of a page like https://hydra.dhall-lang.org/build/4659#tabs-details

[Gabriella439]

@masaeedu: Yeah, as @ocharles already mentioned, hydra.dhall-lang.org already serves its own cache, so you can configure your system to use Hydra's cache by adding these lines to /etc/nix/nix.conf:

substituters = https://cache.nixos.org https://cache.dhall-lang.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.dhall-lang.org:I9/H18WHd60olG5GsIjolp7CtepSgJmM2CsO813VTmM= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

Or if you use NixOS, the corresponding NixOS options would be:

  nix = {
    binaryCaches = [ "https://cache.nixos.org" "https://cache.dhall-lang.org" ];

    binaryCachePublicKeys = [
      "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
      "cache.dhall-lang.org:I9/H18WHd60olG5GsIjolp7CtepSgJmM2CsO813VTmM="
    ];
  };

Then if you build using the Nix derivations provided by this repository it should closely match what hydra.dhall-lang.org already built and cached as part of CI and you will fetch most build products from cache

[masaeedu]

Thanks @ocharles, @Gabriel439!

[Gabriella439]

@masaeedu: You're welcome! 🙂

[Gabriella439]

I'll also go ahead and close this since I'm pretty sure the Dhall Nix build is correctly working since it is regularly exercised by Hydra and I've also recently tested nix-build and nix-shell for all of the projects.