cabalProjects don't accept `cache` parameter

[angerman]

Originally posted by @NinjaTrappeur in #370 (comment)

It appears that cabalProjects don't support the cache argument. This should be rectified.

[hamishmack]

cabalProject must run cabal new-configure before it can run plan-to-nix. The code that makes the --sha256 comment work downloads the source for each source-repository-package and makes a modified cabal.project file that references the downloaded versions (so it can run cabal new-configure without triggering any downloads).

The best way to make cabalProject work with the cache argument will probably be to adapt the code that extracts the source-repository-package sections form the cabal.project file. If it does not find a --sha256 comment, it can look in the cache for a suitable sha256 based on the other source-repository-package attributes.

[Warbo]

This is also needed to make hackage-package work when a package's hackage tarball contains a cabal.project file with external sources. For example, pandoc-2.9.1: https://hackage.haskell.org/package/pandoc-2.9.1.1/src/cabal.project

Here's the definition which I ran into this issue (I'm using a fork of haskell.nix for i686-linux support):

with builtins;
with rec {
  nixpkgsSrc = fetchTarball {
    name   = "nixpkgs1909";
    url    = https://github.com/NixOS/nixpkgs/archive/19.09.tar.gz;
    sha256 = "0mhqhq21y5vrr1f30qd2bvydv4bbbslvyzclhw0kdxmkgg3z4c92";
  };

  haskellNixSrc = fetchTarball {
    name   = "haskell-nix";
    url    = https://github.com/Warbo/haskell.nix/archive/499a761.tar.gz;
    sha256 = "1pnkywswfa71hgc2c3g2cijfk9nysbpyh6jjh455h810n4yhs522";
  };

  pkgs = import nixpkgsSrc { overlays = import "${haskellNixSrc}/overlays"; };

  pandoc = pkgs.haskell-nix.hackage-package {
    name        = "pandoc";
    version     = "2.9.1";
    index-state = "2020-01-11T00:00:00Z";
    ghc         = pkgs.buildPackages.pkgs.haskell-nix.compiler.ghc865;
  };
};
pandoc.components.exes.pandoc

This gives:

$ nix-build --show-trace pandoc.nix
trace: Using index-state: 2020-01-11T00:00:00Z for pandoc
trace: Get `plan-sha256` with `nix-hash --base32 --type sha256 /nix/store/y14nsnwd0liipafpkr43n8z0dn4033gd-pandoc-plan-to-nix-pkgs/`
building '/nix/store/hjly4x6k98bqid3ivrr4ar0ls17iglyj-pandoc-plan-to-nix-pkgs.drv'...
Cloning into '/build/tmp.lTuIDCiKYt/dist-newstyle/src/pandoc-ci_-9734aa59'...
fatal: unable to access 'https://github.com/jgm/pandoc-citeproc/': Could not resolve host: github.com
builder for '/nix/store/hjly4x6k98bqid3ivrr4ar0ls17iglyj-pandoc-plan-to-nix-pkgs.drv' failed with exit code 128
error: while evaluating 'hackage-package' at /nix/store/k8mmpmmp8rwd0hkmwiwr3icfc1r3ivqw-haskell-nix/overlays/haskell.nix:391:27, called from /home/chris/DELETEME/pandoc.nix:19:12:
while evaluating the attribute 'hsPkgs.pandoc' at /nix/store/k8mmpmmp8rwd0hkmwiwr3icfc1r3ivqw-haskell-nix/overlays/haskell.nix:420:42:
while evaluating the attribute 'config' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:87:25:
while evaluating 'yieldConfig' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:74:29, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:73:16:
while evaluating 'mergeModules' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:190:26, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:64:17:
while evaluating 'mergeModules'' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:194:36, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:191:5:
while evaluating 'flip' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/trivial.nix:101:16, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:237:6:
while evaluating 'byName' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:217:25, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:225:21:
while evaluating 'reverseList' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/lists.nix:393:17, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:64:38:
while evaluating 'filterModules' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:93:31, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:64:51:
while evaluating 'closeModules' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:101:27, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:62:16:
while evaluating 'imap1' at /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/lists.nix:116:14, called from /nix/store/lj152b1wmfafjgxf6aqr5x75dqmfiw0y-nixpkgs1909/lib/modules.nix:112:20:
while evaluating the attribute 'pkgs' at /nix/store/k8mmpmmp8rwd0hkmwiwr3icfc1r3ivqw-haskell-nix/lib/import-and-filter-project.nix:10:3:
build of '/nix/store/hjly4x6k98bqid3ivrr4ar0ls17iglyj-pandoc-plan-to-nix-pkgs.drv' failed

I've tried adding a cache argument, but it doesn't do anything. From looking at the definition of hackage-package, it appears to accept arbitrary arguments (i.e. ...) and passes them into cabalProject', so I think adding support for cache to cabalProject' will fix this "for free".

I can make it work by specifying a cabalProject argument which includes a sha256 comment, but I'm not sure if that would work for dependencies (do dependencies' cabal.project files get read by cabal or haskell.nix?)

[balsoft]

Any chance of getting back to this issue? Do you think we should implement something similar to automatic cache generation for stackage projects here too?

[hamishmack]

I think #554 will help. It will use builtins.fetchGit if there is no --sha256 comment int the cabal.project. This will not help if access it restricted though, so you can also provide a lookupSha256 that will be used to determine the sha256 when it is not present.

with builtins;
with rec {
  haskellNix = import (fetchTarball {
    name   = "haskell-nix";
    url    = https://github.com/input-output-hk/haskell.nix/archive/cb0ab3a2.tar.gz;
    sha256 = "1ghhs2ii4ndyzlg8qjb8z3fnjsr6fd90ixn1b81lnxv40x6rkvza";
  }) {};

  pkgs = import haskellNix.sources.nixpkgs-default haskellNix.nixpkgsArgs;

  pandoc = pkgs.haskell-nix.hackage-package {
    name         = "pandoc";
    version      = "2.9.1";
    index-state  = "2020-01-11T00:00:00Z";
    ghc          = pkgs.buildPackages.pkgs.haskell-nix.compiler.ghc865;
    lookupSha256 = repo:
      { "https://github.com/jgm/pandoc-citeproc"."0.16.4.1"
          = "0z7cn1lz3gjllx69q7zqv9w9f16sdcsgps02d4qs43rcyf3pjcdh"; }
        ."${repo.location}"."${repo.tag}";
  };
};
pandoc.components.exes.pandoc

[michaelpj]

That said, if there's a method for doing this for stack projects, making it consistent for cabal projects would be nice!

[hamishmack]

That said, if there's a method for doing this for stack projects, making it consistent for cabal projects would be nice!

The cabalProject requirements are slightly different, so I don't think it will make sense to try to copy what the stackProject cache argument does. In particular for the stackPackage cache to work you have to know the names of the packages (including for each sub directory in the repo). @balsoft help make a cache generator that does the hard work by building the cache from the stack.yaml here. It has to download the repo and use the cabal-name tool from nix-tools to get the name.

For cabalProject the cabal configure step reads the .cabal files and puts everything we need in plan.json (including the names of all the packages), so cabalProject just needs the sha256 of the repo.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[hamishmack]

I'm going to close this. I think inputMap and sha256Map do what we need.