cabal new-build does not track files added via TH’s addDependentFile

[cocreature]

Steps to reproduce

• Download https://gist.github.com/cocreature/d07ef6f35f7bd9043db50794fe31e977.
• Build using cabal new-build
• Edit fileinput
• Try rebuilding using cabal new-build.

Expected behavior

cabal new-build rebuilds because fileinput has changed.

Actual behavior

cabal new-build exits without rebuilding after saying Up to date.

This issue affects both cabal-2.0 as well as the master branch.

[phadej]

One hack solution is to watch extra-source-files too. But that won't solve dynamic (e.g. what inline-c uses?) stuff.

[23Skidoo]

BTW, is there a way to force ghc --make always rerun a TH function? E.g. if I'm doing some kind of I/O at compile-time, like reading from an environment variable.

[phadej]

@23Skidoo that would force GHC to always build that module, that doesn't sound like a very good idea.

[23Skidoo]

Yep, but sometimes you want that in a build system. For example, Shake rules can be marked alwaysRerun.

Re: extra-source-files, that field is package-global and commonly used for things like README and changelog. So we'd need to allow per-component extra-source-files to avoid unnecessary recompilation.

[23Skidoo]

An alternative way to fix this is to get the list of files to be monitored from ghc --make via -ddump-module-graph (a new feature that is going to be in 8.4).

[phadej]

Does -ddump-module-graph dump addDependentFile & co stuff?

[23Skidoo]

@phadej If you click that link, you'll see that I asked that exact question there.

[cocreature]

Given that this feature will only be in GHC 8.4 I don’t think that’s a reasonable solution at this point.

[23Skidoo]

@cocreature With old versions we can simulate that feature with ghc -M (which will be slower because we'll have to do dependency resolution twice - once for ghc -M and once for ghc --make, but for legacy GHC versions we can afford to not care). BTW, ghc -M does support addDependentFile dependencies:

# DO NOT DELETE: Beginning of Haskell dependencies
Data/FileEmbed.so : Data/FileEmbed.hs
Main.so : Main.hs
Main.so : Data/FileEmbed.shi
# DO NOT DELETE: End of Haskell dependencies

[hvr]

@23Skidoo btw, we can avoid paying for the slow-legacy fallback when it's not needed, because we know when TH is needed (we already require TemplateHaskell to be declared in *-extensions in order to enable other logic specific to TH). So I think that's a workable fallback.

[hvr]

Unfortunately we not only missed the boat for GHC 8.4, but most likely also for GHC 8.6 to get supporting features into GHC... as https://phabricator.haskell.org/D3898 got stuck on a decision which design path to go down... :-/

PS: Screen-scraping GHC's dump-hi output would be an obvious but terrible hack for the obvious reasons and since it's not intended to be consumed by tooling its format is neither documented nor guaranteed to be correct and can change without notice.