[nix-local-build] Recompilation avoidance ABA problem #3179
Comments
|
So if GHC had not modified any of the .o files then in principle saying there's nothing to do is correct since we are back to the old state. But the point is the interrupted build has now put the outputs into an inconsistent state. Perhaps we just want to blow away the cache if the action fails. |
|
Yes, I think the cache should be removed. I'd be very surprised if, e.g., this wasn't what Shake does (haven't checked though.) (Maybe @ndmitchell knows.) |
|
Shake checks both the inputs and the outputs are "the same", so the first time the input has changed but the output hasn't. The second time round the output has changed but the input hasn't. Either way, it's a difference of opinion, so it rebuilds. In addition, if Shake rebuilds a bit and fails it writes a failure entry into the log, basically deleting all information about that entry, so it must rebuild next time. If you hit It sounds like both those solutions solve the problem, but the first is probably easier for Cabal to integrate. |
|
Actually, I'd say the second is easier, because Cabal's current rebuild system doesn't know anything about outputs. |
|
Does Cabal's current rebuild system have any form of log about what has happened? Adding such a log can be difficult if it isn't there already. |
|
No, it's a very simple system (I keep telling Duncan we should just use Shake, but there's a lot of legacy code so the current rebuild interface helped us avoid rewriting everything in Cabal). The interface looks something like this (the API of which, BTW, I find HELLA confusing): -- | Filesystem based "keys" which we can monitor for changes, these are keys in Shake's model
data MonitorFilePath
monitorFile :: FilePath -> MonitorFilePath
monitorFileHashed :: FilePath -> MonitorFilePath
monitorNonExistentFile :: FilePath -> MonitorFilePath
monitorFileGlob :: FilePathGlob -> MonitorFilePath
-- etc...
-- | A monitor for detecting changes to a set of files. In the Shake-world, this is like
-- a non-wildcard rule, except that file monitors don't contain the code necessary to rerun the
-- rule (so unlike Shake, there is no database of rules; you just write old-fashioned
-- imperative code to do your full build). The dependencies of this "rule" are 'MonitorFilePath', plus
-- a Haskell key a. The result type of the rule is b. Every rule is associated with its own
-- unique file path which is where the state associated with that rule is cached. Note that
-- the FileMonitor is not a database of keys to values; the "key" is really just a Haskell
-- value which is also considered a dependency.
data FileMonitor a b
newFileMonitor :: Eq a => FilePath -> FileMonitor a b
-- | A monad for tracking 'MonitorFilePath's when building files.
data Rebuild a
instance Monad Rebuild
instance MonadIO Rebuild
monitorFiles :: [MonitorFilePath] -> Rebuild ()
runRebuild :: Rebuild a -> IO a
-- | Given a monitor, check if 'a' matches, and none of the recorded files in the monitor
-- have changed. If these are all unchanged, return the cached value 'b'. Otherwise,
-- rerun the 'Rebuild b' action.
rerunIfChanged :: (Binary a, Binary b) => FileMonitor a b -> a -> Rebuild b -> Rebuild b |
|
I've got a related issue which I'm not sure if it's an instance of this one or a different issue: Steps to reproduce; in Cabal's source-tree:
where EDIT: the problem appears to be that new-build marks invalidated rev-deps which need rebuilding... but doesn't persist that to the filesystem; that way on the next run cabal lost the state about invalidated rev-deps or in @dcoutts' words:
|
|
Milestoned as a must-fix when we switch over to default; but I am not sure if we are going to manage to fix this for 2.0. |
|
UPDATE I can't reproduce with the instructions below with a current cabal 2.5 snapshot; it appears that at some point the file tracking was extended to add the Here's an even easier way to reproduce w/o using CTRL-C:
|
|
My vague idea is to solve it by extending the file monitor stuff with the ability to invalidate. |
|
Unassigning this from myself since it was part of https://github.com/haskell/cabal/projects/7 and I'm not working on it now |
andreabedini
added
the
old-milestone: ⊥
Steps to reproduce:
Expected result: Cabal recompiles
Actual result: Cabal reports that there is nothing to do.
Ostensibly, the problem is that before we start a recompile, we need to invalidate the old cached entry until the recompile finishes, so that we can handle an interrupt.
The text was updated successfully, but these errors were encountered: