internal-json logger improvements: build stats changes

[qknight]

Is your feature request related to a problem?

Using the internal-json logger I've noticed something odd with the counter when building ~320 mkDerivations.

logger.hh / logger.cc

void progress(uint64_t done = 0, uint64_t expected = 0, uint64_t running = 0, uint64_t failed = 0) const { 
  result(resProgress, done, expected, running, failed); 
}

output of nix build ...

@nix {"action":"start","id":185770220454375,"level":0,"parent":0,"text":"","type":104}
@nix {"action":"result","fields":[0,1,0,0],"id":185770220454375,"type":105}
...
@nix {"action":"result","fields":[0,40,0,0],"id":185770220454375,"type":105}
...
@nix {"action":"result","fields":[0,304,0,0],"id":185770220454375,"type":105}
...
@nix {"action":"result","fields":[0,321,0,0],"id":185770220454375,"type":105}
# here it peaks and then the number runs down again
@nix {"action":"result","fields":[0,315,0,0],"id":185770220454375,"type":105}
@nix {"action":"result","fields":[0,116,0,0],"id":185770220454375,"type":105}
# probably because they could be substituded
# below start the actual compiles
@nix {"action":"result","fields":[6,8,1,0],"id":185770220454375,"type":105}
@nix {"action":"result","fields":[7,8,0,0],"id":185770220454375,"type":105}
@nix {"action":"result","fields":[7,8,1,0],"id":185770220454375,"type":105}
# now a compile error hits
@nix {"action":"result","fields":[7,7,0,1],"id":185770220454375,"type":105}
@nix {"action":"stop","id":185770220454375}

My problem is that the status line output in my cargo logger now displays:

[ 7/7 Done | 0 Running | 1 Failed ]

when it should display

[ 7/8 Done | 0 Running | 1 Failed ]

I use [ done / expected, running, failed ]

Two things which should be considered for improvement:

• Once a compile error happens, as in the scenario above, the expected counter should not be 'reset' to the previous state.
• When substitudes were used instead of builds, still keep the overall in expected but increase the counter in a new result type called "substitudes"

For UX comparison, the traditional cargo command line cargo build looks like this:

Building [===> ] 60/387: bytes, jiff, crossbeam-utils(build.rs), yoke, serde, typenum, allocator-api2, litemap, miniz_oxide, walkdir, libc, regex-syntax, memchr, tinyvec, zerocopy...

Since I also build these targets but inside nix, I would also love to show that it is ~220 derivations and not only 8.

Proposed solution

• Change the implementation of result type and keep the total instead of counting up and down again.
• Introduce a new field "substitudes" where: expected = done + running + substitudes + failed is always true.

Additional context

This log can be reproduces with:

nix build --file default.nix --log-format internal-json and one can use nix-output-monitor for checking the data. BUT I've also noticed that the nom displays different values on the expected sometimes than what was actually computed.

Checklist

• checked latest Nix manual (source)
• checked open feature issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.