longer compilation time in GMT.jl

[JeffBezanson]

Pre #43990:

jeff@durandal:~/src/julia-1.8/julia$ ./julia --project=. -e 'using GMT; @time @eval plot(rand(5,2))'
Base.IOError("could not spawn `gmt --version`: no such file or directory (ENOENT)", -2)
  4.968415 seconds (6.68 M allocations: 331.256 MiB, 1.34% gc time, 99.96% compilation time)

with #43990:

jeff@durandal:~/src/julia-1.8/julia$ ./julia --project=. -e 'using GMT; @time @eval plot(rand(5,2))'
Base.IOError("could not spawn `gmt --version`: no such file or directory (ENOENT)", -2)
  7.328750 seconds (13.14 M allocations: 658.323 MiB, 3.48% gc time, 99.97% compilation time)

Using GMT master.
Reported here: https://discourse.julialang.org/t/package-load-time-regressions-in-v1-8-beta3/78875/3

From a quick look, all parts of the compiler appear to be doing more work so I think we are just compiling more methods, possibly consistent with there being more invalidations? @timholy Any insight into why this might happen?

[rikhuijzer]

I found a more minimal reproducible example which might make debugging easier.

With https://github.com/rikhuijzer/PrecompileSignatures.jl at commit f831040 (https://github.com/rikhuijzer/PrecompileSignatures.jl/tree/f831040f0317f29edbdc1d8022ec63f9bee11c6c) and

julia --startup-file=no --project -e 'using PrecompileSignatures; @time @eval precompile_directives(PrecompileSignatures)'

julia 1.7.2

1.648769 seconds (3.16 M allocations: 172.198 MiB, 1.79% gc time, 99.51% compilation time)

julia 1.8-beta3

1.250205 seconds (1.54 M allocations: 79.923 MiB, 1.61% gc time, 99.50% compilation time)

julia 1.9.0-DEV.428 commit 3cff21e

1.349766 seconds (1.15 M allocations: 75.032 MiB, 1.42% gc time, 99.04% compilation time)

@snoopr doesn't report any invalidations.

[vtjnash]

This code has been mostly replaced

[timholy]

Sorry I never saw your ping, @JeffBezanson. GMT was heavily influenced by broadcasting-invalidation, which was an artifact of how we (mis)handled invoked function calls. While this wasn't handled correctly even before #43990, that PR made us more vulnerable to it: the chain of backedges was a() -> b() -> invoke(c, ...), but in the broadcasting pipeline, before #43990 b() simply got dropped as a specialization we couldn't cache. This avoided triggering the invalidation of a.

Fixed in #46010.

[timholy]

1.8:

julia> using GMT; @time @eval plot(rand(5,2))
  8.072855 seconds (9.65 M allocations: 483.123 MiB, 2.79% gc time, 99.88% compilation time)

1.9-beta3:

julia> using GMT; @time @eval plot(rand(5,2))
  1.578212 seconds (3.39 M allocations: 221.354 MiB, 3.76% gc time, 97.21% compilation time)

Could be improved further with more precompilation in GMT, but that's not a Julia issue.