Skip to content

JSON version of -Ztime-passes #36233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Mark-Simulacrum opened this issue Sep 3, 2016 · 7 comments
Closed

JSON version of -Ztime-passes #36233

Mark-Simulacrum opened this issue Sep 3, 2016 · 7 comments
Labels
A-frontend Area: Compiler frontend (errors, parsing and HIR) C-feature-request Category: A feature request, i.e: not implemented / a PR.

Comments

@Mark-Simulacrum
Copy link
Member

Mark-Simulacrum commented Sep 3, 2016

Summary

A -Ztime-passes-json (likely bikeshed on naming, relatedly -Zsave-analysis-api has been accepted) would produce:

{
  "phase name": {
    "rss": 100,
    "time": 0.2312,
  }
  ...
}

Motivation

This would help (internal) tooling such as perf.rlo, since it avoids re-parsing rustc's output into proper data structures.

Implementation details

While the compiler could output each pass separately, this may be difficult to implement in practice (since it would require partial JSON serialization, something neither rustc-serialize or Serde support currently, as far as I know).

Alternatives

  • Changes to the formatting of -Ztime-passes may make it easier to parse.
    • Removing seemingly arbitrary indentation (at least from my perspective).
    • Some other changes? I have not interacted with the parsing code [yet]...
  • Would an alternate "switch" that makes all output from the compiler buffer into JSON (similar to the error system) be preferable?

Unresolved Questions

  • How should users find the start/end of JSON amid other output?
    • Both when using Cargo and rustc directly.
  • How should sub-pass timings be reflected? (Likely depends what is currently done).

@nrc seems to be generally in favor, from short discussion on IRC.

@nagisa
Copy link
Member

nagisa commented Sep 3, 2016

Removing seemingly arbitrary indentation (at least from my perspective).

Identation represents pass nesting. i.e. “llvm function passes[0]” is a part of “LLVM passes”.

@Mark-Simulacrum
Copy link
Member Author

Mark-Simulacrum commented Sep 3, 2016

@nagisa That makes sense, but (in general) indentation is expected to relate to the above item, not the one below.

Example below (I expect the llvm things to be below LLVM passes if indented)

time: 0.000; rss: 72MB  Prepare MIR codegen passes
  time: 0.000; rss: 72MB    write metadata
  time: 0.001; rss: 72MB    translation item collection
  time: 0.000; rss: 74MB    codegen unit partitioning
  time: 0.000; rss: 75MB    internalize symbols
time: 0.015; rss: 75MB  translation
time: 0.000; rss: 75MB  assert dep graph
time: 0.000; rss: 75MB  serialize dep graph
  time: 0.000; rss: 75MB    llvm function passes [0]
  time: 0.000; rss: 75MB    llvm module passes [0]
  time: 0.005; rss: 77MB    codegen passes [0]
  time: 0.000; rss: 77MB    codegen passes [0]
time: 0.007; rss: 77MB  LLVM passes
time: 0.000; rss: 77MB  serialize work products
  time: 0.182; rss: 77MB    running linker
time: 0.183; rss: 78MB  linking

@retep998
Copy link
Member

retep998 commented Sep 3, 2016

@Mark-Simulacrum The times are printed out as they are obtained. This means that the substages like codegen passes and function passes occur before the entire LLVM passes stage finishes. To do otherwise would require storing that substage data somewhere and not printing it out until the overall stage is done. The downside of doing this is that you don't get immediate feedback on the last substage that just happened, so if rustc is taking a very long time on one of the later substages, the user won't know how long the previous substages were until the overall stage is done. For the json version it is fine however because it is intended to be consumed by other programs and not used as a live view of rustc's current progress, so just collecting all the data and spitting out one big json at the end is fine.

@Mark-Simulacrum
Copy link
Member Author

I understand why it's the way it is; I was just clarifying that to me the indentation is initially misleading.

If the indented passes are sub-passes, are they reflected in the total time of the parent pass? (added an unresolved question about this).

@nagisa
Copy link
Member

nagisa commented Sep 3, 2016

If the indented passes are sub-passes, are they reflected in the total time of the parent pass? (added an unresolved question about this).

Yes they are.

@dtolnay
Copy link
Member

dtolnay commented Feb 16, 2017

While the compiler could output each pass separately, this may be difficult to implement in practice (since it would require partial JSON serialization, something neither rustc-serialize or Serde support currently, as far as I know).

Serde 0.8 and 0.9 both support this. 😁

@Mark-Simulacrum Mark-Simulacrum added A-frontend Area: Compiler frontend (errors, parsing and HIR) I-wishlist labels May 23, 2017
@Mark-Simulacrum Mark-Simulacrum added C-feature-request Category: A feature request, i.e: not implemented / a PR. and removed I-wishlist labels Jul 26, 2017
@Mark-Simulacrum
Copy link
Member Author

Closing. We have the new self-profile infrastructure which more or less makes this no longer necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-frontend Area: Compiler frontend (errors, parsing and HIR) C-feature-request Category: A feature request, i.e: not implemented / a PR.
Projects
None yet
Development

No branches or pull requests

4 participants