Skip to content

Improve progress bar (ETA and Unicode-based rendering) #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jan 14, 2020

Conversation

tkf
Copy link
Collaborator

@tkf tkf commented Nov 10, 2019

This PR implements ETA for progress bars. close #3. It is a slightly hacky solution to use ProgressMeter.jl for generating a "smoother" progress bar and also let it compute and format ETA. At some point, we may want to vendor ProgressMeter.jl or write our own functions to do this. But I think this is a good start for now.

export TerminalLogger

const ProgressLevel = LogLevel(-1)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the level that has been used for a while in Juno https://junolab.org/ProgressLogging.jl/dev/#ProgressLogging.progress-Tuple{Any}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just been thinking more about this and about the best way represent progress.

I think it does make sense to have a default urgency associated with a @progress frontend, but I also think there's sense in having progress messages at Debug level, or progress messages at Info level. So I think progress reporting is not inherently tied to the level.

Progress is currently detected via the _id and progress key. But perhaps progress "should" be determined by the type of the message?

@info Progress("msg", fraction)

I think you suggested something similar elsewhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than type of message, why not type of value?
@info "msg" fraction=Progress(fraction)

TensorBoardLogger is all abut the type of the value,
since it logs them differently

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But perhaps progress "should" be determined by the type of the message?

@info Progress("msg", fraction)

This is an interesting approach. It's nice that all the information for "dispatch" is available as the mandatory positional argument. We can also avoid awkward empty message for logging progress.

Rather than type of message, why not type of value?
@info "msg" fraction=Progress(fraction)

That's more or less what I suggested here JuliaLogging/ProgressLogging.jl#7 (comment)

But I think it's important to support the spec that already exists. Apparently this has been used by Juno users for a while.

For the new progress logging API, let's discuss it in https://github.com/JunoLab/ProgressLogging.jl. At some point it would be nice to have similar package for documenting table logging specification and also providing some utility functions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I think it's important to support the spec that already exists.

Agreed completely. This PR just sparked the thought, so I wrote it down!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing it down! I just quoted your comment in JuliaLogging/ProgressLogging.jl#7 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, looks good.

@tkf tkf closed this Nov 10, 2019
@tkf tkf reopened this Nov 10, 2019
@codecov-io
Copy link

codecov-io commented Nov 10, 2019

Codecov Report

Merging #4 into master will decrease coverage by 1.31%.
The diff coverage is 93.84%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master       #4      +/-   ##
==========================================
- Coverage   97.69%   96.37%   -1.32%     
==========================================
  Files           3        4       +1     
  Lines         130      193      +63     
==========================================
+ Hits          127      186      +59     
- Misses          3        7       +4
Impacted Files Coverage Δ
src/TerminalLoggers.jl 100% <ø> (ø) ⬆️
src/TerminalLogger.jl 94.44% <90%> (-1.39%) ⬇️
src/ProgressMeter/ProgressMeter.jl 95.55% <95.55%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cabd0d0...1ab7759. Read the comment docs.

@oxinabox
Copy link
Member

The progress meter code is simple enough there is no reason really to use ProgressMetersfor it.

I vendored it before in

https://github.com/oxinabox/OhMyLog.jl/blob/master/src/progress.jl

Aside:
The cool way to do ETA would be to use a something more clever from OnlineStats.jl
Like a weighted estimate of speed

@@ -225,4 +225,10 @@ import TerminalLoggers.default_metafmt
\e[36m\e[1m│ \e[22m\e[39mline2
\e[36m\e[1m└ \e[22m\e[39m\e[90mSUFFIX\e[39m
"""

# Using infix operator so that `@test` prints lhs and rhs when failed:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha sure! Test really needs some work to fix this kind of thing :-/

@c42f
Copy link
Member

c42f commented Nov 11, 2019

Nice, this is pretty cool.

I kind of agree that it would make sense to copy some code snippets to avoid relying on the ProgressMeter internals. Also because terminal layout is very non-composable and I think we're likely to want to modify the printing / layout code.

@tkf tkf changed the title Use ProgressMeter to draw progress bars Improve progress bar (ETA and Unicode-based rendering) Jan 12, 2020
@tkf
Copy link
Collaborator Author

tkf commented Jan 12, 2020

ProgressMeter.jl is vendored now. I think it's good to go?

Copy link
Member

@c42f c42f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, looks good to me! Do feel free to merge when you're ready.

@tkf tkf merged commit b1b6615 into JuliaLogging:master Jan 14, 2020
@tkf tkf deleted the progressmeter branch January 14, 2020 02:05
@tkf
Copy link
Collaborator Author

tkf commented Jan 14, 2020

Thank you guys for reviewing this!

@c42f How about registering it soon-ish? I find TerminalLoggers.j pretty useful.

@c42f
Copy link
Member

c42f commented Jan 28, 2020

For sure. I reckon we register as soon as we've got some basic docs working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ETA for progress bars
4 participants