Suggestion: write messages to stdout instead of stderr

[gvanrossum]

I often want to grep mypy's output for certain messages (or to strip certain messages) and this is a little tricky because the messages are written to stderr. So I have to do e.g.

mypy file.py 2>&1 | grep -v note

instead of

mypy file.py | grep -v note

I think the only things that should be written to stderr are command line syntax errors (where it refuses to run) or fatal errors like tracebacks. Syntax errors in the modules to be analyzed should probably go to stdout (though since these abort processing it might be okay to send these to stderr).

Also -h output should go to stdout IMO (this is how argparse does things -- it would also be nice to rewrite the argument parsing using argparse).

[refi64]

I'm really -1 on this. Literally every static type checker or compiler I know uses stderr for cough errors.

[gvanrossum]

I think it's an old tradition, but for something that doesn't produce anything on stdout it feels pretty silly. FWIW pylint, pep8 and flake8 also writes to stdout, so I think we'd actually be in good company. Apart from tradition, why are you against this? What use case would this inconvenience?

[JukkaL]

I'm okay with changing this. If mypy would behave like pylint, flake8 in this respect it probably wouldn't be surprising for most users. But I guess we could ask more people about their opinion if this is a controversial issue -- maybe there are some valid reasons for keeping the current behavior.

[darjus]

I'm with Guido on this one. Apart from Python linters writing to stdout - I usually write things that are part of normal program operation to stdout (and in this context most of mypy's output is part of normal operation), and truly erroneous things like stack traces and parse errors to sderr. This way i can grep for specific outputs with ease, and most importantly, if something horrible happens, it would not disappear if grep does not include it.

Here's how i would imagine it working:

# mypy detected type error
STDOUT: outs/int_plus_str.py: note: In function "str_plus_int":
STDOUT: outs/int_plus_str.py:4: error: Incompatible return value type: expected builtins.str, got builtins.int

# mypy could not parse the file
STDERR?: outs/parse_error.py: note: At top level:
STDERR?: outs/parse_error.py:1: error: Parse error before string literal

# mypy crashed
STDERR: Traceback (most recent call last):
STDERR:   File "bin/mypy", line 6, in <module>
STDERR:     main(__file__)
STDERR:   File "lib/python3.4/site-packages/mypy/main.py", line 50, in main
STDERR:     type_check_only(sources, bin_dir, options)
STDERR:   File "lib/python3.4/site-packages/mypy/main.py", line 93, in type_check_only
STDERR:     python_path=options.python_path)
STDERR:   File "lib/python3.4/site-packages/mypy/build.py", line 179, in build
STDERR:     raise RuntimeError("Darjus crashed mypy")
STDERR: RuntimeError: Darjus crashed mypy

[gvanrossum]

Yeah, that would be ideal. Unfortunately the way the parser errors are
generated makes it hard to send those to a different stream -- the code is
in a number of different places and they're not all fatal. We should make
sure that at least one message is written to stderr when the program's exit
status is nonzero though.

On Sun, Dec 6, 2015 at 12:58 AM, Darjus Loktevic [email protected]
wrote:

I'm with Guido on this one. Apart from Python linters writing to stdout -
I usually write things that are part of normal program operation to stdout
(and in this context most of mypy's output is part of normal operation),
and truly erroneous things like stack traces and parse errors to sderr.
This way i can grep for specific outputs with ease, and most importantly,
if something horrible happens, it would not disappear if grep does not
include it.

Here's how i would imagine it working:

mypy detected type error

STDOUT: outs/int_plus_str.py: note: In function "str_plus_int":
STDOUT: outs/int_plus_str.py:4: error: Incompatible return value type: expected builtins.str, got builtins.int

mypy could not parse the file

STDERR?: outs/parse_error.py: note: At top level:
STDERR?: outs/parse_error.py:1: error: Parse error before string literal

mypy crashed

STDERR: Traceback (most recent call last):
STDERR: File "bin/mypy", line 6, in
STDERR: main(file)
STDERR: File "lib/python3.4/site-packages/mypy/main.py", line 50, in main
STDERR: type_check_only(sources, bin_dir, options)
STDERR: File "lib/python3.4/site-packages/mypy/main.py", line 93, in type_check_only
STDERR: python_path=options.python_path)
STDERR: File "lib/python3.4/site-packages/mypy/build.py", line 179, in build
STDERR: raise RuntimeError("Darjus crashed mypy")
STDERR: RuntimeError: Darjus crashed mypy

—
Reply to this email directly or view it on GitHub
#1051 (comment).

--Guido van Rossum (python.org/~guido)

[darjus]

I'm not too familiar with mypy codebase yet but could we use standard python logging facilities?

I remember seeing an issue where Jukka discusses warning messages and that currently everything's an error in mypy and it would be useful to support both. By using logging and handlers we could support this quite easily and could give extra flexibility on what to print where/when.

log = logging.getLogger(__name__)  # or maybe different name/namespace
log.debug("Various debug messages that would be printed to STDOUT if --debug or -vv was passed in with line numbers and additional formatting information useful for debugging")
log.info("Various informational messages (for example: checking file spam/eggs.py) that would be enabled with --verbose")
log.warn("Would print STDOUT warning messages relating to various warnings in code being checked")
log.error("Woud print STDOUT errors relating to errors in code being checked")
log.critical("Would print STDERR various critical things we've touched up on above")

[gvanrossum]

IMO the logging module isn't really meant for this use case. We want the
output to be compatible with grep -n so it can be parsed by various filters
(e.g. Emacs compilation buffers).

On Sun, Dec 6, 2015 at 6:11 PM, Darjus Loktevic [email protected]
wrote:

I'm not too familiar with mypy codebase yet but could we use standard
python logging facilities?

I remember seeing an issue where Jukka discusses warning messages and that
currently everything's an error in mypy and it would be useful to support
both. By using logging and handlers we could support this quite easily and
could give extra flexibility on what to print where/when.

log = logging.getLogger(name) # or maybe different name/namespace
log.debug("Various debug messages that would be printed to STDOUT if --debug or -vv was passed in with line numbers and additional formatting information useful for debugging")log.info("Various informational messages (for example: checking file spam/eggs.py) that would be enabled with --verbose")
log.warn("Would print STDOUT warning messages relating to various warnings in code being checked")
log.error("Woud print STDOUT errors relating to errors in code being checked")
log.critical("Would print STDERR various critical things we've touched up on above")

—
Reply to this email directly or view it on GitHub
#1051 (comment).

--Guido van Rossum (python.org/~guido)

[darjus]

You're right, with different possible reporting output types it would be preferable to have pluggable outputs.
Similar to pylint or coverage.py reporter maybe?
I prefer clang error outputs to gcc and it would be awesome if we could adapt coverage.py annotate reporter for similar purpose.
Though i see mypy already has a report.py module, so maybe a better approach would be to refactor errors.py and messages.py to a default stdout reporter.

Anyway, sorry for the detour :)

https://bitbucket.org/logilab/pylint/src/4e5a1be18df2/pylint/reporters/?at=default
https://github.com/nedbat/coveragepy/blob/master/coverage/report.py
https://github.com/nedbat/coveragepy/blob/master/coverage/annotate.py
http://clang.llvm.org/diagnostics.html

[gvanrossum]

Looks like this was broken again by a978ec0. I'll fix it by adding a flag to CompileError which will be set only when it's raised from Errors.raise_error().