Do not buffer output

[blueyed]

When piping pytests output e.g. into tee it will get buffered.

To work around this, you can e.g. export PYTHONUNBUFFERED=1 or add -u to the shebang line of the pytest script (see http://stackoverflow.com/questions/107705/disable-output-buffering).

I think it makes sense to enable it by default.

[The-Compiler]

I disagree - output buffering is there for a reason, and basically every application buffers its output by default. I don't think there's a good reason to do anything different than that.

[blueyed]

IIRC the buffering does not happen when running on a terminal anyway (the common use case).
It is unusual to redirect its output, and when this gets done nobody relies on or benefits really from it being buffered?!

[flub]

Isn't the "normal" unix behaviour to detect if it's being run in a pipeline and then disable the buffering? In fact I'm surprised python does not behave like that by default. Unless there's something we do in pytest which actually disables this? If so I'd say this is still a bug, we should behave fine either on terminal or in a pipe.

[RonnyPfannschmidt]

this is a python bug

[blueyed]

So we could disable the output buffering explicitly then?

FWIW, to make control characters (e.g. cursor keys, but also C-a) work with pdb.set_trace(), you need to use script around pytest/Python anyway, which then also disables the output buffering:

script -q -c 'pytest …' /dev/null | tee /tmp/foo

Via http://stackoverflow.com/a/39269661/15690.

[RonnyPfannschmidt]

this is handled at the python level, - we can enforce flushes

what use case are you trying to implement?

[blueyed]

My use case is using https://github.com/janko-m/vim-test with https://github.com/radenling/vim-dispatch-neovim: vim-dispatch-neovim uses a Neovim terminal instance and that causes lines to be wrapped at its width (which might be really narrow in a tmux split etc).
Therefore I've thrown in | tee there to capture pytest's output into a file directly (instead of reading it from the terminal buffer) - radenling/vim-dispatch-neovim#7.

[RonnyPfannschmidt]

just do a python -B -m pytest ?

[RonnyPfannschmidt]

i mean for that use case py.test should have a non-gui stdout output format that allows to present things nicely

ideally flushed json-lines or anything that can be piped to a presentation layer directly without enforcing the normal output text limitations

[blueyed]

@RonnyPfannschmidt
How is -B relevant her?

   -B     Don't write .py[co] files on import. See also  PYTHONDONTWRITEBYTE‐
          CODE.

py.test should have a non-gui stdout output format that allows to present things nicely

Agreed.

[RonnyPfannschmidt]

whops, my error, i meant -u

[RonnyPfannschmidt]

@blueyed can this be closed as upstream python bug?

[blueyed]

@RonnyPfannschmidt
Not sure?!
After all we could set pytest up accordingly, and it could be argued to not change this in Python by default.
python -B -m pytest might be a good enough workaround though..

[RonnyPfannschmidt]

@blueyed still an issue for pip/a pep

[blueyed]

@RonnyPfannschmidt
What do you mean? That it's not a pytest issue?
pytest might be able to set this up in its (entrypoint) script..

[RonnyPfannschmidt]

@blueyed no, setuptools/pip doesn't support that in entry-point scripts and i'm inclined to NOT invent it because it will be an maintenance pain wit regular platform issues

[nicoddemus]

Please also note that python supports setting this through an PYTHONUNBUFFERED environment variable:

-u     : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
         see man page for details on internal buffering relating to '-u'

If the heart of this issue is to use pytest in a vim plugin, I think making the plugin set the environment variable before calling the entry point is perfectly acceptable.

Like @RonnyPfannschmidt I also do not want to change the default behavior as we can't know the repercussions of such a fundamental change, and introducing an option to set this seems like overkill when python already provides two ways for users to change that setting.

[blueyed]

Closing it then.
Thanks!

[lesgrieve]

I am using py.test version 2.4.2 and Python version 2.6.6 and want to view ANSI colours within less. These are some of the approaches which do not work

export PYTHONUNBUFFERED=1
py.test |& less -r

python -u -m pytest |& less -r

stdbuf --error=0 py.test | less -r

stdbuf --error=0 python -m pytest | less -r

What does work for me is

sudo yum install -y expect
unbuffer py.test |& less -r

[itayB]

export PYTHONUNBUFFERED=1 + py.test my_test.py -s works for me.
-s == 'disable all capturing'

[schneems]

To add that -s is a shortcut for --capture=no (which I previously couldn't find and landed on this issue as the top result).

$ pytest
# ...
  --capture=method      Per-test capturing method: one of fd|sys|no|tee-sys
  -s                    Shortcut for --capture=no