handle '_DuplicateWriter' object has no attribute 'buffer'

[nchantarotwong]

Handle error case (I believe due to the unittest-xml-reporting module) where sys.stdout has no buffer attribute, causing a failure and hang in subunit:

protocol = TestProtocolServer(result, self._passthrough, self._forward)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/venv/lib/python3.11/site-packages/subunit/init.py", line 510, in init
stream = sys.stdout.buffer
^^^^^^^^^^^^^^^^^
AttributeError: '_DuplicateWriter' object has no attribute 'buffer'

unittest-xml-reporting/result.py overriding sys.stdout, which doesn't include a buffer:

def _setupStdout(self):
    """
    Capture stdout / stderr by replacing sys.stdout / sys.stderr
    """
    super(_XMLTestResult, self)._setupStdout()
    self.__stdout_saved = sys.stdout
    sys.stdout = _DuplicateWriter(sys.stdout, self._stdout_capture)
    self.__stderr_saved = sys.stderr
    sys.stderr = _DuplicateWriter(sys.stderr, self._stderr_capture)

[jelmer]

I don't think this works. We're trying to write bytes to this stream, but sys.stdout would normally be accepting str only.

[nchantarotwong]

I've been using this change for a while and I do believe it works, because _DuplicateWriter() is acting as sys.stdout (it overrides the actual sys.stdout) and expects bytes:

class _DuplicateWriter(io.TextIOBase):
"""
Duplicate output from the first handle to the second handle
The second handle is expected to be a StringIO and not to block.
"""

def __init__(self, first, second):
    super(_DuplicateWriter, self).__init__()
    self._first = first
    self._second = second

def flush(self):
    self._first.flush()
    self._second.flush()

def writable(self):
    return True

def getvalue(self):
    return self._second.getvalue()

def writelines(self, lines):
    self._first.writelines(lines)
    self._second.writelines(lines)

def write(self, b):
    if isinstance(self._first, io.TextIOBase):
        wrote = self._first.write(b)

        if wrote is not None:
            # expected to always succeed to write
            self._second.write(b[:wrote])

        return wrote
    else:
        # file-like object that doesn't return wrote bytes.
        self._first.write(b)
        self._second.write(b)
        return len(b)

[jelmer]

Right, but a regular sys.stdout would not accept bytes. So this might a good workaround for your situation, it's not a proper fix.

% python3
Python 3.11.1 (main, Dec 31 2022, 10:23:59) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdout.write(b'bla')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: write() argument must be str, not bytes

Not all platforms provide sys.stdout.buffer, so we should be handling this properly. But we can't expect sys.stdout to handle bytes.

[nchantarotwong]

added type checking for sys.stdout being overridden by unittest-xml-reporting without trying to import the module

[jelmer]

the problem here is that unittest-xml-reporting sets sys.stdout to an odd value; this problem should be fixed in unittest-xml-reporting, rather than worked around in subunit.

Alternatively, unittest-xml-reporting could pass in a stream explicitly, in which case subunit wouldn't be trying to access sys.stdout.buffer at all

[nchantarotwong]

Ok, can just scrap this PR then. But it seems like if sys.stdout is overriden by potentially anything (for capture purposes) or buffer is not available like you said, this is still not being handled here.

[jelmer]

Right, subunit can only cope with sys.stdout when it has a buffer attribute that it can use to write byte streams (since sys.stdout should be handling regular strings). We can come across situations in which there is no buffer attribute on some platforms (Python on Android doesn't have it I believe) and if sys.stdout is overridden.

There isn't really a good way to deal with a lack of buffer attribute though, since subunit needs to write byte streams. The best thing to do in those situations is for the caller to pass in a byte stream.

[jelmer]

Closing this per discussion