Liskov substitution paradox when inheriting from io.StringIO (and probably other io types)

[solsword]

I've just read through this bug:

#6076

which has finally answered my question, and I'm hoping that as an ugly partial fix, we could at least have a special message warning users what's going on if they ever try to inherit from io.StringIO and similar and override a method. Or if I'm wrong about the root issue here, I'd love to have a deeper discussion of what's going on.

To explain further: I've written a class that inherits from io.StringIO. Then I try to override a method (writelines in my case). I declare the argument type for my override as Iterable[str], and mypy complains this is Liskov-substitution-incompatible with the IOBase supertype which has this argument as an Iterable[Buffer]. So I go, "okay, fine, no big deal, I'll just import Buffer from typing_extensions and change it. But now I get the same complaint, this time about TextIOBase with argument type Iterable[str]. This... makes no sense to me, but after diving down various internet rabbit holes I finally come across the bug linked above which implies that it's in fact impossible to properly declare a type for that argument, due to the weirdness in IO stuff. I can't just follow the suggestion to use typing.IO here because I actually do need to inherit from io.StringIO for functionality (Is there a way to at-runtime inherit from one class but tell mypy we're actually a different class' subtype? I'm guessing not).

An example to reproduce this:

from typing import Iterable, Any
from typing_extensions import Buffer


class A(TextIOBase):
    def writelines(self, lines: Iterable[str]) -> None: # E: Argument 1 of "w…
        pass

class B(TextIOBase):
    def writelines(self, lines: Iterable[Buffer]) -> None: # E: Argument 1 of…
        pass

class C(TextIOBase):
    def writelines(self, lines: Iterable[Any]) -> None:
        pass

In this example, I get the following mypy output:

t.py:8: error: Argument 1 of "writelines" is incompatible with supertype "IOBase"; supertype defines the argument type as "Iterable[Buffer]"  [override]
t.py:8: note: This violates the Liskov substitution principle
t.py:8: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
t.py:12: error: Argument 1 of "writelines" is incompatible with supertype "TextIOBase"; supertype defines the argument type as "Iterable[str]"  [override]
t.py:12: note: This violates the Liskov substitution principle
t.py:12: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
Found 2 errors in 1 file (checked 1 source file)

So neither A nor B works, and C is the closest reasonable thing I can do (I went with an Iterable[str] plus # type: ignore instead).

I realize the underlying issue may be intractable, but I'd have really appreciated a special note with my error messages letting me know that it's impossible to correctly type this, and linking to an explanation in the docs.

[srittau]

This sounds like a bug in mypy to me. When doing LSP checks, mypy should only go up (in MRO order) to the next class that implements this method and check compatibility. It shouldn't continue up the MRO hierarchy after that.

[srittau]

Without using typeshed:

class Base:
    def foo(self, x: int) -> None:
        pass


class Sub1(Base):
    def foo(self, x: str) -> None:  # type: ignore[override]
        pass


class Sub2(Sub1):
    def foo(self, x: str) -> None:  # Argument 1 of "foo" is incompatible with supertype "Base"; supertype defines the argument type as "int"  [override]
        pass

[srittau]

This has been reported before here: python/mypy#9643. (Similar situation with overriding StringIO.) Closing it here, thanks for reporting!

[Akuli]

(Is there a way to at-runtime inherit from one class but tell mypy we're actually a different class' subtype? I'm guessing not).

Not sure if it helps, but yes, there is a way:

import io
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    _BaseClass = SomeClass  # mypy thinks you inherit from this class
else:
    _BaseClass = io.StringIO  # at runtime you actually inherit from this class

class MyStringIO(_BaseClass):
    ...

[solsword]

Thanks @Akuli , that's a useful way of doing things that I didn't think of!

And thanks @srittau for confirming this. Sorry that I didn't find the other bug before reporting a duplicate; that one would have perfectly explained my situation!