Skip to content

Internal Error w/ aio-pika==1.7.0 mypy=0.550 #4370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
xlevus opened this issue Dec 15, 2017 · 8 comments
Closed

Internal Error w/ aio-pika==1.7.0 mypy=0.550 #4370

xlevus opened this issue Dec 15, 2017 · 8 comments
Assignees
@ilevkivskyi
Labels
crash priority-1-normal

Comments

@xlevus
Copy link

xlevus commented Dec 15, 2017

Reporting an INTERNAL_ERROR as requested.

The error appears to happen both from within my project and when running directy against aio_pika.

I am not the developer of aio-pika.

$ pip freeze | grep "mypy\|aio-pika" 
aio-pika==1.7.0
flake8-mypy==17.8.0
mypy==0.550

$ mypy -p common --show-traceback                                                                                                                                                               
/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/aio_pika/patterns/rpc.py:186: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.550
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/__main__.py", line 11, in <module>
    main(None)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/main.py", line 66, in main
    res = type_check_only(sources, bin_dir, options)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/main.py", line 119, in type_check_only
    options=options)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/build.py", line 214, in build
    graph = dispatch(sources, manager)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/build.py", line 1935, in dispatch
    process_graph(graph, manager)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/build.py", line 2199, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/build.py", line 2327, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/build.py", line 1836, in type_check_first_pass
    self.type_checker.check_first_pass()
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 196, in check_first_pass
    self.accept(d)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 690, in accept
    return visitor.visit_class_def(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 1185, in visit_class_def
    self.accept(defn.defs)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 750, in accept
    return visitor.visit_block(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 1305, in visit_block
    self.accept(s)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 567, in accept
    return visitor.visit_decorator(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 2470, in visit_decorator
    self.check_func_item(e.func, name=e.func.name())
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 591, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 751, in check_func_def
    self.accept(item.body)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 750, in accept
    return visitor.visit_block(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 1305, in visit_block
    self.accept(s)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 952, in accept
    return visitor.visit_try_stmt(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 2265, in visit_try_stmt
    self.visit_try_without_finally(s, try_frame=bool(s.finally_body))
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 2300, in visit_try_without_finally
    self.accept(s.body)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 750, in accept
    return visitor.visit_block(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 1305, in visit_block
    self.accept(s)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 794, in accept
    return visitor.visit_assignment_stmt(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 1312, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checker.py", line 1406, in check_assignment
    self.infer_variable_type(inferred, lvalue, self.expr_checker.accept(rvalue),
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checkexpr.py", line 2363, in accept
    typ = node.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 1245, in accept
    return visitor.visit_yield_from_expr(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checkexpr.py", line 2486, in visit_yield_from_expr
    subexpr_type = self.accept(e.expr)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checkexpr.py", line 2363, in accept
    typ = node.accept(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/nodes.py", line 1235, in accept
    return visitor.visit_call_expr(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checkexpr.py", line 268, in visit_call_expr
    ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname, object_type)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checkexpr.py", line 505, in check_call_expr_with_callee_type
    object_type=object_type)[0]
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/checkexpr.py", line 563, in check_call
    callee = freshen_function_type_vars(callee)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/expandtype.py", line 46, in freshen_function_type_vars
    fresh = cast(CallableType, expand_type(callee, tvmap)).copy_modified(variables=tvdefs)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/expandtype.py", line 16, in expand_type
    return typ.accept(ExpandTypeVisitor(env))
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/types.py", line 748, in accept
    return visitor.visit_callable_type(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/expandtype.py", line 97, in visit_callable_type
    return t.copy_modified(arg_types=self.expand_types(t.arg_types),
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/expandtype.py", line 132, in expand_types
    a.append(t.accept(self))
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/types.py", line 1424, in accept
    return visitor.visit_forwardref_type(self)
  File "/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/mypy/types.py", line 1509, in visit_forwardref_type
    raise RuntimeError('Internal error: unresolved forward reference')
RuntimeError: Internal error: unresolved forward reference
/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/aio_pika/patterns/rpc.py:186: : note: use --pdb to drop into pdb

$ mypy -p aio_pika 
... <snip> ...
/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/aio_pika/patterns/rpc.py:186: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.550
/home/USER/.virtualenvs/backoffice-py/lib64/python3.6/site-packages/aio_pika/patterns/rpc.py:186: : note: please use --show-traceback to print a traceback when reporting a bug
@ilevkivskyi
Copy link
Member

Hm, this looks important. Could you please provide the code you checked with mypy that triggers this (or some hints about it, if it is private). Are there union types or generics involved? It looks like this is triggered by some assignment statement that looks like this:

x = yield from function_call(...)

In particular could you please tell are there forward references (annotations with types that are defined later in code) involved?

@ilevkivskyi ilevkivskyi self-assigned this Dec 15, 2017
@xlevus
Copy link
Author

xlevus commented Dec 15, 2017 •
edited
Loading

@ilevkivskyi I am able to reproduce the INTERNAL_ERROR independently from my project with the following:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip install mypy=0.550 aio-pika==1.7.0
$ MYPYPATH=venv/lib/python3.6/site-packages mypy -p aio_pika --show-traceback

I don't actually use the aio_pika.packages.rpc module (this appears to be an addition ~6 hours ago). Given this, and the re-produce-ability outside of my project, I don't expect the error is caused by any of my code.

I'm also not a contributor to aio-pika, and have worked-around this by ignoring the module in my mypy.ini.

@ilevkivskyi
Copy link
Member

OK, so it involves type-checking something in site-packages. This is discouraged until PEP 561 is fully implemented, so this is a lower priority. I think I have found the root cause, they write:

P = [TypeVar('P')]
# ... then later
    result = yield from self.execute(func, payload)
# ... and even later
    @asyncio.coroutine
    def execute(self, func: Callable[P, R], payload: P) -> R:
        ...

@gvanrossum
Copy link
Member

gvanrossum commented Dec 15, 2017 •
edited
Loading

We got the same exception (though with a somewhat different traceback). The user sent me a repro (update: which I have not managed to confirm):

class BaseClass(object):
    def some_method(self):
        # type: () -> BaseClass
        pass
class SubClass1(BaseClass):
    def some_method(self):
        # type: () -> AnotherNameForSubClass1  # <--- Forward ref to alias causes crash
        pass
AnotherNameForSubClass1 = SubClass1

The traceback ended in:

File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checker.py", line 2101, in visit_return_stmt
self.check_return_stmt(s)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checker.py", line 2132, in check_return_stmt
allow_none_return=allow_none_func_call)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checkexpr.py", line 2365, in accept
typ = node.accept(self)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/nodes.py", line 1245, in accept
return visitor.visit_call_expr(self)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checkexpr.py", line 268, in visit_call_expr
ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname, object_type)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checkexpr.py", line 505, in check_call_expr_with_callee_type
object_type=object_type)[0]
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checkexpr.py", line 578, in check_call
messages=arg_messages)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checkexpr.py", line 1048, in check_argument_types
actual + 1, i + 1, callee, context, messages)
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/checkexpr.py", line 1080, in check_arg
elif not is_subtype(caller_type, callee_type):
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/subtypes.py", line 83, in is_subtype
ignore_declared_variance=ignore_declared_variance))
File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/types.py", line 1415, in accept
    return visitor.visit_forwardref_type(self)
  File "/Users/xxx/src/server/.mypy/venv/lib/python3.5/site-packages/mypy/types.py", line 1500, in visit_forwardref_type
    raise RuntimeError('Internal error: unresolved forward reference')
RuntimeError: Internal error: unresolved forward reference

@ilevkivskyi
Copy link
Member

@gvanrossum I have a fix for the original crash, but I cannot reproduce your crash, there should be something else.

@ilevkivskyi ilevkivskyi mentioned this issue Dec 16, 2017
Merged
@ilevkivskyi
Copy link
Member

PR #4375 should fix the original crash.

gvanrossum pushed a commit that referenced this issue Dec 16, 2017
@ilevkivskyi @gvanrossum
Process the decorator's Vars (#4375)
8861e28 
Fixes original crash in #4370
@gvanrossum gvanrossum mentioned this issue Dec 16, 2017
Closed
@gvanrossum
Copy link
Member

I've made another issue for the crash my use saw: #4376

@mitar
Copy link

mitar commented Dec 19, 2017

Maybe related: #4385

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ilevkivskyi ilevkivskyi

Labels
crash priority-1-normal
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@xlevus @mitar @gvanrossum @ilevkivskyi