assert file not in self.flushed_files (without MYPY_PATH) #7510

chriselion · 2019-09-13T20:19:35Z

Are you reporting a bug, or opening a feature request?

Bug

Please insert below the code you are checking with mypy, or a mock-up repro if the source is private. We would appreciate if you try to simplify your case to a minimal repro.

mypy_repro.tar.gz
This is taken from https://github.com/Unity-Technologies/ml-agents/tree/develop - there are a lot of other large files in there, so this is only the relevant python files.

What is the actual behavior/output?

$ mypy --ignore-missing-imports --disallow-incomplete-defs --namespace-packages --show-traceback --cache-dir=/dev/null ml-agents/setup.py  ml-agents/mlagents/trainers/components/reward_signals/curiosity/model.py 
ml-agents/mlagents/trainers/components/reward_signals/gail/model.py:223: error: "LearningModel" has no attribute "selected_actions"
ml-agents/mlagents/trainers/components/reward_signals/gail/model.py:242: error: "LearningModel" has no attribute "selected_actions"
ml-agents/mlagents/trainers/components/bc/model.py:57: error: "LearningModel" has no attribute "output"
ml-agents/mlagents/trainers/components/bc/model.py:63: error: "LearningModel" has no attribute "all_log_probs"
ml-agents/mlagents/trainers/components/reward_signals/curiosity/model.py:125: error: "LearningModel" has no attribute "selected_actions"
ml-agents/mlagents/trainers/components/reward_signals/curiosity/model.py:142: error: "LearningModel" has no attribute "selected_actions"
ml-agents/mlagents/trainers/components/reward_signals/curiosity/model.py:159: error: "LearningModel" has no attribute "selected_actions"
ml-agents/mlagents/trainers/components/reward_signals/curiosity/model.py:125: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.730+dev.28423cd78062b81365ec9aaa591b4e823af1606d
Traceback (most recent call last):
  File "/Users/chris.elion/code/mlagents_mypy/ml-agents/venv/bin/mypy", line 10, in <module>
    sys.exit(console_entry())
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/__main__.py", line 8, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/main.py", line 88, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/build.py", line 164, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache, stdout, stderr)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/build.py", line 229, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/build.py", line 2566, in dispatch
    process_graph(graph, manager)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/build.py", line 2875, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/build.py", line 2974, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/build.py", line 2070, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 289, in check_first_pass
    self.accept(d)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 927, in accept
    return visitor.visit_class_def(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 1671, in visit_class_def
    self.accept(defn.defs)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 992, in accept
    return visitor.visit_block(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 1867, in visit_block
    self.accept(s)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 668, in accept
    return visitor.visit_func_def(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 716, in visit_func_def
    self._visit_func_def(defn)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 720, in _visit_func_def
    self.check_func_item(defn, name=defn.name())
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 782, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 953, in check_func_def
    self.accept(item.body)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 992, in accept
    return visitor.visit_block(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 1867, in visit_block
    self.accept(s)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 1183, in accept
    return visitor.visit_if_stmt(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 2987, in visit_if_stmt
    self.accept(b)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 992, in accept
    return visitor.visit_block(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 1867, in visit_block
    self.accept(s)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 396, in accept
    stmt.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 1050, in accept
    return visitor.visit_assignment_stmt(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 1901, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checker.py", line 2036, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 3575, in accept
    typ = node.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 1532, in accept
    return visitor.visit_call_expr(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 263, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 337, in visit_call_expr_inner
    object_type, member)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 705, in check_call_expr_with_callee_type
    object_type=object_type)[0]
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 768, in check_call
    return self.check_any_type_call(args, callee)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 1833, in check_any_type_call
    self.infer_arg_types_in_empty_context(args)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 919, in infer_arg_types_in_empty_context
    arg_type = self.accept(arg)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 3575, in accept
    typ = node.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 1532, in accept
    return visitor.visit_call_expr(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 263, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 337, in visit_call_expr_inner
    object_type, member)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 705, in check_call_expr_with_callee_type
    object_type=object_type)[0]
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 768, in check_call
    return self.check_any_type_call(args, callee)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 1833, in check_any_type_call
    self.infer_arg_types_in_empty_context(args)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 919, in infer_arg_types_in_empty_context
    arg_type = self.accept(arg)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 3575, in accept
    typ = node.accept(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/nodes.py", line 1484, in accept
    return visitor.visit_member_expr(self)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 1859, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkexpr.py", line 1874, in analyze_ordinary_member_access
    in_literal_context=self.is_literal_context())
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkmember.py", line 117, in analyze_member_access
    result = _analyze_member_access(name, typ, mx, override_info)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkmember.py", line 134, in _analyze_member_access
    return analyze_instance_member_access(name, typ, mx, override_info)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkmember.py", line 210, in analyze_instance_member_access
    return analyze_member_var_access(name, typ, info, mx)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/checkmember.py", line 405, in analyze_member_var_access
    return mx.msg.has_no_attr(mx.original_type, itype, name, mx.context)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/messages.py", line 265, in has_no_attr
    code=codes.ATTR_DEFINED)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/messages.py", line 147, in fail
    self.report(msg, context, 'error', code=code, file=file, origin=origin)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/messages.py", line 137, in report
    code=code)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/errors.py", line 310, in report
    self.add_error_info(info)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/errors.py", line 341, in add_error_info
    self._add_error_info(file, info)
  File "/Users/chris.elion/code/ml-agents/venv/lib/python3.7/site-packages/mypy/errors.py", line 313, in _add_error_info
    assert file not in self.flushed_files
AssertionError:

What is the behavior/output you expect?

No assert (I still expect errors from my code though).

What are the versions of mypy and Python you are using?

$ python --version
Python 3.7.3
$ mypy --version
mypy 0.730+dev.28423cd78062b81365ec9aaa591b4e823af1606d

This also happens using mypy==0.7.2

What are the mypy flags you are using? (For example --strict-optional)

--ignore-missing-imports --disallow-incomplete-defs --namespace-packages --show-traceback --cache-dir=/dev/null
--cache-dir was just to simplify the repro

If mypy crashed with a traceback, please paste the full traceback below.

See above

This seems similar to some other issues where this assert was hit (e.g. #4881, #6717) but in those, MYPY_PATH was set and here it's not.

The text was updated successfully, but these errors were encountered:

JukkaL · 2019-09-16T13:29:40Z

I was able to reproduce the crash. If I comment out the assert, everything seems to work okay.

ilevkivskyi · 2019-09-16T13:42:19Z

Isn't this a duplicate of #4881?

JukkaL · 2019-09-16T14:00:43Z

Isn't this a duplicate of #4881?

Based on a quick investigation it wasn't clear to me whether the same fix would work for both issues. They are closely related at least. In this issue the key seems to be --namespace-packages, whereas #4881 doesn't use it but uses mypy_path.

The root cause seems to be the same -- mypy assigns multiple module names to a file, based on different package root paths.

chriselion · 2019-09-16T16:32:54Z

Thanks, glad the repro worked.

Given that this assert is the cause of a few issues, is it worth removing it (or maybe converting to a warning instead)?

JukkaL · 2019-09-16T16:50:19Z

Given that this assert is the cause of a few issues, is it worth removing it (or maybe converting to a warning instead)?

Yeah, maybe it should be turned into an error message. When we hit it something has gone wrong, but the crash is really unhelpful.

chriselion · 2019-09-16T18:10:45Z

OK, are you going to do that, or should I try it out?

JukkaL · 2019-09-16T18:14:03Z

@chriselion I'm going to be busy with other things for the next week and a half, so if you'd like to try it out, please go ahead!

chriselion · 2019-09-27T01:38:10Z

Hi @JukkaL,
Sorry, finally trying to get back to this. First attempt is in #7567, although I could use some guidance on where to add tests.

chriselion · 2019-11-02T21:15:18Z

@JukkaL Were you ever able to look into this further? My PR to disable the assert got some pushback, and I'm still not sure how to diagnose and resolve what's causing a file to get processed multiple times.

I'm currently running mypy without --namespace-packages but this means it's missing out on flagging some legitimate problems that need to be fixed.

hauntsaninja · 2022-07-27T20:30:39Z

We added a check for "Source file found twice under different module names" a long time back that means this no longer reproes

JukkaL added bug mypy got something wrong crash priority-0-high labels Sep 16, 2019

chriselion mentioned this issue Sep 27, 2019

Replace assert with warning #7567

Closed

chriselion mentioned this issue Nov 13, 2019

Move mlagents.envs Unity-Technologies/ml-agents#2902

Closed

This was referenced Dec 7, 2019

Improve mypy coverage by adding --namespace-packages Unity-Technologies/ml-agents#3049

Merged

Rename mlagents.envs to mlagents_envs Unity-Technologies/ml-agents#3083

Merged

ilevkivskyi mentioned this issue Mar 27, 2020

AssertionError in untyped function definition #8587

Closed

hauntsaninja closed this as completed Jul 27, 2022

hauntsaninja mentioned this issue Sep 26, 2022

Internal error depending on mypy_path in setup.cfg file #8770

Closed

CoolCat467 mentioned this issue Nov 27, 2022

AssertionError: assert file not in self.flushed_files #14202

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

assert file not in self.flushed_files (without MYPY_PATH) #7510

assert file not in self.flushed_files (without MYPY_PATH) #7510

chriselion commented Sep 13, 2019

JukkaL commented Sep 16, 2019

Uh oh!

ilevkivskyi commented Sep 16, 2019

Uh oh!

JukkaL commented Sep 16, 2019

Uh oh!

chriselion commented Sep 16, 2019

Uh oh!

JukkaL commented Sep 16, 2019

Uh oh!

chriselion commented Sep 16, 2019

Uh oh!

JukkaL commented Sep 16, 2019

Uh oh!

chriselion commented Sep 27, 2019

Uh oh!

chriselion commented Nov 2, 2019

Uh oh!

hauntsaninja commented Jul 27, 2022

Uh oh!

Uh oh!

assert file not in self.flushed_files (without MYPY_PATH) #7510

assert file not in self.flushed_files (without MYPY_PATH) #7510

Comments

chriselion commented Sep 13, 2019

JukkaL commented Sep 16, 2019

Uh oh!

ilevkivskyi commented Sep 16, 2019

Uh oh!

JukkaL commented Sep 16, 2019

Uh oh!

chriselion commented Sep 16, 2019

Uh oh!

JukkaL commented Sep 16, 2019

Uh oh!

chriselion commented Sep 16, 2019

Uh oh!

JukkaL commented Sep 16, 2019

Uh oh!

chriselion commented Sep 27, 2019

Uh oh!

chriselion commented Nov 2, 2019

Uh oh!

hauntsaninja commented Jul 27, 2022

Uh oh!