Skip to content

Bug, while stubbing a third party library locally #5695

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mchlnix opened this issue Sep 29, 2018 · 4 comments
Closed

Bug, while stubbing a third party library locally #5695

mchlnix opened this issue Sep 29, 2018 · 4 comments
Labels
crash priority-1-normal

Comments

@mchlnix
Copy link

mchlnix commented Sep 29, 2018

These two crashes happened, while working on stubbing pycryptocome. Not sure what caused them, other than the errors caught by mypy itself.

I'm trying to stub the Crypto.Hash.* files of the library in a top level directory stubs/Crypto/Hash. Stubs for Crypto.Cipher.* exist already, locally as well.

The crash was maybe caused by me erroneously putting a __init__.pyi file into the stubs dir as well, causing it to become a package. When I removed the file, leaving the ones in Crypto and its subdirs, it worked again.

mypy -s --show-traceback stubs/ Test_Client.py EntryPoint.py ExitPoint.py Mix.py Recv.py
Warning: --silent-imports has been replaced by --ignore-missing-imports --follow-imports=skip
stubs/Crypto/Cipher/PKCS1_OAEP.pyi:9: error: Invalid type "Crypto.PublicKey.RSA"
stubs/Crypto/Cipher/PKCS1_v1_5.pyi:7: error: Invalid type "Crypto.PublicKey.RSA"
stubs/Crypto/Cipher/PKCS1_OAEP.pyi:9: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.640-dev
Traceback (most recent call last):
  File "/home/michael/.local/bin/mypy", line 11, in <module>
    load_entry_point('mypy==0.640.dev0', 'console_scripts', 'mypy')()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/main.py", line 95, in main
    res = type_check_only(sources, bin_dir, options, flush_errors, fscache)  # noqa
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/main.py", line 159, in type_check_only
    fscache=fscache)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 184, in build
    flush_errors, fscache)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 357, in _build
    graph = dispatch(sources, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2544, in dispatch
    process_graph(graph, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2841, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2958, in process_stale_scc
    graph[id].semantic_analysis()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2084, in semantic_analysis
    self.manager.semantic_analyzer.visit_file(self.tree, self.xpath, self.options, patches)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 302, in visit_file
    self.accept(d)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 3589, in accept
    node.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 845, in accept
    return visitor.visit_class_def(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 760, in visit_class_def
    defn.defs.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 910, in accept
    return visitor.visit_block(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 1641, in visit_block
    self.accept(s)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 3589, in accept
    node.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 608, in accept
    return visitor.visit_func_def(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 371, in visit_func_def
    self._visit_func_def(defn)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 442, in _visit_func_def
    defn.type = analyzer.visit_callable_type(defn.type, nested=False)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/typeanal.py", line 421, in visit_callable_type
    ret = t.copy_modified(arg_types=self.anal_array(t.arg_types, nested=nested),
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/typeanal.py", line 639, in anal_array
    res.append(self.anal_type(t, nested))
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/typeanal.py", line 646, in anal_type
    return t.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/types.py", line 242, in accept
    return visitor.visit_unbound_type(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/typeanal.py", line 182, in visit_unbound_type
    typ = self.visit_unbound_type_nonoptional(t)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/typeanal.py", line 330, in visit_unbound_type_nonoptional
    self.fail('Invalid type "{}"'.format(name), t)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/typeanal.py", line 565, in fail
    self.fail_func(msg, ctx)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 3569, in fail
    self.errors.report(ctx.get_line(), ctx.get_column(), msg, blocker=blocker)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 261, in report
    self.add_error_info(info)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 282, in add_error_info
    self._add_error_info(file, info)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 264, in _add_error_info
    assert file not in self.flushed_files
AssertionError: 
stubs/Crypto/Cipher/PKCS1_OAEP.pyi:9: : note: use --pdb to drop into pdb
Makefile:15: recipe for target 'lint' failed
make: *** [lint] Error 1

...

9:     def __init__(self, key: RSA, hashAlgo: HashAlgorithm, mgfunc: Callable, label: Union[bytes, bytearray, memoryview], randfunc: Callable) -> None: ...

mypy -s --show-traceback stubs/ Test_Client.py EntryPoint.py ExitPoint.py Mix.py Recv.py
Warning: --silent-imports has been replaced by --ignore-missing-imports --follow-imports=skip
stubs/Crypto/Cipher/_mode_cbc.pyi:11: error: The return type of "__init__" must be None
stubs/Crypto/Cipher/_mode_cbc.pyi:11: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.640-dev
Traceback (most recent call last):
  File "/home/michael/.local/bin/mypy", line 11, in <module>
    load_entry_point('mypy==0.640.dev0', 'console_scripts', 'mypy')()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/main.py", line 95, in main
    res = type_check_only(sources, bin_dir, options, flush_errors, fscache)  # noqa
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/main.py", line 159, in type_check_only
    fscache=fscache)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 184, in build
    flush_errors, fscache)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 357, in _build
    graph = dispatch(sources, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2544, in dispatch
    process_graph(graph, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2841, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2964, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2104, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 266, in check_first_pass
    self.accept(d)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 365, in accept
    stmt.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 845, in accept
    return visitor.visit_class_def(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 1483, in visit_class_def
    self.accept(defn.defs)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 365, in accept
    stmt.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 910, in accept
    return visitor.visit_block(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 1635, in visit_block
    self.accept(s)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 365, in accept
    stmt.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 608, in accept
    return visitor.visit_func_def(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 676, in visit_func_def
    self._visit_func_def(defn)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 680, in _visit_func_def
    self.check_func_item(defn, name=defn.name())
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 742, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 775, in check_func_def
    item)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/checker.py", line 3613, in fail
    self.msg.fail(msg, context)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/messages.py", line 199, in fail
    self.report(msg, context, 'error', file=file, origin=origin)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/messages.py", line 194, in report
    origin_line=origin.get_line() if origin else None)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 261, in report
    self.add_error_info(info)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 282, in add_error_info
    self._add_error_info(file, info)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 264, in _add_error_info
    assert file not in self.flushed_files
AssertionError: 
stubs/Crypto/Cipher/_mode_cbc.pyi:11: : note: use --pdb to drop into pdb
Makefile:15: recipe for target 'lint' failed
make: *** [lint] Error 1

...

11:    def __init__(self, block_cipher: SmartPointer, iv: Union[bytes, bytearray, memoryview]): ...
@ilevkivskyi
Copy link
Member

Just to clarify, do you still see this problem when using mypy version from GitHub master?

@emmatyping
Copy link
Member

(You can install mypy from master with pip install git+https://github.com/python/mypy.git)

@mchlnix
Copy link
Author

mchlnix commented Sep 29, 2018 •
edited
Loading

Yes

> make lint
mypy --show-traceback stubs/ Test_Client.py EntryPoint.py ExitPoint.py Mix.py Recv.py
stubs/Crypto/Hash/SHA512.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA512.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA512.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA512.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA384.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA384.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA384.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA384.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA256.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA256.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA256.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA256.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA224.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA224.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA224.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA224.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA512.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA512.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA512.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA512.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA384.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA384.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA384.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA384.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA256.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA256.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA256.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA256.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA224.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA224.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA224.py:21: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA224.py:23: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHAKE256.py:31: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHAKE256.py:33: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHAKE256.py:31: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHAKE256.py:33: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHAKE256.py:39: error: Module 'Crypto.Hash.keccak' has no attribute '_raw_keccak_lib'
stubs/Crypto/Hash/SHAKE128.py:31: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHAKE128.py:33: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHAKE128.py:31: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHAKE128.py:33: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHAKE128.py:39: error: Module 'Crypto.Hash.keccak' has no attribute '_raw_keccak_lib'
stubs/Crypto/Hash/SHA3_512.pyi:3: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA3_512.pyi:3: note: (Stub files are from https://github.com/python/typeshed)
stubs/Crypto/Hash/SHA3_512.pyi:5: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA3_512.pyi:3: error: No library stub file for module 'Crypto.Util.py3compat'
stubs/Crypto/Hash/SHA3_512.pyi:5: error: No library stub file for module 'Crypto.Util._raw_api'
stubs/Crypto/Hash/SHA3_512.pyi:11: error: Module 'Crypto.Hash.keccak' has no attribute '_raw_keccak_lib'
stubs/Crypto/Hash/SHAKE256.py:39: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.640+dev.b37d08b5d48bcbb8a2461d36541099f4b135a08b
Traceback (most recent call last):
  File "/home/michael/.local/bin/mypy", line 11, in <module>
    load_entry_point('mypy==0.640+dev.b37d08b5d48bcbb8a2461d36541099f4b135a08b', 'console_scripts', 'mypy')()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/main.py", line 93, in main
    res = build.build(sources, options, None, flush_errors, fscache)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 155, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 204, in _build
    graph = dispatch(sources, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2160, in dispatch
    process_graph(graph, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2457, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 2574, in process_stale_scc
    graph[id].semantic_analysis()
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/build.py", line 1700, in semantic_analysis
    self.manager.semantic_analyzer.visit_file(self.tree, self.xpath, self.options, patches)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 302, in visit_file
    self.accept(d)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 3607, in accept
    node.accept(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/nodes.py", line 329, in accept
    return visitor.visit_import_from(self)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 1498, in visit_import_from
    self.fail(message, imp)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/semanal.py", line 3587, in fail
    self.errors.report(ctx.get_line(), ctx.get_column(), msg, blocker=blocker)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 261, in report
    self.add_error_info(info)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 282, in add_error_info
    self._add_error_info(file, info)
  File "/home/michael/.local/lib/python3.6/site-packages/mypy/errors.py", line 264, in _add_error_info
    assert file not in self.flushed_files
AssertionError: 
stubs/Crypto/Hash/SHAKE256.py:39: : note: use --pdb to drop into pdb
Makefile:15: recipe for target 'lint' failed
make: *** [lint] Error 1

I don't think the errors mypy throws are relevant to the crash, but who knows. They appear, because I copied the .py files of the module and rename/reduce/type them into .pyi files bit by bit.

PS: I had to put the __init__.pyi file back into the stubs directory to reproduce this crash. So that is probably a key piece of the puzzle.

@ilevkivskyi
Copy link
Member

After some thinking, it looks like this is essentially a duplicate of #4881

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash priority-1-normal
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mchlnix @emmatyping @ilevkivskyi