False positive syntax-error for badly placed type annotations

[ghost]

Steps to reproduce

1. Install "pyinputplus" with pip3
2. Make a file with content of "import pyinputplus"
3. Run code in vsCode, and look at the "problems" and/or output section to see output from PyLint

Current behavior

I posted a similar issue in the repo of the python extension for vsCode (Made by microsoft), but they said it was an issue with PyLint. The current behaviour is this:

When I run my code, the "problem" output looks like this:

Cannot import 'pyinputplus' due to syntax error 'invalid syntax (<unknown>, line 268)'

The output looks like this:

##########Linting Output - pylint##########
************* Module error
1,0,error,syntax-error:Cannot import 'pyinputplus' due to syntax error 'invalid syntax (<unknown>, line 268)'

-------------------------------------
Your code has been rated at -40.00/10

Expected behavior

Expected behaviour is a code being able to run, with no errors shown.

pylint --version output

2.5.2

What I tried

I tried reload VSCode, reinstall it and I tried reinstalling the module. I checked the path and python versions, and everything was good. I'm not 100% sure if this is a bug, but I can't get it to work properly, so I suppose it is a bug.

[dbaty]

Here is a more minimal example code to reproduce the error with pylint 2.6.0 and astroid 2.4.2:

def func(
    # type: () -> int
):
    return 2

If it is a bug, it is probably in astroid, as we end up here in pylint: https://github.com/PyCQA/pylint/blob/8112aae184f73bb780c7918821f0cac7fef7c74a/pylint/lint/pylinter.py#L1033-L1040

However, I think that the type comment is misplaced. It should not appear within the signature itself (i.e. inside the parentheses), but rather in the function body, like this:

def func():
    # type: () -> int
    return 2

In that case, pylint does not raise a syntax error. I am not familiar with type comments, so I cannot say for sure. But all examples I could find (in 5 minutes) use that latter syntax. So does the cheat sheet in mypy documentation, which is probably a good sign that it is indeed the right way to write type comments. In which case the bug should be reported to pyinputplus itself. ;)

[AWhetter]

I can still reproduce this on master.

Raising a syntax error is a bit extreme so we should fix that, but @dbaty is correct that the type comment is in the wrong place and it should be below the function signature.

This looks like an astroid bug (possibly even a Python one) rather than a pylint one. Here's the traceback I get when running this code through astroid:

>>> import astroid
>>> astroid.parse("def func(\n    # type: () -> int\n):\n    return 2")
Traceback (most recent call last):
  File "/home/ashley/workspace/astroid/astroid/builder.py", line 168, in _data_build
    node, parser_module = _parse_string(data, type_comments=True)
  File "/home/ashley/workspace/astroid/astroid/builder.py", line 445, in _parse_string
    parsed = parser_module.parse(data + "\n", type_comments=type_comments)
  File "/home/ashley/workspace/astroid/astroid/_ast.py", line 48, in parse
    return parse_func(string)
  File "/usr/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 2
    # type: () -> int
            ^
SyntaxError: invalid syntax

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ashley/workspace/astroid/astroid/builder.py", line 279, in parse
    return builder.string_build(code, modname=module_name, path=path)
  File "/home/ashley/workspace/astroid/astroid/builder.py", line 142, in string_build
    module = self._data_build(data, modname, path)
  File "/home/ashley/workspace/astroid/astroid/builder.py", line 170, in _data_build
    raise exceptions.AstroidSyntaxError(
astroid.exceptions.AstroidSyntaxError: Parsing Python code failed:
invalid syntax (<unknown>, line 2)

[dbaty]

Another issue in astroid (or maybe the same issue) causes #3556.

[rioj7]

I have reported an issue at pyinputplus that is the cause of this problem.

The error message says syntax error.
It is not a syntax error because the python interpreter has no problem with it.
The python interpreter compiles the code with type_comments=False and treats any type annotation comment as a regular comment.

Better to tell:

Type annotation comment placed incorrectly

[rioj7]

After reading AWhetter comment I found that astroid has some code that deals with misplaced type annotations causing syntax errors in the compiler but that Python3.8 uses a different message ('invalid syntax') in the exception.

I have added a test in _parse_string() to deal with this.

It would be great to log the line from the exception as a Misplaced type annotation.

astroid/builder.py

def _parse_string(data, type_comments=True):
    def parse(data, type_comments):
        parser_module = get_parser_module(type_comments=type_comments)
        parsed = parser_module.parse(data + "\n", type_comments=type_comments)
        return parsed, parser_module
    try:
        parsed, parser_module = parse(data, type_comments)
    except SyntaxError as exc:
        # If the type annotations are misplaced for some reason, we do not want
        # to fail the entire parsing of the file, so we need to retry the parsing without
        # type comment support.
        def isTypeAnnotationError(msg):
            return msg == MISPLACED_TYPE_ANNOTATION_ERROR or msg == 'invalid syntax'
        if not (type_comments and isTypeAnnotationError(exc.args[0])):
            raise
        if getattr(exc, 'text', '').strip().startswith('# type:'):
            # TODO log this line as a misplaced type annotation
            pass

        parsed, parser_module = parse(data, type_comments=False)
    return parsed, parser_module

[Pierre-Sassoulas]

I can reproduce this. Pycharm also think it's a syntax error. Classyfying as minor because proper typing is available since a long time and I don't think it's a big issue currently.

[DanielNoord]

Before fixing this we should investigate the effect of calling ast.parse twice. The only way we can effectively narrow down whether this issue is due to incorrect type comments is by also running ast.parse(code, type_comments=False). This can be costly for every reported SyntaxError compared to the number of those errors caused by incorrect type comments/

It is indeed a duplicate of #3556.

The decision needed here is: do we want to retry parsing without type comments whenever ast.parse returns a SyntaxError. That discussion should probably be done in astroid.