Report failed if \u001d presented in the sources

[romantolkachyov]

Describe the bug

Failed to build report if some of the source files contains \u001d unicode character (GS):

Couldn't parse '/opt/app/tests/module_tests.py' as Python source: 'EOF in multi-line statement' at line 3306

And there is no such a line in the file (there is less than 3100 lines at all)

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

1. What version of Python are you using?
   Python 3.9.16
2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.
   6.5, 7.0
3. What versions of what packages do you have installed? The output of pip freeze is helpful.
   Private repo, can't do so without modifications so useless
4. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

example = "�"

1. What commands did you run?

python -m coverage run python manage.py test
python -m coverage html

Expected behavior
Report should be generated.

Additional context
Everything else works correctly: tests are passed and pylint check is successful

[nedbat]

I'm not seeing the same thing you are:

% hexdump -C example.py
00000000  65 78 61 6d 70 6c 65 20  3d 20 22 1d 22 0a        |example = ".".|
0000000e

% coverage run example.py

% coverage html
Wrote HTML report to htmlcov/index.html

Can you create a zip file with the files that are causing the problems for you? I might be doing something wrong.

[romantolkachyov]

@nedbat Sorry it is not enough. Just didn't test example first. This should work:

from unittest import TestCase


class ExampleTest(TestCase):
    def test_example(self):
        test = {
            "identifiers": ["�01099912345612321128033010111111111128�21420127�8"], "mode": "serialize"
        }

Don't know why but any simplification like removing mode key from the dict cause successful report build.

Then

$ coverage run test.py
$ coverage xml
Couldn't parse '/Users/roman/work/mahaon/tmp/test.py' as Python source: 'EOF in multi-line statement' at line 12
$ coverage html
Couldn't parse '/Users/roman/work/mahaon/tmp/test.py' as Python source: 'EOF in multi-line statement' at line 12

hexdump:

$ hexdump -C test.py 
00000000  66 72 6f 6d 20 75 6e 69  74 74 65 73 74 20 69 6d  |from unittest im|
00000010  70 6f 72 74 20 54 65 73  74 43 61 73 65 0a 0a 0a  |port TestCase...|
00000020  63 6c 61 73 73 20 45 78  61 6d 70 6c 65 54 65 73  |class ExampleTes|
00000030  74 28 54 65 73 74 43 61  73 65 29 3a 0a 20 20 20  |t(TestCase):.   |
00000040  20 64 65 66 20 74 65 73  74 5f 65 78 61 6d 70 6c  | def test_exampl|
00000050  65 28 73 65 6c 66 29 3a  0a 20 20 20 20 20 20 20  |e(self):.       |
00000060  20 74 65 73 74 20 3d 20  7b 0a 20 20 20 20 20 20  | test = {.      |
00000070  20 20 20 20 20 20 22 69  64 65 6e 74 69 66 69 65  |      "identifie|
00000080  72 73 22 3a 20 5b 22 1d  30 31 30 39 39 39 31 32  |rs": [".01099912|
00000090  33 34 35 36 31 32 33 32  31 31 32 38 30 33 33 30  |3456123211280330|
000000a0  31 30 31 31 31 31 31 31  31 31 31 31 32 38 1d 32  |10111111111128.2|
000000b0  31 34 32 30 31 32 37 1d  38 22 5d 2c 20 22 6d 6f  |1420127.8"], "mo|
000000c0  64 65 22 3a 20 22 73 65  72 69 61 6c 69 7a 65 22  |de": "serialize"|
000000d0  0a 20 20 20 20 20 20 20  20 7d 0a                 |.        }.|
000000db

test.py.zip

[nedbat]

Thanks, this is fixed in commit 35e249f.

Note that the HTML report doesn't display these characters well, and differently than GitHub does. That can be a new issue if it needs to be improved.

[nedbat]

This is now released as part of coverage 7.0.1.