Skip to content

Generate report (mypy.xml) fails when running mypy in parallel with another job #16630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lukin0110 opened this issue Dec 7, 2023 · 3 comments · Fixed by #16637
Closed

Generate report (mypy.xml) fails when running mypy in parallel with another job #16630

lukin0110 opened this issue Dec 7, 2023 · 3 comments · Fixed by #16637
Labels
bug mypy got something wrong

Comments

@lukin0110
Copy link
Contributor

Bug Report

In pyproject.toml I've configured mypy and coverage to generate their reports in the same directory: reports. In Gitlab CI I'm running 2 jobs in parallel to run mypy and coverage. Sometimes the mypy jobs fails.

To Reproduce

Run mypy and coverage in parallel in Gitlab CI.

Expected Behavior

Always generate the report in reports/mypy.xml.

Actual Behavior

Traceback (most recent call last):
  File "/opt/my-package-env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
             ^^^^^^^^^^^^^^^
  File "/opt/my-package-env/lib/python3.11/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "mypy/main.py", line 99, in main
  File "mypy/main.py", line 203, in run_build
  File "mypy/main.py", line 1489, in maybe_write_junit_xml
  File "mypy/util.py", line 280, in write_junit_xml
  File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/builds/my-package/reports'
Success: no issues found in 10 source files
Error: Sequence aborted after failed subtask 'lint[0]' 

Your Environment

  • Mypy version used: 1.7.0
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files):
  • Python version used: 3.11.6
[tool.mypy]
junit_xml = "reports/mypy.xml"
plugins = "pydantic.mypy"
strict = true
disallow_subclassing_any = false
disallow_untyped_decorators = false
explicit_package_bases = true
ignore_missing_imports = true
pretty = true
show_column_numbers = true
show_error_codes = true
show_error_context = true
warn_unreachable = true
mypy_path = "src"

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true
@lukin0110 lukin0110 added the bug mypy got something wrong label Dec 7, 2023
@lukin0110
Copy link
Contributor Author

lukin0110 commented Dec 7, 2023

A possible solution can be to change the following in the function write_junit_xml:

os.makedirs(xml_dirs)

to

os.makedirs(xml_dirs, exist_ok=True)

https://github.com/python/mypy/blame/master/mypy/util.py#L320

@JelleZijlstra
Copy link
Member

Your fix makes sense, feel free to submit a PR.

@lukin0110
Copy link
Contributor Author

Thanks. Will make a PR tomorrow.

JelleZijlstra pushed a commit that referenced this issue Dec 9, 2023
…6642)

This PR replaces explicit `os.path.exists(...)` checks with
`os.makedirs(..., exist_ok=True)` where possible.

This removes the need for an extra existence check and slightly
simplifies the code. This can also prevent race conditions like #16630.
kourbou pushed a commit to kourbou/mypy that referenced this issue Dec 11, 2023
kourbou pushed a commit to kourbou/mypy that referenced this issue Dec 11, 2023
…thon#16642)

This PR replaces explicit `os.path.exists(...)` checks with
`os.makedirs(..., exist_ok=True)` where possible.

This removes the need for an extra existence check and slightly
simplifies the code. This can also prevent race conditions like python#16630.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants