Skip to content

gh-103186: Fix or catch 'extra' stderr output from unittests #103196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 10, 2023

Conversation

TabLand
Copy link
Contributor

@TabLand TabLand commented Apr 2, 2023

Reduce test noise by fixing or catching and testing stderr messages from individual tests.

test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call.

test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it.

test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it.

@bedevere-bot

This comment was marked as resolved.

@ghost
Copy link

ghost commented Apr 2, 2023

All commit authors signed the Contributor License Agreement.
CLA signed

@TabLand
Copy link
Contributor Author

TabLand commented Apr 2, 2023

Please let me know if changes to individual files should be separated, would this separation have to be done both at the commit and branch level?

@terryjreedy
Copy link
Member

The azure pipeline failure page is https://dev.azure.com/Python/cpython/_build/results?buildId=123893&view=logs&j=d554cd63-f8f4-5b2d-871b-33e4ea76e915&t=5a14d0eb-dbd4-5b80-f5d0-7909f950a1cc.

In the first run, "3 tests failed: test_asyncgen test_logging test_subprocess". In the rerun, all 3 passed, but a normal sys.exit(0) failed with a PermissionError. We have seen this before, but I thought it was fixed. I am dubious that the patch would be responsible. I may ask about this tomorrow.

These three files together look fine together. I think that changes look good; will try to do some testing tomorrow.

@TabLand
Copy link
Contributor Author

TabLand commented Apr 3, 2023

Thanks Terry!

@arhadthedev arhadthedev added the tests Tests in the Lib/test dir label Apr 6, 2023
@TabLand
Copy link
Contributor Author

TabLand commented Jul 8, 2023

bump!

@terryjreedy terryjreedy changed the title gh-103186: Make test output cleaner gh-103186: Fix or catch 'extra' stderr output from unittests Jul 10, 2023
Copy link
Member

@terryjreedy terryjreedy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that these 3 tests run cleanly on Windows.

@terryjreedy terryjreedy added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Jul 10, 2023
@terryjreedy terryjreedy merged commit 9d58225 into python:main Jul 10, 2023
@miss-islington
Copy link
Contributor

Thanks @TabLand for the PR, and @terryjreedy for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry, @TabLand and @terryjreedy, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 9d582250d8fde240b8e7299b74ba888c574f74a3 3.11

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 10, 2023
…ythonGH-103196)

Reduce test noise by fixing or catching and testing stderr messages from individual tests.

test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call.

test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it.

test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it.
---------

(cherry picked from commit 9d58225)

Co-authored-by: Ijtaba Hussain <[email protected]>
Co-authored-by: Oleg Iarygin <[email protected]>
@bedevere-bot
Copy link

GH-106605 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jul 10, 2023
terryjreedy added a commit to terryjreedy/cpython that referenced this pull request Jul 10, 2023
…ests (python#103196)

Reduce test noise by fixing or catching and testing stderr messages from individual tests.

test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call.

test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it.

test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it.
---------

Co-authored-by: Oleg Iarygin <[email protected]>
(cherry picked from commit 9d58225)
@bedevere-bot
Copy link

GH-106606 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Jul 10, 2023
terryjreedy pushed a commit that referenced this pull request Jul 10, 2023
…H-103196) (#106605)

gh-103186: Fix or catch 'extra' stderr output from unittests (GH-103196)

Reduce test noise by fixing or catching and testing stderr messages from individual tests.

test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call.

test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it.

test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it.
---------

(cherry picked from commit 9d58225)

Co-authored-by: Ijtaba Hussain <[email protected]>
Co-authored-by: Oleg Iarygin <[email protected]>
terryjreedy added a commit that referenced this pull request Jul 10, 2023
…103196) (#106606)

Reduce test noise by fixing or catching and testing stderr messages from individual tests.

test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call.

test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it.

test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it.
---------

Co-authored-by: Oleg Iarygin <[email protected]>
(cherry picked from commit 9d58225)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants