Skip to content

gh-115122: Add --bisect option to regrtest #115123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Feb 18, 2024
Merged

gh-115122: Add --bisect option to regrtest #115123

merged 2 commits into from
Feb 18, 2024

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Feb 7, 2024 •
edited
Loading

  • test.bisect_cmd now exit with code 0 on success, and code 1 on failure. Before, it was the opposite.
  • Add create_python_cmd() and bisect_cmd() methods to RunTests.

@vstinner vstinner requested a review from rhettinger as a code owner February 7, 2024 05:05
@bedevere-app bedevere-app bot mentioned this pull request Feb 7, 2024
Closed
@vstinner
Copy link
Member Author

vstinner commented Feb 7, 2024

!buildbot Windows11 Bigmem

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 0781d23 🤖

The command will test the builders whose names match following regular expression: Windows11 Bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR

@rhettinger rhettinger removed their request for review February 8, 2024 02:00
@vstinner vstinner force-pushed the regrtest_bisect branch 4 times, most recently from 7cbb66d to e87af95 Compare February 18, 2024 13:25
@vstinner
pythongh-115122: Add --bisect option to regrtest
156297d 
* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.
@vstinner vstinner enabled auto-merge (squash) February 18, 2024 19:48
@vstinner vstinner merged commit 1e5719a into python:main Feb 18, 2024
@vstinner vstinner deleted the regrtest_bisect branch February 18, 2024 20:06
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull request Mar 4, 2024
@vstinner @woodruffw
pythongh-115122: Add --bisect option to regrtest (python#115123)
3cbf4d8 
* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.
vstinner added a commit to vstinner/cpython that referenced this pull request Mar 26, 2024
@vstinner
pythongh-115122: Add --bisect option to regrtest (python#115123)
a3c5f91 
* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.

(cherry picked from commit 1e5719a)
vstinner added a commit to vstinner/cpython that referenced this pull request Mar 26, 2024
@vstinner
pythongh-115122: Add --bisect option to regrtest (python#115123)
efc1ea7 
* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.

(cherry picked from commit 1e5719a)
@vstinner vstinner mentioned this pull request Mar 26, 2024
Merged
vstinner added a commit that referenced this pull request Mar 26, 2024
@vstinner @encukou
[3.12] gh-83434: Sync libregrtest and test_regrtest with the main bra…
477ef90 
…nch (#117250)

* gh-115122: Add --bisect option to regrtest (#115123)

* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.

(cherry picked from commit 1e5719a)

* gh-115720: Show number of leaks in huntrleaks progress reports (GH-115726)

Instead of showing a dot for each iteration, show:
- '.' for zero (on negative) leaks
- number of leaks for 1-9
- 'X' if there are more leaks

This allows more rapid iteration: when bisecting, I don't need
to wait for the final report to see if the test still leaks.

Also, show the full result if there are any non-zero entries.
This shows negative entries, for the unfortunate cases where
a reference is created and cleaned up in different runs.

Test *failure* is still determined by the existing heuristic.

(cherry picked from commit af5f9d6)

* gh-83434: Disable XML in regrtest when -R option is used (#117232)

(cherry picked from commit d52bdfb)

---------

Co-authored-by: Petr Viktorin <[email protected]>
@miss-islington miss-islington mentioned this pull request Mar 26, 2024
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 26, 2024
@vstinner @encukou @miss-islington
[3.12] pythongh-83434: Sync libregrtest and test_regrtest with the ma…
23a5deb 
…in branch (pythonGH-117250)

* pythongh-115122: Add --bisect option to regrtest (pythonGH-115123)

* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.

(cherry picked from commit 1e5719a)

* pythongh-115720: Show number of leaks in huntrleaks progress reports (pythonGH-115726)

Instead of showing a dot for each iteration, show:
- '.' for zero (on negative) leaks
- number of leaks for 1-9
- 'X' if there are more leaks

This allows more rapid iteration: when bisecting, I don't need
to wait for the final report to see if the test still leaks.

Also, show the full result if there are any non-zero entries.
This shows negative entries, for the unfortunate cases where
a reference is created and cleaned up in different runs.

Test *failure* is still determined by the existing heuristic.

(cherry picked from commit af5f9d6)

* pythongh-83434: Disable XML in regrtest when -R option is used (pythonGH-117232)

(cherry picked from commit d52bdfb)

---------

(cherry picked from commit 477ef90)

Co-authored-by: Victor Stinner <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
vstinner added a commit that referenced this pull request Mar 26, 2024
@miss-islington @vstinner @encukou
[3.11] [3.12] gh-83434: Sync libregrtest and test_regrtest with the m…
04b26ee 
…ain branch (GH-117250) (#117251)

[3.12] gh-83434: Sync libregrtest and test_regrtest with the main branch (GH-117250)

* gh-115122: Add --bisect option to regrtest (GH-115123)

* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.

(cherry picked from commit 1e5719a)

* gh-115720: Show number of leaks in huntrleaks progress reports (GH-115726)

Instead of showing a dot for each iteration, show:
- '.' for zero (on negative) leaks
- number of leaks for 1-9
- 'X' if there are more leaks

This allows more rapid iteration: when bisecting, I don't need
to wait for the final report to see if the test still leaks.

Also, show the full result if there are any non-zero entries.
This shows negative entries, for the unfortunate cases where
a reference is created and cleaned up in different runs.

Test *failure* is still determined by the existing heuristic.

(cherry picked from commit af5f9d6)

* gh-83434: Disable XML in regrtest when -R option is used (GH-117232)

(cherry picked from commit d52bdfb)

---------

(cherry picked from commit 477ef90)

Co-authored-by: Victor Stinner <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@vstinner @diegorusso
pythongh-115122: Add --bisect option to regrtest (python#115123)
48ba5db 
* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.
LukasWoodtli pushed a commit to LukasWoodtli/cpython that referenced this pull request Jan 22, 2025
@vstinner @LukasWoodtli
pythongh-115122: Add --bisect option to regrtest (python#115123)
b9adb79 
* test.bisect_cmd now exit with code 0 on success, and code 1 on
  failure. Before, it was the opposite.
* test.bisect_cmd now runs the test worker process with
  -X faulthandler.
* regrtest RunTests: Add create_python_cmd() and bisect_cmd()
  methods.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vstinner @bedevere-bot