gh-137242: Add a --no-randomize option, and use it in Android CI #138649

pythonGH-138649) Adds a --no-randomize option to the CI runner, so that randomisation can be easily disabled for --fast-ci and --slow-ci configurations on single-threaded testing platforms like Android, iOS, and Emscripten. --------- (cherry picked from commit 01895d233b4370e0c99f16c01a57cfd73c434b28) Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Malcolm Smith <[email protected]>

miss-islington-app · 2025-09-08T22:45:52Z

Sorry, @vstinner and @freakboy3742, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 01895d233b4370e0c99f16c01a57cfd73c434b28 3.13

bedevere-app · 2025-09-08T22:45:57Z

GH-138684 is a backport of this pull request to the 3.14 branch.

…droid CI (pythonGH-138649) Adds a --no-randomize option to the CI runner, so that randomisation can be easily disabled for --fast-ci and --slow-ci configurations on single-threaded testing platforms like Android, iOS, and Emscripten. --------- (cherry picked from commit 01895d2) Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Malcolm Smith <[email protected]>

bedevere-app · 2025-09-08T22:53:24Z

GH-138685 is a backport of this pull request to the 3.13 branch.

…CI (GH-138649) (#138684) Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Malcolm Smith <[email protected]>

…CI (GH-138649) (#138685) Adds a --no-randomize option to the CI runner, so that randomisation can be easily disabled for --fast-ci and --slow-ci configurations on single-threaded testing platforms like Android, iOS, and Emscripten. --------- (cherry picked from commit 01895d2) Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Malcolm Smith <[email protected]>

bedevere-bot · 2025-09-09T09:15:49Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.13 (tier-3) has failed when building commit 85649b7.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1590/builds/972) and take a look at the build logs.
Check if the failure is related to this commit (85649b7) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1590/builds/972

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 9, done.        
remote: Counting objects:  20% (1/5)        
remote: Counting objects:  40% (2/5)        
remote: Counting objects:  60% (3/5)        
remote: Counting objects:  80% (4/5)        
remote: Counting objects: 100% (5/5)        
remote: Counting objects: 100% (5/5), done.        
remote: Compressing objects:  20% (1/5)        
remote: Compressing objects:  40% (2/5)        
remote: Compressing objects:  60% (3/5)        
remote: Compressing objects:  80% (4/5)        
remote: Compressing objects: 100% (5/5)        
remote: Compressing objects: 100% (5/5), done.        
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 4 (from 1)        
From https://github.com/python/cpython
 * branch                    3.13       -> FETCH_HEAD
Note: switching to '85649b769f0db4992c061c8d17155c07553dda46'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 85649b769f0 [3.13] gh-137242: Add a --no-randomize option, and use it in Android CI (GH-138649) (#138685)
Switched to and reset branch '3.13'

configure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  192k  100  192k    0     0  2196k      0 --:--:-- --:--:-- --:--:-- 2196k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 42455  100 42455    0     0   468k      0 --:--:-- --:--:-- --:--:--  468k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 5041k  100 5041k    0     0  26.0M      0 --:--:-- --:--:-- --:--:-- 26.0M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 1257k  100 1257k    0     0  10.6M      0 --:--:-- --:--:-- --:--:-- 10.6M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  635k  100  635k    0     0  4814k      0 --:--:-- --:--:-- --:--:-- 4814k
../../configure: line 4013: pkg-config: command not found
configure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

../../Python/fileutils.c:455:1: warning: unused function 'decode_current_locale' [-Wunused-function]
  455 | decode_current_locale(const char* arg, wchar_t **wstr, size_t *wlen,
      | ^~~~~~~~~~~~~~~~~~~~~
../../Python/fileutils.c:674:1: warning: unused function 'encode_current_locale' [-Wunused-function]
  674 | encode_current_locale(const wchar_t *text, char **str,
      | ^~~~~~~~~~~~~~~~~~~~~
2 warnings generated.
../../Modules/_testexternalinspection.c:538:5: warning: code will never be executed [-Wunreachable-code]
  538 |     int pid;
      |     ^~~~~~~~
1 warning generated.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  8784  100  8784    0     0   242k      0 --:--:-- --:--:-- --:--:--  245k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  2894  100  2894    0     0  73375      0 --:--:-- --:--:-- --:--:-- 74205
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 43504  100 43504    0     0  1133k      0 --:--:-- --:--:-- --:--:-- 1148k

vstinner · 2025-09-09T09:19:02Z

@freakboy3742: I'm not sure what's going on, aarch64 Android 3.13 buildbot failed with:

C/C++: ld.lld: error: /Users/android/buildarea/3.13.mhsmith-android-aarch64/build/cross-build/aarch64-linux-android/prefix/lib/libpython3.13.so: unknown file type

mhsmith · 2025-09-09T09:53:30Z

This happened before in #137186 (comment), but I wasn't sure of the cause. I'll look into it.

mhsmith · 2025-09-09T14:08:43Z

@freakboy3742: Like the last time this problem occurred, the buildbot server had about 12 GB of free space. The biggest offender was /Users/buildbot/Library/Logs/CoreSimulator:

-rw-r--r--   1 buildbot  staff  30042226464 Jun 17 13:32 CoreSimulator.prev.log
-rw-r--r--   1 buildbot  staff  14734765773 Sep  9 13:20 CoreSimulator.log
-rw-r--r--   1 buildbot  staff      1100845 Feb 10  2025 Simulator.log

I deleted CoreSimulator.prev.log (30 GB), but I didn't restart the Android worker as I did last time, so this should be a better test of whether disk space is indeed the cause.

python#138649) Adds a --no-randomize option to the CI runner, so that randomisation can be easily disabled for --fast-ci and --slow-ci configurations on single-threaded testing platforms like Android, iOS, and Emscripten. --------- Co-authored-by: Malcolm Smith <[email protected]>

freakboy3742 · 2025-09-09T22:18:41Z

I deleted CoreSimulator.prev.log (30 GB), but I didn't restart the Android worker as I did last time, so this should be a better test of whether disk space is indeed the cause.

Sounds like a cron job to rotate/clean up those logs (or better still, direct them to the build directory) might be called for. I'll investigate what can be done.

mhsmith and others added 4 commits August 31, 2025 18:56

Change CI arguments: remove --randomize, add --fail-rerun

e96c92a

Remove outdated comment

f69095c

Revert changes to default args; add a --no-randomize option instead

b64cace

Don't use None for randomize

754a8c4

bedevere-app bot mentioned this pull request Sep 8, 2025

Android CI and binary releases #137242

Closed

vstinner mentioned this pull request Sep 8, 2025

gh-137242: Add a --no-randomize option, and use it in Android CI #138303

Closed

mhsmith added the skip news label Sep 8, 2025

mhsmith approved these changes Sep 8, 2025

View reviewed changes

bedevere-app bot added the awaiting core review label Sep 8, 2025

mhsmith marked this pull request as ready for review September 8, 2025 12:48

mhsmith requested a review from freakboy3742 as a code owner September 8, 2025 12:48

mhsmith added needs backport to 3.13 bugs and security fixes infra CI, GitHub Actions, buildbots, Dependabot, etc. needs backport to 3.14 bugs and security fixes labels Sep 8, 2025

mhsmith removed the request for review from freakboy3742 September 8, 2025 12:52

freakboy3742 approved these changes Sep 8, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Sep 8, 2025

freakboy3742 merged commit 01895d2 into python:main Sep 8, 2025
67 checks passed

bedevere-app bot removed the awaiting merge label Sep 8, 2025

miss-islington-app bot assigned freakboy3742 Sep 8, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Sep 8, 2025

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Sep 8, 2025

hugovk pushed a commit that referenced this pull request Sep 9, 2025

[3.14] gh-137242: Add a --no-randomize option, and use it in Android …

6f66590

…CI (GH-138649) (#138684) Co-authored-by: Victor Stinner <[email protected]> Co-authored-by: Malcolm Smith <[email protected]>

vstinner deleted the regrtest_no_randomize branch September 9, 2025 08:53

mhsmith mentioned this pull request Sep 9, 2025

Use --slow-ci on Android python/buildmaster-config#628

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gh-137242: Add a --no-randomize option, and use it in Android CI #138649

gh-137242: Add a --no-randomize option, and use it in Android CI #138649

vstinner commented Sep 8, 2025 •

edited by mhsmith

Loading

Uh oh!

vstinner commented Sep 8, 2025

Uh oh!

mhsmith left a comment

Uh oh!

vstinner commented Sep 8, 2025

Uh oh!

bedevere-bot commented Sep 8, 2025

Uh oh!

freakboy3742 left a comment

Uh oh!

Uh oh!

miss-islington-app bot commented Sep 8, 2025

Uh oh!

miss-islington-app bot commented Sep 8, 2025

Uh oh!

bedevere-app bot commented Sep 8, 2025

Uh oh!

bedevere-app bot commented Sep 8, 2025

Uh oh!

bedevere-bot commented Sep 9, 2025

Uh oh!

vstinner commented Sep 9, 2025

Uh oh!

mhsmith commented Sep 9, 2025

Uh oh!

mhsmith commented Sep 9, 2025 •

edited

Loading

Uh oh!

freakboy3742 commented Sep 9, 2025

Uh oh!

Uh oh!

Uh oh!

gh-137242: Add a --no-randomize option, and use it in Android CI #138649

gh-137242: Add a --no-randomize option, and use it in Android CI #138649

Conversation

vstinner commented Sep 8, 2025 • edited by mhsmith Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Sep 8, 2025

Uh oh!

mhsmith left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner commented Sep 8, 2025

Uh oh!

bedevere-bot commented Sep 8, 2025

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Sep 8, 2025

Uh oh!

miss-islington-app bot commented Sep 8, 2025

Uh oh!

bedevere-app bot commented Sep 8, 2025

Uh oh!

bedevere-app bot commented Sep 8, 2025

Uh oh!

bedevere-bot commented Sep 9, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

vstinner commented Sep 9, 2025

Uh oh!

mhsmith commented Sep 9, 2025

Uh oh!

mhsmith commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

freakboy3742 commented Sep 9, 2025

Uh oh!

Uh oh!

vstinner commented Sep 8, 2025 •

edited by mhsmith

Loading

mhsmith commented Sep 9, 2025 •

edited

Loading