test-with-buildbots is not triggering buildbots

[tiran]

On PR python/cpython#25553 the test-with-buildbots label doesn't trigger buildbot testing properly. In the first attempt no buildbot was triggered. I tried again 15 minutes later. The second time only buildbot/s390x Debian PR got started.

CC: @pablogsal @vstinner

[pablogsal]

On PR python/cpython#25553 the test-with-buildbots label doesn't trigger buildbot testing properly. In the first attempt no buildbot was triggered. I tried again 15 minutes later. The second time only buildbot/s390x Debian PR got started.

That's because the builbots are busy. They only start reporting when they receive your request. You can see the queried requests here:

https://buildbot.python.org/all/#/pendingbuildrequests

[tiran]

Thanks @pablogsal, I didn't know that. Is there a simple way to improve reporting? Or should I create a PR to include the link into the Bedevere reply?

[pablogsal]

For example, for your PR python/cpython#25553 :

this buildbot has these builds:

https://buildbot.python.org/all/#/builders/500

and here is your pending request:

https://buildbot.python.org/all/#/buildrequests/199658

[pablogsal]

Thanks @pablogsal, I didn't know that. Is there a simple way to improve reporting? Or should I create a PR to include the link into the Bedevere reply?

We should investigate if is possible to report a "pending" check as soon as they trigger

[pablogsal]

In the buildbot docs, seems that the only generators that they have are:

http://docs.buildbot.net/latest/manual/configuration/report_generators/index.html

and the one we use only reports starts/ends:

http://docs.buildbot.net/latest/manual/configuration/report_generators/build_start_end.html#reportgen-buildstartendstatusgenerator

We could either improve the error message or/and try to make our custom generator to report as soon as we receive the request.

[pablogsal]

Or should I create a PR to include the link into the Bedevere reply?

Do you mind making a PR to include the link? The code is here:

https://github.com/python/buildmaster-config/blob/f0b45256606d6173924c5b35e718944eaf0e2795/master/custom/pr_testing.py#L16

[tiran]

I'm bit busy right now. Can you assign this ticket to me, please? I'll take a look later or early next week. I don't have permission to self-assign on this tracker.

[pablogsal]

I don't have permission to self-assign on this tracker.

Me neither (have permissions over the tracker), but don't worry, given that everyone is using the label these two weeks quite a lot I will try to do it today so there is no more confusion regarding this.

Thanks for raising this @tiran

[pablogsal]

I think this may improve the situation: python/buildmaster-config#242

[tiran]

I think this may improve the situation: python/buildmaster-config#242

Fantastic!

I noticed that cancelled jobs are marked as failed.

[pablogsal]

Yeah, I scheduled a new run for your PR

[pablogsal]

You need to still manually inspect failures because there are still known problems :( :

https://buildbot.python.org/all/#/release_status

[vstinner]

This psf-infra-meta is a broad project, I suggest to use the more specific list https://mail.python.org/mailman3/lists/python-buildbots.python.org/ (or https://github.com/python/core-workflow/issues ) to report buildbot issues.