GH Actions: Create daily integration branch

[mkoeppe]

We create a GH Actions workflow, run daily (intended for running on sagetrac-mirror) as a cron job (
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule).

It can also be triggered manually via workflow_dispatch.

Example run at
https://github.com/mkoeppe/sage/runs/5060070979

1. Merge mergeable Trac tickets from the current milestone, ordered by priority/deps, using git releasemgr merge-all --milestone=current

2. Run using tox/docker on a stable platform (ubuntu-focal-recommended). (The new package factor recommended is like standard, but with texlive etc. added, as needed for doc-pdf). We set SAGE_DOCTEST_RANDOM_SEED=0 to reduce random failures.
   Only testsuite failures that do not already fail in the baseline are considered failures.

3. Docker images are pushed to ghcr.io, see https://github.com/mkoeppe?tab=packages&q=recommended

Related issues and PRs:

• Run patchbot on all tickets, including those with package changes sage-patchbot#148
• Compile the pdf documentation and report on error sage-patchbot#64
• setup.cfg: New, add extras_require 'releasemgr' git-trac-command#53
• git releasemgr {todo,merge-all}: Add option --milestone git-trac-command#54
• ReleaseApplication._commit: Python 3 fix git-trac-command#55
• ReleaseApplication.merge: Actually handle merge conflicts git-trac-command#56
• git releasemgr merge-all: Put tickets with merge conflict to needs_work WITH useful info git-trac-command#57
• /ticket/<int:ticket>/status: New route sage-patchbot#150
• Do not build branches that only have CI changes sage-patchbot#153
• tox.ini: Update git-trac-command#59
• ReleaseApplication._current_milestone: Search for first release tag git-trac-command#60
• git releasemgr: Add option --trac-context git-trac-command#61
• git releasemgr merge: Handle errors, print summary like merge-all git-trac-command#62
• git releasemgr {todo,merge-all}: Add options --patchbot-status and --status git-trac-command#58

Refinements:

• Provide it with Trac credentials so it can set tickets back to needs_work on merge failures (using git releasemgr merge-all: Put tickets with merge conflict to needs_work WITH useful info git-trac-command#57)

  --> Trac account requested

• Run git log --first-parent on files with merge conflict

• Provide it with GitHub credentials so it can push a branch $RELEASE_TAG+integration-$DATE, if it passes build + doctests + doc-pdf.

• Perhaps git bisect --first-parent for passing; set first failing ticket back to needs_work

• Another workflow for integration testing of all tickets in the current milestone with --patchbot-status=Spkg and --status=needs_review positive_review with the full portability CI.

Depends on #33233
Depends on #31529
Depends on #33103
Depends on #33277

CC: @vbraun @dimpase

Component: distribution

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/gh_actions__create_daily_integration_branch @ 37c76a6

Issue created by migration from https://trac.sagemath.org/ticket/33222

[mkoeppe]

comment:1

Volker, would you be willing to share your existing scripts that probably already implement something like step 2 above? It does not have to be pretty.

[vbraun]

comment:2

Scripts are part of https://github.com/sagemath/git-trac-command:

$ git releasemgr merge-all -h
usage: git-releasemgr merge-all [-h] [--limit LIMIT]

optional arguments:
  -h, --help     show this help message and exit
  --limit LIMIT  Merge this many tickets

[mkoeppe]

comment:3

Thanks for the pointer

[mkoeppe]

Description changed:

--- 
+++ 
@@ -6,6 +6,8 @@
 2. Pick mergeable Trac tickets from the current milestone, order by priority/deps
 
 3.  Merge one by one and push to branch "$RELEASE_TAG+integration-$DATE", if it passes doctests + doc-pdf.
-   - Report failures as comment / status change on the Trac ticket.
+   - Report failures as comment / status change on the Trac ticket. Link to the GH Actions run: `$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID` (from https://docs.github.com/en/actions/learn-github-actions/environment-variables)
 
-
+Related PRs:
+- https://github.com/sagemath/git-trac-command/pull/53
+- https://github.com/sagemath/git-trac-command/pull/54

[mkoeppe]

Description changed:

--- 
+++ 
@@ -11,3 +11,7 @@
 Related PRs:
 - https://github.com/sagemath/git-trac-command/pull/53
 - https://github.com/sagemath/git-trac-command/pull/54
+- https://github.com/sagemath/git-trac-command/pull/55
+- https://github.com/sagemath/git-trac-command/pull/56
+- https://github.com/sagemath/git-trac-command/pull/57
+

[mkoeppe]

Description changed:

--- 
+++ 
@@ -3,7 +3,7 @@
 
 1. Start from current release tag, use the corresponding Docker image on gchr.io from a stable platform such as `ubuntu-focal-standard` 
 
-2. Pick mergeable Trac tickets from the current milestone, order by priority/deps
+2. Merge mergeable Trac tickets from the current milestone, ordered by priority/deps, using `git releasemgr merge-all --milestone=MILESTONE`
 
 3.  Merge one by one and push to branch "$RELEASE_TAG+integration-$DATE", if it passes doctests + doc-pdf.
    - Report failures as comment / status change on the Trac ticket. Link to the GH Actions run: `$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID` (from https://docs.github.com/en/actions/learn-github-actions/environment-variables)

[mkoeppe]

Dependencies: #30933

[mkoeppe]

Description changed:

--- 
+++ 
@@ -14,4 +14,5 @@
 - https://github.com/sagemath/git-trac-command/pull/55
 - https://github.com/sagemath/git-trac-command/pull/56
 - https://github.com/sagemath/git-trac-command/pull/57
+- https://github.com/sagemath/sage-patchbot/pull/150
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -8,7 +8,9 @@
 3.  Merge one by one and push to branch "$RELEASE_TAG+integration-$DATE", if it passes doctests + doc-pdf.
    - Report failures as comment / status change on the Trac ticket. Link to the GH Actions run: `$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID` (from https://docs.github.com/en/actions/learn-github-actions/environment-variables)
 
-Related PRs:
+Related issues and PRs:
+- https://github.com/sagemath/sage-patchbot/issues/148
+- https://github.com/sagemath/sage-patchbot/issues/64
 - https://github.com/sagemath/git-trac-command/pull/53
 - https://github.com/sagemath/git-trac-command/pull/54
 - https://github.com/sagemath/git-trac-command/pull/55

[mkoeppe]

Description changed:

--- 
+++ 
@@ -17,4 +17,5 @@
 - https://github.com/sagemath/git-trac-command/pull/56
 - https://github.com/sagemath/git-trac-command/pull/57
 - https://github.com/sagemath/sage-patchbot/pull/150
+- https://github.com/sagemath/git-trac-command/pull/58
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -18,4 +18,5 @@
 - https://github.com/sagemath/git-trac-command/pull/57
 - https://github.com/sagemath/sage-patchbot/pull/150
 - https://github.com/sagemath/git-trac-command/pull/58
+- https://github.com/sagemath/sage-patchbot/pull/153
 

[mkoeppe]

Branch: u/mkoeppe/gh_actions__create_daily_integration_branch

[sagetrac-git]

Commit: e803539

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

e803539

.github/workflows/integration.yml: New

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,12 +1,14 @@
-We create a GH Actions workflow, run daily on `sagetrac-mirror` (
-https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule):
+We create a GH Actions workflow, run daily (intended for running on `sagetrac-mirror`) as a cron job (
+https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule).
 
-1. Start from current release tag, use the corresponding Docker image on gchr.io from a stable platform such as `ubuntu-focal-standard` 
+It can also be triggered manually via `workflow_dispatch`.
 
-2. Merge mergeable Trac tickets from the current milestone, ordered by priority/deps, using `git releasemgr merge-all --milestone=MILESTONE`
+Example run at 
+https://github.com/mkoeppe/sage/runs/4940850295
 
-3.  Merge one by one and push to branch "$RELEASE_TAG+integration-$DATE", if it passes doctests + doc-pdf.
-   - Report failures as comment / status change on the Trac ticket. Link to the GH Actions run: `$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID` (from https://docs.github.com/en/actions/learn-github-actions/environment-variables)
+1. Merge mergeable Trac tickets from the current milestone, ordered by priority/deps, using `git releasemgr merge-all --milestone=current`
+
+2. Run using tox/docker on a stable platform (`ubuntu-focal-standard`)
 
 Related issues and PRs:
 - https://github.com/sagemath/sage-patchbot/issues/148
@@ -20,3 +22,15 @@
 - https://github.com/sagemath/git-trac-command/pull/58
 - https://github.com/sagemath/sage-patchbot/pull/153
 
+Refinements:
+- Provide it with Trac credentials so it can set tickets back to `needs_work` on merge failures (using https://github.com/sagemath/git-trac-command/pull/57)
+
+- Link to the GH Actions run: `$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID` (from https://docs.github.com/en/actions/learn-github-actions/environment-variables)
+
+- Provide it with [GitHub](../wiki/GitHub) credentials so it can push a branch `$RELEASE_TAG+integration-$DATE`, if it passes build + doctests + doc-pdf.
+
+- Perhaps `git bisect --first-parent` for passing; set first failing ticket back to `needs_review`
+
+- Warmstart from a Docker image on gchr.io (#30933) 
+
+