Translation bot

.github/workflows/README.md

@@ -0,0 +1,54 @@
+## Setting up the Bot 
+- Create a new GitHub account and set up [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token).
+- Go to repository Settings -> Security -> Secrets -> Actions -> Repository secrets
+- create new secret: `PAT` with Personal Access Token of a bot
+- in `create-pull-request.yaml` update bot related fields:
+  - `jobs.createPullRequest.env.bot_username`
+  - `jobs.createPullRequest.env.bot_email`
+
+
+## Adding new translation repository
+
+PR Bot requires related histories between original and translated repositories. To achieve common history, we can either:
+- clone [ethereum/solidity](https://github.com/ethereum/solidity/)
+- remove everything but `docs/` directory
+- remove all branches but `develop`
+- create a new repository in the [solidity-docs](https://github.com/solidity-docs) organization
+- grant bot write access to that repository
+- push from your clone to the newly created repository
+
+Another option would be maintaining an up-to-date `en-english` repository as a template - with only the `docs` directory. That repository would be ready to fork and create a translation repository. In that case, the procedure would look like this:
+- `git clone --bare [email protected]:solidity-docs/en-english.git`
+- create a new GitHub repository, e.g. `solidity-docs/pl-polish`
+- grant bot write access to that repository
+- `cd en-english && git push [email protected]:solidity-docs-test/pl-polish.git`
+
+In either case, pull request workflow file has to be updated:
+- edit `.github/workflow/create-pull-request.yaml` 
+- add language code to `jobs.createPullRequest.strategy.matrix.repos`
+
+## How to transform translation repository to compatible form
+- check for the date of first translation commit, i.e [french](https://github.com/solidity-docs/fr-french/commits/main?after=2419c07e094306460d439da8c4db9ec15363b10c+34&branch=main), first commit is 04.02.2022
+- clone solidity repository `git clone [email protected]:ethereum/solidity.git`
+- rename solidity repository as translation repository, eg `mv solidity fr-french`
+- change directory to just created repository, i.e.: `cd fr-french`
+- check out to the date of first translation commit and overwrite develop branch: ``git checkout -B develop `git rev-list -n1 --before=2022-02-04 develop``
+- remove everything but `docs/` directory
+- commit your changes: `git add . && git commit -am "prepare translation repository"`
+- clone old translation repository, add `-old` suffix, e.g. `cd .. && git clone [email protected]:solidity-docs/fr-french.git fr-french-old`
+- if a translation repository has the wrong structure (root directory instead of `docs`), temporarily move all the files to the root directory and commit the change: `mv docs/* . && git add . && git commit -am "temporarly moving documentation outside docs/ dir"`
+- go to translation repository and add a remote - old translation: `git remote add french-old ../fr-french-old/`
+- fetch:   `git fetch french-old`
+- go to old translation repository and pick first and last translation commits, i.e for [french](https://github.com/solidity-docs/fr-french/commits/main?after=2419c07e094306460d439da8c4db9ec15363b10c+34&branch=main) first commit is `42b7772a145aab0cdbf4fbc300051cbba6d721df` and last `2419c07e094306460d439da8c4db9ec15363b10c`
+- create a range: `first_sha..last_sha`
+- cherry pick commits using range: `git cherry-pick --strategy=recursive -X theirs 42b7772a145aab0cdbf4fbc300051cbba6d721df..2419c07e094306460d439da8c4db9ec15363b10c`
+- for merge commit, you need to skip commit: `git cherry-pick --skip`
+- if necessary, move back everything to the `docs` directory and create a commit
+- create new GitHub repository
+- edit `.git/config` and change origin URL, eg:
+```
+  6 [remote "origin"]
+  7         url = [email protected]:solidity-docs-test/fr-french.git
+```
+- create branch main: `git checkout -b main`
+- push branch main: `git push origin main`

.github/workflows/create-daily-docs-sync-pr.yaml

@@ -0,0 +1,70 @@
+name: daily docs sync PR
+
+on:
+  schedule:
+    # Runs "at minute 44 past midnight" (see https://crontab.guru)
+    # Reason is avoiding full hour peak
+    - cron: "44 0 * * *"
+
+env:
+  BOT_USERNAME: soldocsbot
+  BOT_EMAIL: [email protected]
+  GITHUB_REPOSITORY_OWNER: solidity-docs
+  # comma separated list of labels
+  LABELS: sync,automated-pr
+
+jobs:
+  createPullRequest:
+    runs-on: ubuntu-latest
+    strategy:
+      # In the context matrix strategy, fail-fast means, if one of the jobs
+      # fails,the rest of the jobs will be canceled. In our case, this can
+      # be a limitation. When merging or creating PR to one repository fails,
+      # the rest still can be good.
+      fail-fast: false
+      matrix:
+        # This means, that all pull requests will be processed simultaneously
+        # and independently of each other.
+        repos: ["placeholder"]
+    steps:
+      - name: Fetch translation repository
+        uses: actions/checkout@v2
+        with:
+          token: ${{ secrets.PAT }}
+          repository: ${{ env.GITHUB_REPOSITORY_OWNER }}/${{ matrix.repos }}
+          # By default, checkout is fetching only the last commit. This will
+          # cause "unrelated histories" error. "0" means unlimited fetch-depth.
+          fetch-depth: 0
+
+      - name: Fetch this repository
+        uses: actions/checkout@v2
+        with:
+          token: ${{ secrets.PAT }}
+          repository: ${{ github.repository }}
+          path: .github-workflow
+
+      - name: Prepare Pull Request
+        id: prepare-pr
+        run: |
+          .github-workflow/scripts/merge-conflicts.sh "$BOT_USERNAME" "$BOT_EMAIL"
+          .github-workflow/scripts/generate-PR-body.sh
+          .github-workflow/scripts/set-assignees.sh "${{ matrix.repos }}"
+
+      - name: Remove this repository
+        run: |
+          rm -rf .github-workflow
+
+      - name: Create Pull Request
+        if: ${{ steps.prepare-pr.outputs.branch_exists == 'false' }}
+        uses: peter-evans/create-pull-request@v3
+        with:
+          token: "${{ secrets.PAT }}"
+          commit-message: "${{ env.pr_title }}"
+          committer: "${{ env.BOT_USERNAME }} <${{ env.BOT_EMAIL }}>"
+          author: "${{ env.BOT_USERNAME }} <${{ env.BOT_EMAIL }}>"
+          branch: "${{ steps.prepare-pr.outputs.branch_name }}"
+          title: "${{ env.pr_title }}"
+          body: "${{ env.pr_body }}"
+          labels: "${{ env.LABELS }}"
+          assignees: ${{ env.assignee }}
+          reviewers: ${{ env.reviewer }}

scripts/generate-PR-body.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+set -euo pipefail
+
+function title {
+    echo "Sync with ethereum/solidity@$(git describe --tags --always english/develop) $(date -u +%Y-%m-%d)"
+}
+
+function pr_body {
+    echo "This PR was automatically generated."
+    echo
+    git status --short |
+        grep "docs" |
+        awk '{print "- [ ] ["$2"](https://github.com/ethereum/solidity/tree/develop/"$2")"}'
+    echo
+    echo "Merge changes from [solidity](https://github.com/ethereum/solidity)@develop"
+    echo "Please fix the conflicts by pushing new commits to this pull request, either by editing the files directly on GitHub or by checking out this branch."
+    echo "## DO NOT SQUASH MERGE THIS PULL REQUEST!"
+    echo "Doing so will erase the commits from main and cause them to show up as conflicts the next time we merge."
+}
+
+printf "pr_title=%s\n" "$(title)"             >> "$GITHUB_ENV"
+printf "pr_body<<EOF\n%s\nEOF\n" "$(pr_body)" >> "$GITHUB_ENV"

scripts/merge-conflicts.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+set -euo pipefail
+
+USERNAME="$1"
+EMAIL="$2"
+
+orginal_url="https://github.com/ethereum/solidity.git"
+git remote add english "$orginal_url"
+
+git config user.name "$USERNAME"
+git config user.email "$EMAIL"
+
+## Fetch from translated origin
+git fetch english --quiet
+sync_branch="sync-$(git describe --tags --always english/develop)"
+
+# pass the hash and the branch name to action "create PR"
+echo "::set-output name=branch_name::$sync_branch"
+
+# check if sync branch exists
+if (git ls-remote --exit-code --heads origin "$sync_branch") 
+then
+    branch_exists=true
+    echo "sync_branch $sync_branch exists"
+else 
+    branch_exists=false
+    echo "sync_branch $sync_branch does not exist"
+fi
+
+echo "::set-output name=branch_exists::$branch_exists"
+
+# pull from ethereum/solidity develop
+git pull english develop --rebase=false --squash || true
+
+# unstage everything
+git rm -r --cached .
+
+# stage only selected files / directories
+git add docs/*
+
+# remove untracked files
+git clean -d --force --exclude=.gitignore --exclude=README.md

scripts/set-assignees.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+set -euo pipefail
+LANGUAGE=$1
+
+# extracts de from de-german
+lang_code=${LANGUAGE:0:2}
+number_of_maintainers=$(jq '.maintainers | length' ".github-workflow/langs/$lang_code.json")
+
+
+assignee=$((($RANDOM%$number_of_maintainers)))
+reviewer=$((($RANDOM%$number_of_maintainers)))
+
+while [[ $assignee == "$reviewer" && number_of_maintainers -gt 1 ]]
+do
+    reviewer=$((($RANDOM%$number_of_maintainers)))
+done
+
+echo "reviewer=$(
+    jq --raw-output ".maintainers[$reviewer]" ".github-workflow/langs/$lang_code.json"
+)" >> $GITHUB_ENV
+
+echo "assignee=$(
+    jq --raw-output ".maintainers[$assignee]" ".github-workflow/langs/$lang_code.json"
+)" >> $GITHUB_ENV
+