Migrate site to Cloudflare Workers and GitHub Actions #493

Copilot · 2025-09-05T13:55:44Z

This PR implements a complete migration of the unionai-docs project from Cloudflare Pages to a GitHub Actions build system with Cloudflare Pages deployment.

Migration Overview

The current setup builds and hosts the site on Cloudflare Pages. This migration moves all building to GitHub Actions while keeping Cloudflare for hosting, providing better control over the build process and enabling advanced deployment strategies.

Key Features Implemented

Multi-Version Combined Builds

When either main or v1 branch is updated, both versions are built and combined into a single deployment
Site structure includes both v2 (from main) and v1 (from v1 branch) under /docs/v2/ and /docs/v1/ respectively
Automatic redirects handle version-less URLs (defaults to v2/latest)

Advanced PR Previews

PR builds combine the changed branch with the stable version of the other branch
All PR contributors (including from forks) get preview environments
Automatic PR comments include direct links to all variants in the preview
Preview environments are automatically cleaned up when PRs are closed

Robust Build Process

Gracefully handles missing submodules and dependencies
CI-friendly Hugo builds that don't fail on warnings from missing code fragments
Comprehensive error handling and logging throughout
Build script supports both production and preview modes

Technical Implementation

The migration includes three main workflows:

Production Deployment (build-and-deploy.yml): Triggered on pushes to main or v1, builds both versions and deploys combined site
PR Preview (pr-preview.yml): Creates preview deployments with PR timeline comments
PR Cleanup (cleanup-pr.yml): Removes preview environments when PRs close

A unified build script (scripts/build-combined.sh) orchestrates the complex multi-branch building and combining logic, while a CI-friendly Hugo wrapper (scripts/run_hugo_ci.sh) ensures builds succeed even with missing external dependencies.

Site Structure

The deployed site maintains the existing URL structure while supporting both versions:

production-site/
├── index.html (redirects to /docs/byoc/)
├── _redirects (Cloudflare Pages routing rules)  
└── docs/
    ├── v2/ (from main branch - latest)
    │   ├── flyte/
    │   ├── serverless/
    │   ├── byoc/
    │   └── selfmanaged/
    └── v1/ (from v1 branch - archived)
        ├── flyte/
        ├── serverless/
        ├── byoc/
        └── selfmanaged/

Setup Requirements

The migration requires configuring two Cloudflare secrets (CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID) and creating a Cloudflare Pages project. Complete setup instructions are provided in MIGRATION.md.

Fixes #492.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

esm.ubuntu.com
- Triggering command: /usr/lib/apt/methods/https (dns block)
github.com (HTTP Only)
- Triggering command: /usr/bin/ssh -o SendEnv=GIT_PROTOCOL [email protected] git-upload-pack 'unionai/unionai-examples' (packet block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

cloudflare-workers-and-pages · 2025-09-05T13:55:47Z

Deploying docs with Cloudflare Pages

Latest commit:	`b2f8906`
Status:	✅ Deploy successful!
Preview URL:	https://0bbf833a.docs-dog.pages.dev
Branch Preview URL:	https://copilot-fix-492.docs-dog.pages.dev

View logs

Co-authored-by: ppiegaze <[email protected]>

Copilot

Pull Request Overview

This PR migrates the unionai-docs project from Cloudflare Pages to a GitHub Actions-based build system with Cloudflare Workers deployment. The migration enables better control over the build process while maintaining Cloudflare hosting, and introduces advanced multi-version deployment capabilities.

Key changes implemented:

Multi-version combined builds that automatically build and deploy both v1 and v2 versions when either branch is updated
Enhanced PR preview system with automatic comments containing direct links to all documentation variants
Robust CI-friendly build process with graceful handling of missing submodules and dependencies

Reviewed Changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`.github/workflows/build-and-deploy.yml`	Production deployment workflow for main/v1 branches
`.github/workflows/pr-preview.yml`	PR preview workflow with automatic commenting
`.github/workflows/cleanup-pr.yml`	Cleanup workflow for removing PR preview environments
`scripts/build-combined.sh`	Main build orchestration script for multi-version builds
`scripts/run_hugo_ci.sh`	CI-friendly Hugo wrapper that handles warnings gracefully
`worker.js`	Cloudflare Worker for routing and redirects
`wrangler.toml`	Cloudflare Workers deployment configuration
`Makefile`	Added CI-friendly build targets
`MIGRATION.md`	Comprehensive setup and migration instructions
`.github/README.md`	Documentation for CI/CD architecture and workflows

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-05T14:24:46Z

worker.js

+      if (!url.pathname.startsWith('/docs/')) {
+        newUrl.pathname = '/docs/flyte' + url.pathname;
+      }
+      return Response.redirect(newUrl.toString(), 301);


The legacy domain redirect logic (lines 29-35) appears to be handling docs.flyte.org redirects but may conflict with the main routing logic. Consider adding a comment explaining the expected URL patterns and ensure this doesn't interfere with normal routing.

Copilot · 2025-09-05T14:24:46Z

scripts/build-combined.sh

+    log "Switching to $OTHER_BRANCH for $OTHER_VERSION"
+    git checkout "$OTHER_BRANCH"
+
+    log "Building other version ($OTHER_VERSION) from $OTHER_BRANCH"
+    make dist-ci
+
+    # Switch back to PR branch
+    git checkout "$HEAD_REF" || git checkout -


The fallback git checkout - may not reliably return to the intended PR branch if $HEAD_REF checkout fails. Consider storing the current commit hash before branch switching and checking out that specific commit as a more reliable fallback.

Suggested change

log "Switching to $OTHER_BRANCH for $OTHER_VERSION"

git checkout "$OTHER_BRANCH"

log "Building other version ($OTHER_VERSION) from $OTHER_BRANCH"

make dist-ci

# Switch back to PR branch

git checkout "$HEAD_REF" || git checkout -

# Store current commit hash before switching branches

ORIGINAL_COMMIT="$(git rev-parse HEAD)"

log "Switching to $OTHER_BRANCH for $OTHER_VERSION"

git checkout "$OTHER_BRANCH"

log "Building other version ($OTHER_VERSION) from $OTHER_BRANCH"

make dist-ci

# Switch back to PR branch, or fallback to original commit hash

git checkout "$HEAD_REF" || git checkout "$ORIGINAL_COMMIT"

Copilot · 2025-09-05T14:24:47Z

.github/workflows/cleanup-pr.yml

+          # Try to delete the preview deployment
+          # This might fail if the deployment doesn't exist, which is OK
+          wrangler pages deployment list --project-name=unionai-docs --compatibility-date=2024-01-01 | \
+          grep "${{ steps.preview.outputs.preview_name }}" | \
+          head -1 | \
+          awk '{print $1}' | \
+          xargs -r wrangler pages deployment delete --project-name=unionai-docs || true


The pipeline for deleting preview deployments is complex and fragile. The grep pattern could match partial strings, and the awk command assumes a specific output format. Consider using wrangler pages deployment delete with more specific parameters or JSON output parsing for more reliable cleanup.

Suggested change

# Try to delete the preview deployment

# This might fail if the deployment doesn't exist, which is OK

wrangler pages deployment list --project-name=unionai-docs --compatibility-date=2024-01-01 | \

grep "${{ steps.preview.outputs.preview_name }}" | \

head -1 | \

awk '{print $1}' | \

xargs -r wrangler pages deployment delete --project-name=unionai-docs || true

# Try to delete the preview deployment(s) using robust JSON parsing

# This might fail if the deployment doesn't exist, which is OK

DEPLOYMENT_IDS=$(wrangler pages deployment list --project-name=unionai-docs --compatibility-date=2024-01-01 --output json | \

jq -r --arg env "${{ steps.preview.outputs.preview_name }}" '.[] | select(.environment==$env) | .id')

for id in $DEPLOYMENT_IDS; do

wrangler pages deployment delete --project-name=unionai-docs "$id" || true

done

Copilot · 2025-09-05T14:24:47Z

.github/workflows/build-and-deploy.yml

+      - name: Upload build artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: production-build-${{ steps.versions.outputs.current_version }}-${{ github.sha }}


The artifact name references steps.versions.outputs.current_version but there is no step with id 'versions' defined in this workflow. This will result in an empty string in the artifact name.

Initial plan

bc52923

Copilot AI assigned Copilot and ppiegaze Sep 5, 2025

Copilot started work on behalf of ppiegaze September 5, 2025 13:55 View session

Copilot AI and others added 3 commits September 5, 2025 14:07

Add GitHub Actions workflows and Cloudflare Workers setup

2cd6c8d

Co-authored-by: ppiegaze <[email protected]>

Improve CI build process and add redirects handling

ab1537e

Co-authored-by: ppiegaze <[email protected]>

Final migration implementation with documentation and testing

b2f8906

Co-authored-by: ppiegaze <[email protected]>

Copilot AI changed the title ~~[WIP] Migrate site to Cloudflare Workers and GitHub Actions~~ Migrate site to Cloudflare Workers and GitHub Actions Sep 5, 2025

Copilot AI requested a review from ppiegaze September 5, 2025 14:18

Copilot finished work on behalf of ppiegaze September 5, 2025 14:18

ppiegaze requested a review from Copilot September 5, 2025 14:23

Copilot AI reviewed Sep 5, 2025

View reviewed changes

ppiegaze requested a review from nelsonjr September 5, 2025 14:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate site to Cloudflare Workers and GitHub Actions #493

Migrate site to Cloudflare Workers and GitHub Actions #493

Uh oh!

Copilot AI commented Sep 5, 2025 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages bot commented Sep 5, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 5, 2025

Uh oh!

Copilot AI Sep 5, 2025

Uh oh!

Copilot AI Sep 5, 2025

Uh oh!

Copilot AI Sep 5, 2025

Uh oh!

Uh oh!

Migrate site to Cloudflare Workers and GitHub Actions #493

Are you sure you want to change the base?

Migrate site to Cloudflare Workers and GitHub Actions #493

Uh oh!

Conversation

Copilot AI commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Migration Overview

Key Features Implemented

Technical Implementation

Site Structure

Setup Requirements

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

cloudflare-workers-and-pages bot commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying docs with Cloudflare Pages

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI commented Sep 5, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Sep 5, 2025 •

edited

Loading