Skip to content

Support Speedscope format to profiling pipeline #1405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jan 23, 2024
2 changes: 1 addition & 1 deletion .github/workflows/ci_consumption_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
fail-fast: false
matrix:
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]

permissions: read-all
steps:
- name: Checkout code.
uses: actions/checkout@v2
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci_docker_con_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
fail-fast: false
matrix:
python-version: [ 3.7, 3.8, 3.9, "3.10" ]
permissions: read-all
env:
CONSUMPTION_DOCKER_TEST: "true"

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci_docker_custom_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
fail-fast: false
permissions: read-all
env:
DEDICATED_DOCKER_TEST: ${{ github.event.inputs.DEDICATED_DOCKER_TEST }}
CONSUMPTION_DOCKER_TEST: ${{ github.event.inputs.CONSUMPTION_DOCKER_TEST }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci_docker_ded_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
fail-fast: false
matrix:
python-version: [ 3.7, 3.8, 3.9, "3.10" ]
permissions: read-all
env:
DEDICATED_DOCKER_TEST: "true"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci_e2e_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
fail-fast: false
matrix:
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]

permissions: read-all
steps:
- name: Checkout code.
uses: actions/checkout@v2
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci_ut_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ jobs:
fail-fast: false
matrix:
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]
permissions: read-all
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ jobs:
language: [ 'python' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://git.io/codeql-language-support

permissions: read-all

steps:
- name: Checkout repository
uses: actions/checkout@v3
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
name: Lint Code Base
# Set the agent to run on
runs-on: ubuntu-latest
permissions: read-all

##################
# Load all steps #
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ogf_workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ jobs:
STORAGE_ACCOUNT_NAME: "ogf${{ matrix.sku }}${{ matrix.region }}${{ github.run_number }}"
FUNCTION_APP_PLAN_NAME: "ogf-epplan-${{ matrix.sku }}-${{ matrix.region }}-${{ github.run_number }}"
runs-on: ubuntu-latest
permissions: read-all
timeout-minutes: 30
strategy:
# V. IMPORTANT: Copy this strategy section to the dependent
Expand Down
27 changes: 20 additions & 7 deletions .github/workflows/perf-testing-setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,13 @@ on:
description: 'List of perf tests to run'
required: false
default: SyncHelloWorld
report_format:
description: 'Format of profiling report'
type: choice
required: true
options:
- speedscope
- flamegraph
issue_comment:
types: [created]
env:
Expand All @@ -21,8 +28,9 @@ env:

jobs:
build:
if: ${{ github.event_name == 'workflow_dispatch' || github.event.issue.pull_request && contains(github.event.comment.body, '/profile') }}
if: ${{ github.event_name == 'workflow_dispatch' || github.event.issue.pull_request != null && contains(github.event.comment.body, '/profile') }}
runs-on: ubuntu-latest
permissions: read-all
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -74,9 +82,16 @@ jobs:
docker exec $container_id sh -c "pip install py-spy"
docker exec $container_id sh -c "mkdir /home/profiling_reports"
profiling_sampling_rate=${{ github.event.inputs.profiling_sampling_rate }}
# report_name="${GITHUB_REF#refs/heads/}_${{ github.run_id }}.svg"
report_name="${{ github.run_id }}.svg"
docker exec -d $container_id sh -c "RUST_BACKTRACE=1 py-spy record -p $worker_pid -o /home/profiling_reports/$report_name -f flamegraph --idle --nonblocking --rate $profiling_sampling_rate > /home/site/wwwroot/py-spy.log 2>&1 &"
report_format=${{ github.event.inputs.report_format }}
if [ "$report_format" == "flamegraph" ]; then
report_name="${{ github.run_id }}.svg"
elif [ "$report_format" == "speedscope" ]; then
report_name="${{ github.run_id }}.speedscope.json"
else
echo "Unsupported report format: $report_format"
exit 1
fi
docker exec -d $container_id sh -c "RUST_BACKTRACE=1 py-spy record -p $worker_pid -o /home/profiling_reports/$report_name -f $report_format --idle --nonblocking --rate $profiling_sampling_rate > /home/site/wwwroot/py-spy.log 2>&1 &"
sleep 2 # Give it a moment to start
py_spy_id=$(docker exec $container_id sh -c "ps aux | grep '[p]y-spy record'" | awk '{print $2}')

Expand All @@ -94,11 +109,9 @@ jobs:
docker exec $container_id cat /home/site/wwwroot/py-spy.log
docker exec $container_id sh -c "kill -2 $py_spy_id"
sleep 2

mkdir profiling_reports
chmod 777 profiling_reports
docker cp $container_id:/home/profiling_reports/$report_name profiling_reports

- name: Upload SVG to Azure Blob Storage
uses: bacongobbler/[email protected]
with:
Expand All @@ -111,7 +124,7 @@ jobs:
run: |
blob_url="https://${{ env.PYTHON_FUNCTION_PROFILING_STORAGE_ACCT }}.blob.core.windows.net/profiling/${{ env.report_name }}"
echo "You can view the Blob at: $blob_url"

- name: Upload profiling result to artifact
uses: actions/upload-artifact@v2
with:
Expand Down