Skip to content

ignore: just trying to nail down a bug #124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 53 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
90c23be
Add entity-manager dep
rklaehn Jul 8, 2025
90db719
Remove unused dependencies
rklaehn Jul 8, 2025
24c7f2c
Merge branch 'lose-futures-buffered-dep' into entity-manager
rklaehn Jul 8, 2025
126d834
Merge branch 'main' into entity-manager
rklaehn Jul 8, 2025
264ed84
Add import
rklaehn Jul 8, 2025
a859674
Add helpers for set and update
rklaehn Jul 8, 2025
baf4d54
Make bipolar clippy happy again
rklaehn Jul 8, 2025
66f8540
Move message sending into meta
rklaehn Jul 8, 2025
9193b08
Merge branch 'remove-db-public-send' into entity-manager
rklaehn Jul 8, 2025
8cae7cd
WIP
rklaehn Jul 8, 2025
161a70b
Replace slots with EntityHandler
rklaehn Jul 8, 2025
169261a
Get rid of the weak reference
rklaehn Jul 8, 2025
6fe2e07
Inline entity manager and streamline the dispatcher code a bit
rklaehn Jul 9, 2025
fd9faab
revert a few changes to minimize the diff
rklaehn Jul 9, 2025
8ac1b78
codespell
rklaehn Jul 9, 2025
011746d
remove debug code
rklaehn Jul 9, 2025
def9abc
more comments
rklaehn Jul 9, 2025
3d603b0
remove options from BaoFileHandleInner
rklaehn Jul 9, 2025
e65bd07
Get rid of BaoFileHandleInner and all that indirection
rklaehn Jul 9, 2025
e81b1f3
change a few fns to take HashContext directly instead of 2 args
rklaehn Jul 9, 2025
27df6da
more using HashContext instead of TaskContext
rklaehn Jul 9, 2025
77fbb05
First step for removing the slot mutex
rklaehn Jul 9, 2025
ae95f28
prepare BaoFileHandle to be the full state
rklaehn Jul 9, 2025
c786a04
Flatten the state we keep per active blob
rklaehn Jul 10, 2025
94df43f
Remove debug statements
rklaehn Jul 10, 2025
48ffd48
Remove unused code
rklaehn Jul 10, 2025
a540002
Remove more debug code
rklaehn Jul 10, 2025
6e237f8
instrument the per-hash tasks as well
rklaehn Jul 10, 2025
47cab9c
Add property based tests for the entity manager
rklaehn Jul 10, 2025
00c54f3
Add a property based test that tests basic functionality of the entit…
rklaehn Jul 10, 2025
664beb8
clippy
rklaehn Jul 10, 2025
7b1b71b
Merge branch 'main' into entity-manager
rklaehn Jul 10, 2025
e584ad1
fmt
rklaehn Jul 10, 2025
4412be9
Add tests for the busy and dead cases which are hard to trigger witho…
rklaehn Jul 10, 2025
c26e555
clippy again
rklaehn Jul 10, 2025
002469a
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 11, 2025
7857e38
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 11, 2025
f559598
Move things around in preparation for introducing traits
rklaehn Jul 11, 2025
bc6bc71
Group high level fns.
rklaehn Jul 11, 2025
632131f
Introduce traits to structure the API
rklaehn Jul 11, 2025
8004115
Merge branch 'main' into entity-manager
rklaehn Jul 11, 2025
4020bd4
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 11, 2025
1665781
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 11, 2025
9a8547a
clippy
rklaehn Jul 11, 2025
c57f259
Merge branch 'main' into entity-manager
rklaehn Jul 23, 2025
331a3aa
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 23, 2025
e2b76fb
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 23, 2025
42bbbec
Merge branch 'main' into entity-manager
rklaehn Jul 24, 2025
b595923
Merge branch 'entity-manager' into slim-down-state
rklaehn Jul 24, 2025
6d39d40
Merge branch 'slim-down-state' into remove-slot-mutex
rklaehn Jul 24, 2025
48e01dd
disable most CI
rklaehn Jul 24, 2025
dc1fa56
Disable everything but the ubuntu tests
rklaehn Jul 24, 2025
580259f
Make the test run 100x
rklaehn Jul 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
502 changes: 251 additions & 251 deletions .github/workflows/ci.yaml

Large diffs are not rendered by default.

18 changes: 9 additions & 9 deletions .github/workflows/commit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ on:
env:
IROH_FORCE_STAGING_RELAYS: "1"

jobs:
check-for-cc:
runs-on: ubuntu-latest
steps:
- name: check-for-cc
id: check-for-cc
uses: agenthunt/[email protected]
with:
pr-title-regex: "^(.+)(?:(([^)s]+)))?!?: (.+)"
# jobs:
# check-for-cc:
# runs-on: ubuntu-latest
# steps:
# - name: check-for-cc
# id: check-for-cc
# uses: agenthunt/[email protected]
# with:
# pr-title-regex: "^(.+)(?:(([^)s]+)))?!?: (.+)"
124 changes: 62 additions & 62 deletions .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
@@ -1,73 +1,73 @@
name: Docs Preview
# name: Docs Preview

on:
pull_request:
workflow_dispatch:
inputs:
pr_number:
required: true
type: string
# on:
# pull_request:
# workflow_dispatch:
# inputs:
# pr_number:
# required: true
# type: string

# ensure job runs sequentially so pushing to the preview branch doesn't conflict
concurrency:
group: ci-docs-preview
# # ensure job runs sequentially so pushing to the preview branch doesn't conflict
# concurrency:
# group: ci-docs-preview

env:
IROH_FORCE_STAGING_RELAYS: "1"
# env:
# IROH_FORCE_STAGING_RELAYS: "1"

jobs:
preview_docs:
permissions: write-all
timeout-minutes: 30
name: Docs preview
if: ${{ (github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch' ) && !github.event.pull_request.head.repo.fork }}
runs-on: ubuntu-latest
env:
RUSTC_WRAPPER: "sccache"
SCCACHE_GHA_ENABLED: "on"
SCCACHE_CACHE_SIZE: "50G"
PREVIEW_PATH: pr/${{ github.event.pull_request.number || inputs.pr_number }}/docs
# jobs:
# preview_docs:
# permissions: write-all
# timeout-minutes: 30
# name: Docs preview
# if: ${{ (github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch' ) && !github.event.pull_request.head.repo.fork }}
# runs-on: ubuntu-latest
# env:
# RUSTC_WRAPPER: "sccache"
# SCCACHE_GHA_ENABLED: "on"
# SCCACHE_CACHE_SIZE: "50G"
# PREVIEW_PATH: pr/${{ github.event.pull_request.number || inputs.pr_number }}/docs

steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@master
with:
toolchain: nightly-2024-11-30
- name: Install sccache
uses: mozilla-actions/[email protected]
# steps:
# - uses: actions/checkout@v4
# - uses: dtolnay/rust-toolchain@master
# with:
# toolchain: nightly-2024-11-30
# - name: Install sccache
# uses: mozilla-actions/[email protected]

- name: Generate Docs
run: cargo doc --workspace --all-features --no-deps
env:
RUSTDOCFLAGS: --cfg iroh_docsrs
# - name: Generate Docs
# run: cargo doc --workspace --all-features --no-deps
# env:
# RUSTDOCFLAGS: --cfg iroh_docsrs

- name: Deploy Docs to Preview Branch
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./target/doc/
destination_dir: ${{ env.PREVIEW_PATH }}
publish_branch: generated-docs-preview
# - name: Deploy Docs to Preview Branch
# uses: peaceiris/actions-gh-pages@v4
# with:
# github_token: ${{ secrets.GITHUB_TOKEN }}
# publish_dir: ./target/doc/
# destination_dir: ${{ env.PREVIEW_PATH }}
# publish_branch: generated-docs-preview

- name: Find Docs Comment
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number || inputs.pr_number }}
comment-author: 'github-actions[bot]'
body-includes: Documentation for this PR has been generated
# - name: Find Docs Comment
# uses: peter-evans/find-comment@v3
# id: fc
# with:
# issue-number: ${{ github.event.pull_request.number || inputs.pr_number }}
# comment-author: 'github-actions[bot]'
# body-includes: Documentation for this PR has been generated

- name: Get current timestamp
id: get_timestamp
run: echo "TIMESTAMP=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
# - name: Get current timestamp
# id: get_timestamp
# run: echo "TIMESTAMP=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV

- name: Create or Update Docs Comment
uses: peter-evans/create-or-update-comment@v4
with:
issue-number: ${{ github.event.pull_request.number || inputs.pr_number }}
comment-id: ${{ steps.fc.outputs.comment-id }}
body: |
Documentation for this PR has been generated and is available at: https://${{ github.repository_owner }}.github.io/${{ github.event.repository.name }}/${{ env.PREVIEW_PATH }}/iroh_blobs/
# - name: Create or Update Docs Comment
# uses: peter-evans/create-or-update-comment@v4
# with:
# issue-number: ${{ github.event.pull_request.number || inputs.pr_number }}
# comment-id: ${{ steps.fc.outputs.comment-id }}
# body: |
# Documentation for this PR has been generated and is available at: https://${{ github.repository_owner }}.github.io/${{ github.event.repository.name }}/${{ env.PREVIEW_PATH }}/iroh_blobs/

Last updated: ${{ env.TIMESTAMP }}
edit-mode: replace
# Last updated: ${{ env.TIMESTAMP }}
# edit-mode: replace
188 changes: 94 additions & 94 deletions .github/workflows/flaky.yaml
Original file line number Diff line number Diff line change
@@ -1,99 +1,99 @@
# Run all tests, including flaky test.
#
# The default CI workflow ignores flaky tests. This workflow will run
# all tests, including ignored ones.
#
# To use this workflow you can either:
#
# - Label a PR with "flaky-test", the normal CI workflow will not run
# any jobs but the jobs here will be run. Note that to merge the PR
# you'll need to remove the label eventually because the normal CI
# jobs are required by branch protection.
#
# - Manually trigger the workflow, you may choose a branch for this to
# run on.
#
# Additionally this jobs runs once a day on a schedule.
#
# Currently doctests are not run by this workflow.
# # Run all tests, including flaky test.
# #
# # The default CI workflow ignores flaky tests. This workflow will run
# # all tests, including ignored ones.
# #
# # To use this workflow you can either:
# #
# # - Label a PR with "flaky-test", the normal CI workflow will not run
# # any jobs but the jobs here will be run. Note that to merge the PR
# # you'll need to remove the label eventually because the normal CI
# # jobs are required by branch protection.
# #
# # - Manually trigger the workflow, you may choose a branch for this to
# # run on.
# #
# # Additionally this jobs runs once a day on a schedule.
# #
# # Currently doctests are not run by this workflow.

name: Flaky CI
# name: Flaky CI

on:
pull_request:
types: [ 'labeled', 'unlabeled', 'opened', 'synchronize', 'reopened' ]
schedule:
# 06:30 UTC every day
- cron: '30 6 * * *'
workflow_dispatch:
inputs:
branch:
description: 'Branch to run on, defaults to main'
required: true
default: 'main'
type: string
# on:
# pull_request:
# types: [ 'labeled', 'unlabeled', 'opened', 'synchronize', 'reopened' ]
# schedule:
# # 06:30 UTC every day
# - cron: '30 6 * * *'
# workflow_dispatch:
# inputs:
# branch:
# description: 'Branch to run on, defaults to main'
# required: true
# default: 'main'
# type: string

concurrency:
group: flaky-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
# concurrency:
# group: flaky-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
# cancel-in-progress: true

env:
IROH_FORCE_STAGING_RELAYS: "1"
# env:
# IROH_FORCE_STAGING_RELAYS: "1"

jobs:
tests:
if: "contains(github.event.pull_request.labels.*.name, 'flaky-test') || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'"
uses: './.github/workflows/tests.yaml'
with:
flaky: true
git-ref: ${{ inputs.branch }}
notify:
needs: tests
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Extract test results
run: |
printf '${{ toJSON(needs) }}\n'
result=$(echo '${{ toJSON(needs) }}' | jq -r .tests.result)
echo TESTS_RESULT=$result
echo "TESTS_RESULT=$result" >>"$GITHUB_ENV"
- name: download nextest reports
uses: actions/download-artifact@v4
with:
pattern: libtest_run_${{ github.run_number }}-${{ github.run_attempt }}-*
merge-multiple: true
path: nextest-results
- name: create summary report
id: make_summary
run: |
# prevent the glob expression in the loop to match on itself when the dir is empty
shopt -s nullglob
# to deal with multiline outputs it's recommended to use a random EOF, the syntax is based on
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings
EOF=aP51VriWCxNJ1JjvmO9i
echo "summary<<$EOF" >> $GITHUB_OUTPUT
echo "Flaky tests failure:" >> $GITHUB_OUTPUT
echo " " >> $GITHUB_OUTPUT
for report in nextest-results/*.json; do
# remove the name prefix and extension, and split the parts
name=$(echo ${report:16:-5} | tr _ ' ')
echo $name
echo "- **$name**" >> $GITHUB_OUTPUT
# select the failed tests
# the tests have this format "crate::module$test_name", the sed expressions remove the quotes and replace $ for ::
failure=$(jq --slurp '.[] | select(.["type"] == "test" and .["event"] == "failed" ) | .["name"]' $report | sed -e 's/^"//g' -e 's/\$/::/' -e 's/"//')
echo "$failure"
echo "$failure" >> $GITHUB_OUTPUT
done
echo "" >> $GITHUB_OUTPUT
echo "See https://github.com/${{ github.repository }}/actions/workflows/flaky.yaml" >> $GITHUB_OUTPUT
echo "$EOF" >> $GITHUB_OUTPUT
- name: Notify discord on failure
uses: n0-computer/discord-webhook-notify@v1
if: ${{ env.TESTS_RESULT == 'failure' || env.TESTS_RESULT == 'success' }}
with:
text: "Flaky tests in **${{ github.repository }}**:"
severity: ${{ env.TESTS_RESULT == 'failure' && 'warn' || 'info' }}
details: ${{ env.TESTS_RESULT == 'failure' && steps.make_summary.outputs.summary || 'No flaky failures!' }}
webhookUrl: ${{ secrets.DISCORD_N0_GITHUB_CHANNEL_WEBHOOK_URL }}
# jobs:
# tests:
# if: "contains(github.event.pull_request.labels.*.name, 'flaky-test') || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'"
# uses: './.github/workflows/tests.yaml'
# with:
# flaky: true
# git-ref: ${{ inputs.branch }}
# notify:
# needs: tests
# if: ${{ always() }}
# runs-on: ubuntu-latest
# steps:
# - name: Extract test results
# run: |
# printf '${{ toJSON(needs) }}\n'
# result=$(echo '${{ toJSON(needs) }}' | jq -r .tests.result)
# echo TESTS_RESULT=$result
# echo "TESTS_RESULT=$result" >>"$GITHUB_ENV"
# - name: download nextest reports
# uses: actions/download-artifact@v4
# with:
# pattern: libtest_run_${{ github.run_number }}-${{ github.run_attempt }}-*
# merge-multiple: true
# path: nextest-results
# - name: create summary report
# id: make_summary
# run: |
# # prevent the glob expression in the loop to match on itself when the dir is empty
# shopt -s nullglob
# # to deal with multiline outputs it's recommended to use a random EOF, the syntax is based on
# # https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings
# EOF=aP51VriWCxNJ1JjvmO9i
# echo "summary<<$EOF" >> $GITHUB_OUTPUT
# echo "Flaky tests failure:" >> $GITHUB_OUTPUT
# echo " " >> $GITHUB_OUTPUT
# for report in nextest-results/*.json; do
# # remove the name prefix and extension, and split the parts
# name=$(echo ${report:16:-5} | tr _ ' ')
# echo $name
# echo "- **$name**" >> $GITHUB_OUTPUT
# # select the failed tests
# # the tests have this format "crate::module$test_name", the sed expressions remove the quotes and replace $ for ::
# failure=$(jq --slurp '.[] | select(.["type"] == "test" and .["event"] == "failed" ) | .["name"]' $report | sed -e 's/^"//g' -e 's/\$/::/' -e 's/"//')
# echo "$failure"
# echo "$failure" >> $GITHUB_OUTPUT
# done
# echo "" >> $GITHUB_OUTPUT
# echo "See https://github.com/${{ github.repository }}/actions/workflows/flaky.yaml" >> $GITHUB_OUTPUT
# echo "$EOF" >> $GITHUB_OUTPUT
# - name: Notify discord on failure
# uses: n0-computer/discord-webhook-notify@v1
# if: ${{ env.TESTS_RESULT == 'failure' || env.TESTS_RESULT == 'success' }}
# with:
# text: "Flaky tests in **${{ github.repository }}**:"
# severity: ${{ env.TESTS_RESULT == 'failure' && 'warn' || 'info' }}
# details: ${{ env.TESTS_RESULT == 'failure' && steps.make_summary.outputs.summary || 'No flaky failures!' }}
# webhookUrl: ${{ secrets.DISCORD_N0_GITHUB_CHANNEL_WEBHOOK_URL }}
Loading
Loading