Skip to content

feat: Add initial benchmarks, integrate them into CI & add getters/settters for Scripts resource #381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 22, 2025

Conversation

makspll
Copy link
Owner

@makspll makspll commented Mar 22, 2025

Summary

  • Moves tests to run centrally in the workspace tests directory
  • Refactors test utilities to share a lot of the test discovery and running code between languages and benchmarks
  • Adds criterion benchmarks
  • Adds CI for snapshoting performance from main to bencher.dev

I've also noticed that Scripts didn't really have nice ways to be accessed by external crates, so I added a bunch of accessors

@makspll makspll requested a review from Copilot March 22, 2025 14:18
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR centralizes test discovery and execution while introducing basic benchmarking support with Criterion and CI integration for performance snapshoting.

  • Centralizes test running by moving tests to a common directory and refactoring test utilities.
  • Introduces benchmark executors and grouping logic to run language-specific benchmarks.
  • Adds CI workflow for continuous performance benchmarking with bencher.dev.

Reviewed Changes

Copilot reviewed 12 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/script_tests.rs Introduces a centralized test runner with improved test discovery for Lua and Rhai integration tests.
.github/workflows/snapshot_benchmark_main.yml Adds a new GitHub Actions workflow to run and snapshot performance benchmarks.
benches/benchmarks.rs Implements benchmark grouping and execution for scripts using Criterion.
crates/testing_crates/test_utils/src/lib.rs Updates test discovery logic with minor refactoring of directory traversal.
crates/testing_crates/script_integration_test_harness/src/lib.rs Refactors integration test harness to support benchmarking and adds language-specific plugin initializers.
crates/bevy_mod_scripting_core/src/script.rs Extends the Scripts API with additional collection methods.
Cargo.toml and language crate Cargo.toml files Updates dependency and workspace configurations to support the changes and remove redundant tests from language crates.
Files not reviewed (6)
  • assets/benchmarks/component/access.lua: Language not supported
  • assets/benchmarks/component/access.rhai: Language not supported
  • assets/benchmarks/component/get.lua: Language not supported
  • assets/benchmarks/component/get.rhai: Language not supported
  • crates/languages/bevy_mod_scripting_lua/tests/data/construct/construct_unit_struct.lua: Language not supported
  • crates/languages/bevy_mod_scripting_rhai/tests/data/construct/unit_struct.rhai: Language not supported
Comments suppressed due to low confidence (2)

tests/script_tests.rs:35

  • Using unwrap on the result of splitting the path assumes the delimiter 'tests{sep}data' is always present, which may lead to a panic if the expected structure is not met. Consider using a more robust method or adding a check to handle missing segments gracefully.
                .to_string_lossy().split(&format!("tests{}data", std::path::MAIN_SEPARATOR)).last().unwrap()

benches/benchmarks.rs:29

  • Indexing directly into the split result at index 1 may lead to a panic if the resulting vector has fewer elements than expected. Consider verifying the vector length before indexing to ensure robust benchmark grouping.
let first_folder = path.split("/").collect::<Vec<&str>>()[1];

@makspll makspll changed the title chore: centralize tests & add basic benchmarks feat: Add initial benchmarks, integrate them into CI & add getters/settters for Scripts resource Mar 22, 2025
@makspll makspll enabled auto-merge (squash) March 22, 2025 14:22
@makspll makspll mentioned this pull request Mar 22, 2025
5 tasks
@makspll makspll merged commit 06c1537 into main Mar 22, 2025
17 checks passed
@makspll makspll deleted the chore--benchmarks branch March 22, 2025 15:27
This was referenced Mar 22, 2025
makspll added a commit that referenced this pull request Mar 29, 2025
## 🤖 New release

* `bevy_mod_scripting_derive`: 0.10.0 -> 0.11.0
* `bevy_mod_scripting_core`: 0.10.0 -> 0.11.0 (✓ API compatible changes)
* `bevy_mod_scripting_lua`: 0.10.0 -> 0.11.0 (✓ API compatible changes)
* `bevy_mod_scripting_rhai`: 0.10.0 -> 0.11.0 (✓ API compatible changes)
* `bevy_mod_scripting_functions`: 0.10.0 -> 0.11.0 (✓ API compatible
changes)
* `ladfile`: 0.4.0 -> 0.5.0 (⚠ API breaking changes)
* `mdbook_lad_preprocessor`: 0.1.4 -> 0.1.5 (✓ API compatible changes)
* `ladfile_builder`: 0.2.6 -> 0.3.0 (⚠ API breaking changes)
* `bevy_mod_scripting`: 0.10.0 -> 0.11.0 (✓ API compatible changes)

### ⚠ `ladfile` breaking changes

```text
--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field LadType.generated in /tmp/.tmpI1fySQ/bevy_mod_scripting/crates/ladfile/src/lib.rs:458
  field LadType.insignificance in /tmp/.tmpI1fySQ/bevy_mod_scripting/crates/ladfile/src/lib.rs:465

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/method_parameter_count_changed.ron

Failed in:
  ladfile::LadFile::get_type_identifier now takes 3 parameters instead of 2, in /tmp/.tmpI1fySQ/bevy_mod_scripting/crates/ladfile/src/lib.rs:55
```

### ⚠ `ladfile_builder` breaking changes

```text
--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field LadFileSettings.exclude_types_containing_unregistered in /tmp/.tmpI1fySQ/bevy_mod_scripting/crates/ladfile_builder/src/plugin.rs:42

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/method_parameter_count_changed.ron

Failed in:
  ladfile_builder::plugin::ScriptingDocgenPlugin::new now takes 4 parameters instead of 3, in /tmp/.tmpI1fySQ/bevy_mod_scripting/crates/ladfile_builder/src/plugin.rs:61
```

<details><summary><i><b>Changelog</b></i></summary><p>

## `bevy_mod_scripting_derive`

<blockquote>

##
[0.11.0](bevy_mod_scripting_derive-v0.10.0...bevy_mod_scripting_derive-v0.11.0)
- 2025-03-29

### Added

- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
</blockquote>

## `bevy_mod_scripting_core`

<blockquote>

##
[0.11.0](bevy_mod_scripting_core-v0.10.0...bevy_mod_scripting_core-v0.11.0)
- 2025-03-29

### Added

- optimize access map
([#395](#395))
- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- improve tracing spans, add `profile_with_tracy` feature flag
([#394](#394))
- add `profile_with_tracy` feature which plays nicely with bevy's
`bevy/trace_tracy` feature
([#393](#393))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
- add ScriptValue override for printing opaque values
([#380](#380))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))

### Fixed

- fix global type cache not containing generic types
([#388](#388))

### Other

- switch to hashbrown hashmap in the function registry
([#399](#399))
- try play with hashing for access maps
([#398](#398))
- allow check creation for bencher
</blockquote>

## `bevy_mod_scripting_lua`

<blockquote>

##
[0.11.0](bevy_mod_scripting_lua-v0.10.0...bevy_mod_scripting_lua-v0.11.0)
- 2025-03-29

### Added

- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- allow the conversion of lua functions into `ScriptValue` via
`DynamicScriptFunction`
([#396](#396))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
</blockquote>

## `bevy_mod_scripting_rhai`

<blockquote>

##
[0.11.0](bevy_mod_scripting_rhai-v0.10.0...bevy_mod_scripting_rhai-v0.11.0)
- 2025-03-29

### Added

- [**breaking**] bump bersion
- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
</blockquote>

## `bevy_mod_scripting_functions`

<blockquote>

##
[0.11.0](bevy_mod_scripting_functions-v0.10.0...bevy_mod_scripting_functions-v0.11.0)
- 2025-03-29

### Added

- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
</blockquote>

## `ladfile`

<blockquote>

##
[0.5.0](v0.4.0-ladfile...v0.5.0-ladfile)
- 2025-03-29

### Added

- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
</blockquote>

## `mdbook_lad_preprocessor`

<blockquote>

##
[0.1.5](v0.1.4-mdbook_lad_preprocessor...v0.1.5-mdbook_lad_preprocessor)
- 2025-03-29

### Added

- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))

### Fixed

- make all links in the mdbook preprocessor relative
([#392](#392))
- mdbook preprocessor links not taking into account root url
([#391](#391))
</blockquote>

## `ladfile_builder`

<blockquote>

##
[0.3.0](v0.2.6-ladfile_builder...v0.3.0-ladfile_builder)
- 2025-03-29

### Added

- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
</blockquote>

## `bevy_mod_scripting`

<blockquote>

##
[0.11.0](v0.10.0...v0.11.0)
- 2025-03-29

### Added

- allow the conversion of lua functions into `ScriptValue` via
`DynamicScriptFunction`
([#396](#396))
- improve tracing spans, add `profile_with_tracy` feature flag
([#394](#394))
- add `profile_with_tracy` feature which plays nicely with bevy's
`bevy/trace_tracy` feature
([#393](#393))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- optimize access map
([#395](#395))
- add ScriptValue override for printing opaque values
([#380](#380))
- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
- [**breaking**] bump bersion

### Fixed

- fix global type cache not containing generic types
([#388](#388))

### Other

- switch to hashbrown hashmap in the function registry
([#399](#399))
- try play with hashing for access maps
([#398](#398))
- allow check creation for bencher
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Maksymilian Mozolewski <[email protected]>
makspll pushed a commit that referenced this pull request Mar 30, 2025
## 🤖 New release

* `bevy_mod_scripting_derive`: 0.11.0 -> 0.11.1
* `bevy_system_reflection`: 0.1.0 -> 0.1.1 (✓ API compatible changes)
* `bevy_mod_scripting_core`: 0.11.0 -> 0.11.1 (✓ API compatible changes)
* `bevy_mod_scripting_lua`: 0.11.0 -> 0.11.1 (✓ API compatible changes)
* `bevy_mod_scripting_rhai`: 0.11.0 -> 0.11.1 (✓ API compatible changes)
* `bevy_mod_scripting_functions`: 0.11.0 -> 0.11.1 (✓ API compatible
changes)
* `ladfile_builder`: 0.3.0 -> 0.3.1 (✓ API compatible changes)
* `bevy_mod_scripting`: 0.11.0 -> 0.11.1 (✓ API compatible changes)

<details><summary><i><b>Changelog</b></i></summary><p>

## `bevy_mod_scripting_derive`

<blockquote>

##
[0.11.0](bevy_mod_scripting_derive-v0.10.0...bevy_mod_scripting_derive-v0.11.0)
- 2025-03-29

### Added

- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
</blockquote>


## `bevy_mod_scripting_core`

<blockquote>

##
[0.11.0](bevy_mod_scripting_core-v0.10.0...bevy_mod_scripting_core-v0.11.0)
- 2025-03-29

### Added

- optimize access map
([#395](#395))
- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- improve tracing spans, add `profile_with_tracy` feature flag
([#394](#394))
- add `profile_with_tracy` feature which plays nicely with bevy's
`bevy/trace_tracy` feature
([#393](#393))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
- add ScriptValue override for printing opaque values
([#380](#380))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))

### Fixed

- fix global type cache not containing generic types
([#388](#388))

### Other

- switch to hashbrown hashmap in the function registry
([#399](#399))
- try play with hashing for access maps
([#398](#398))
- allow check creation for bencher
</blockquote>

## `bevy_mod_scripting_lua`

<blockquote>

##
[0.11.0](bevy_mod_scripting_lua-v0.10.0...bevy_mod_scripting_lua-v0.11.0)
- 2025-03-29

### Added

- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- allow the conversion of lua functions into `ScriptValue` via
`DynamicScriptFunction`
([#396](#396))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
</blockquote>

## `bevy_mod_scripting_rhai`

<blockquote>

##
[0.11.0](bevy_mod_scripting_rhai-v0.10.0...bevy_mod_scripting_rhai-v0.11.0)
- 2025-03-29

### Added

- [**breaking**] bump bersion
- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- Add initial benchmarks, integrate them into CI & add getters/settters
for `Scripts` resource
([#381](#381))
</blockquote>

## `bevy_mod_scripting_functions`

<blockquote>

##
[0.11.0](bevy_mod_scripting_functions-v0.10.0...bevy_mod_scripting_functions-v0.11.0)
- 2025-03-29

### Added

- optimize `get` and `set` functions, add `MagicFunctions` sub-registry
([#397](#397))
- ✨ Dynamic Script Components, `register_new_component`
binding, `remove_component` no longer requires `ReflectComponent` data
([#379](#379))
- overhaul mdbook preprocessor, prettify generated docs, support dummy
globals ([#377](#377))
</blockquote>

## `ladfile_builder`

<blockquote>

##
[0.3.1](v0.3.0-ladfile_builder...v0.3.1-ladfile_builder)
- 2025-03-29

### Added

- bump bevy to 0.15.3
([#401](#401))
</blockquote>

## `bevy_mod_scripting`

<blockquote>

##
[0.11.1](v0.11.0...v0.11.1)
- 2025-03-29

### Added

- bump bevy to 0.15.3
([#401](#401))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This was referenced Mar 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant