[Perf] Windows/arm64: 3 Improvements on 9/20/2023 11:58:56 PM #22357
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 41a8e3963f2f94bc19565553433a3f48c8af052f |
| Compare | a54ccf39b17586e7c2a38b882e80a3cc62d83d7d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_BigInteger
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.45 μs | 1.36 μs | 0.94 | 0.00 | True | ||||
| 1.36 ms | 1.29 ms | 0.95 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'Payloads
System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 1024,1024 bits)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/20/2023 12:12:52 PM, 9/27/2023 2:51:05 PM falls between 9/13/2023 3:32:54 PM and 9/27/2023 2:51:05 PM.
IsImprovementStdDev: Marked as improvement because 106.63444357170007 (T) = (0 -1360.646672034162) / Math.Sqrt((30.54426416307803 / (299)) + (6.219068434306748 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.06524271743105499 = (1455.6149466894415 - 1360.646672034162) / 1455.6149466894415 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 65536,65536 bits)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/20/2023 12:12:52 PM, 9/27/2023 2:51:05 PM falls between 9/13/2023 3:32:54 PM and 9/27/2023 2:51:05 PM.
IsImprovementStdDev: Marked as improvement because 94.43106645569746 (T) = (0 -1291787.907827942) / Math.Sqrt((33138845.38829703 / (299)) + (4699014.487524241 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.05496121105323883 = (1366915.223942956 - 1291787.907827942) / 1366915.223942956 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 41a8e3963f2f94bc19565553433a3f48c8af052f |
| Compare | a54ccf39b17586e7c2a38b882e80a3cc62d83d7d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.BenchI.TreeInsert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 53.56 μs | 48.17 μs | 0.90 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeInsert*'Payloads
Benchstone.BenchI.TreeInsert.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 9/20/2023 12:12:52 PM, 9/27/2023 2:51:05 PM falls between 9/13/2023 3:32:54 PM and 9/27/2023 2:51:05 PM.
IsImprovementStdDev: Marked as improvement because 86.497149381874 (T) = (0 -48270.22047670709) / Math.Sqrt((212367.00982562377 / (299)) + (22162.391648494042 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09168084454873779 = (53142.356612226184 - 48270.22047670709) / 53142.356612226184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository