[Perf] Windows/x64: 2 Regressions on 12/2/2022 2:38:36 AM #10384
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 41f57b7862dca221687a6cb638c35e33ee8e25db |
| Compare | 09613f3ed6cb5ce62e955d2a1979115879d707bb |
| Diff | Diff |
Regressions in System.Tests.Perf_Int32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToString - Duration of single invocation | 1.92 ns | 2.94 ns | 1.53 | 0.17 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'Payloads
Histogram
System.Tests.Perf_Int32.ToString(value: 4)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9411093443688876 > 2.0611424495873205.
IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 11/18/2022 8:48:12 PM, 12/1/2022 11:48:13 PM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -68.69648151935434 (T) = (0 -3.0636329343163826) / Math.Sqrt((0.005975952696769367 / (34)) + (0.001444398381566148 / (18))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (18) - 2, .025) and -0.5595191681443235 = (1.9644727662833465 - 3.0636329343163826) / 1.9644727662833465 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 41f57b7862dca221687a6cb638c35e33ee8e25db |
| Compare | 09613f3ed6cb5ce62e955d2a1979115879d707bb |
| Diff | Diff |
Regressions in System.Tests.Perf_UInt64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToString - Duration of single invocation | 1.95 ns | 3.34 ns | 1.71 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'Payloads
Histogram
System.Tests.Perf_UInt64.ToString(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3409850839247968 > 2.0470592227640685.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 12/1/2022 11:48:13 PM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -46.59807429323151 (T) = (0 -4.135817867368059) / Math.Sqrt((9.950761052123618E-05 / (34)) + (0.039452873324174964 / (18))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (18) - 2, .025) and -1.1179102668290597 = (1.9527823874994548 - 4.135817867368059) / 1.9527823874994548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository