-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.Linq.Tests.Perf_Enumerable
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WhereSingleOrDefault_LastElementMatches - Duration of single invocation | 308.59 ns | 336.64 ns | 1.09 | 0.12 | False | |||||
CastToSameType - Duration of single invocation | 532.39 ns | 564.83 ns | 1.06 | 0.03 | False | |||||
Repeat - Duration of single invocation | 429.21 ns | 487.39 ns | 1.14 | 0.05 | False | |||||
Take_All - Duration of single invocation | 1.02 μs | 1.08 μs | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 336.6357383309594 > 326.976847171257.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/2/2022 4:08:26 PM, 10/20/2022 10:19:58 PM, 10/26/2022 7:11:30 AM, 11/2/2022 4:25:40 PM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -12.62536495802817 (T) = (0 -336.63052942852784) / Math.Sqrt((51.97469718404282 / (15)) + (1.8527509673431304 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.07563473129505179 = (312.95989208458207 - 336.63052942852784) / 312.95989208458207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 564.825858199903 > 556.3527116439398.
IsChangePoint: Marked as a change because one of 8/29/2022 8:26:03 AM, 9/1/2022 8:04:48 PM, 9/3/2022 11:44:47 AM, 9/6/2022 1:06:44 PM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -43.0410043309542 (T) = (0 -562.9083087439859) / Math.Sqrt((4.638374511765552 / (15)) + (8.064558983889496 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.058780405247419384 = (531.6572784631801 - 562.9083087439859) / 531.6572784631801 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Repeat
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 487.3857833501984 > 450.4557167560602.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -24.819390298840084 (T) = (0 -484.04228326430786) / Math.Sqrt((0.6686813237093917 / (15)) + (178.33463761779154 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.12751003782193734 = (429.30197251223984 - 484.04228326430786) / 429.30197251223984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.0828432729071418 > 1.062263202248215.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/18/2022 1:35:00 AM, 10/19/2022 7:27:15 PM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -23.482369191455785 (T) = (0 -1066.6106489658964) / Math.Sqrt((7.290387572974124 / (15)) + (179.27539925293974 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.0535563027323582 = (1012.3907438071247 - 1066.6106489658964) / 1012.3907438071247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.Drawing.Tests.Perf_Color
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetSaturation - Duration of single invocation | 924.79 ns | 1.01 μs | 1.09 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Drawing.Tests.Perf_Color*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Drawing.Tests.Perf_Color.GetSaturation
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.0073092595324138 > 970.502148577789.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -21.67834718804766 (T) = (0 -1003.7254593115406) / Math.Sqrt((5.9946985961737616 / (15)) + (491.7756448386303 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.08685580331916276 = (923.5129961548263 - 1003.7254593115406) / 923.5129961548263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in PerfLabTests.CastingPerf2.CastingPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FooObjIsFoo - Duration of single invocation | 773.59 μs | 829.77 μs | 1.07 | 0.08 | False | |||||
ObjrefValueTypeObj - Duration of single invocation | 1.12 ms | 1.36 ms | 1.22 | 0.49 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 829.769692982456 > 819.7253555803572.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -17.075542153843973 (T) = (0 -838425.7613844868) / Math.Sqrt((35092062.96182607 / (15)) + (332964289.68061215 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.07362774972991208 = (780927.8044419083 - 838425.7613844868) / 780927.8044419083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.364290833333333 > 1.145289349330357.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -7.039356165733803 (T) = (0 -1244425.7244832) / Math.Sqrt((4730279244.674516 / (15)) + (8436102045.214048 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.15188496524646405 = (1080338.5425010175 - 1244425.7244832) / 1080338.5425010175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
MultiplyOperatorBenchmark - Duration of single invocation | 52.20 ns | 59.27 ns | 1.14 | 0.05 | False | |||||
MultiplyBenchmark - Duration of single invocation | 52.27 ns | 60.17 ns | 1.15 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyOperatorBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 59.2654905131144 > 54.621966811253735.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -31.537217196588042 (T) = (0 -59.64060252065972) / Math.Sqrt((0.027731181577245644 / (15)) + (2.123463249790367 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.14771383903680838 = (51.96469755101296 - 59.64060252065972) / 51.96469755101296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 60.16902623244489 > 54.52967844130765.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -106.39882955252455 (T) = (0 -59.72506953355831) / Math.Sqrt((0.03508602971588263 / (15)) + (0.11579631449887245 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.15173056695087653 = (51.856806832587694 - 59.72506953355831) / 51.856806832587694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.IO.Tests.StreamReaderReadLineTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ReadLineAsync - Duration of single invocation | 169.01 μs | 183.36 μs | 1.08 | 0.07 | False | |||||
ReadLine - Duration of single invocation | 58.84 μs | 134.42 μs | 2.28 | 0.00 | False | |||||
ReadLineAsync - Duration of single invocation | 342.82 μs | 452.75 μs | 1.32 | 0.42 | False | |||||
ReadLine - Duration of single invocation | 84.92 μs | 125.18 μs | 1.47 | 0.05 | False | |||||
ReadLineAsync - Duration of single invocation | 270.79 μs | 325.51 μs | 1.20 | 0.29 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 8])
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 183.35676431127013 > 178.97808153269636.
IsChangePoint: Marked as a change because one of 10/13/2022 2:25:23 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -14.308614766476499 (T) = (0 -186963.02120726026) / Math.Sqrt((7900448.165571078 / (15)) + (24529823.166047025 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.091077195059571 = (171356.36420029143 - 186963.02120726026) / 171356.36420029143 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0])
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 134.421926510989 > 62.221840566840115.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -430.41169407428737 (T) = (0 -135209.4156405322) / Math.Sqrt((138106.509138594 / (15)) + (812190.6666824007 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -1.2826126884521132 = (59234.49752319589 - 135209.4156405322) / 59234.49752319589 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 0])
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 452.7459642857143 > 361.3089154211956.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -12.71908801535692 (T) = (0 -416373.05169178307) / Math.Sqrt((307561542.202551 / (15)) + (414641983.7579215 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.20775687317148686 = (344749.06410461233 - 416373.05169178307) / 344749.06410461233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 125.18337992221684 > 88.86905368821361.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -95.02451338698683 (T) = (0 -122984.79041514218) / Math.Sqrt((1252749.305061498 / (15)) + (3139809.5339939627 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.4642313716538465 = (83992.73010810518 - 122984.79041514218) / 83992.73010810518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1])
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 325.51014166666664 > 283.75659004237286.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -7.102983727294331 (T) = (0 -312515.2346038728) / Math.Sqrt((364906680.34217656 / (15)) + (104831655.10902599 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.13436653332118337 = (275497.57985974324 - 312515.2346038728) / 275497.57985974324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SquareRootBenchmark - Duration of single invocation | 67.59 ns | 71.69 ns | 1.06 | 0.01 | False | |||||
DivisionOperatorBenchmark - Duration of single invocation | 110.03 ns | 115.71 ns | 1.05 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Byte>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Numerics.Tests.Perf_VectorOf<Byte>.SquareRootBenchmark
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 71.68624091516399 > 70.83734909982604.
IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -22.625137546227794 (T) = (0 -71.93145433025597) / Math.Sqrt((0.03739260935653188 / (15)) + (1.3813129205178203 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.06697420913434024 = (67.41630089504746 - 71.93145433025597) / 67.41630089504746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Byte>.DivisionOperatorBenchmark
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 115.71008634480391 > 115.54162104224558.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -24.556338563803035 (T) = (0 -117.28061751456148) / Math.Sqrt((0.027710631600485064 / (15)) + (3.1090999451705374 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.06536968588621517 = (110.08443272628223 - 117.28061751456148) / 110.08443272628223 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.Collections.TryGetValueTrue<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IDictionary - Duration of single invocation | 5.34 μs | 5.77 μs | 1.08 | 0.39 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Collections.TryGetValueTrue<Int32, Int32>.IDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.770797353271553 > 5.560379370862149.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/23/2022 4:51:48 AM, 10/21/2022 5:53:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -7.739136551917257 (T) = (0 -5954.090909875712) / Math.Sqrt((9627.724735006264 / (15)) + (252894.45736230645 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.12662363122031442 = (5284.89794184991 - 5954.090909875712) / 5284.89794184991 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.Buffers.Text.Tests.Utf8ParserTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryParseInt32 - Duration of single invocation | 3.99 ns | 5.10 ns | 1.28 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 4)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.10072963689795 > 4.190038847096318.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/17/2022 3:19:47 PM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -103.81037070483109 (T) = (0 -5.112352963056487) / Math.Sqrt((0.001651039804835194 / (15)) + (0.0005224402437620371 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.2924710395502749 = (3.9554874396530915 - 5.112352963056487) / 3.9554874396530915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
Diff | Diff |
Regressions in System.MathBenchmarks.Double
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ScaleB - Duration of single invocation | 11.33 μs | 13.49 μs | 1.19 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.MathBenchmarks.Double.ScaleB
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 13.489147433132011 > 11.837373727018745.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsRegressionStdDev: Marked as regression because -24.775178968686163 (T) = (0 -13377.490293430954) / Math.Sqrt((624.3793989111992 / (15)) + (261043.88959726293 / (37))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (37) - 2, .025) and -0.1848597651298392 = (11290.357464340957 - 13377.490293430954) / 11290.357464340957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository