-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | 8a2b72e3140b14a2ec915345b2747eecdecff943 |
Compare | 2038514d8dff40dc7add9e89389c856f73125cb2 |
Diff | Diff |
Improvements in PerfLabTests.GetMember
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetMethod5 - Duration of single invocation | 337.18 μs | 282.96 μs | 0.84 | 0.07 | False | |||||
GetMethod4 - Duration of single invocation | 268.05 μs | 229.93 μs | 0.86 | 0.04 | False | |||||
GetMethod12 - Duration of single invocation | 808.31 μs | 681.06 μs | 0.84 | 0.07 | False | |||||
GetMethod15 - Duration of single invocation | 1.01 ms | 852.42 μs | 0.84 | 0.07 | False | |||||
GetMethod10 - Duration of single invocation | 676.40 μs | 590.40 μs | 0.87 | 0.04 | False | |||||
GetMethod2 - Duration of single invocation | 133.33 μs | 114.65 μs | 0.86 | 0.10 | False | |||||
GetMethod3 - Duration of single invocation | 213.62 μs | 173.71 μs | 0.81 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
Payloads
Histogram
PerfLabTests.GetMember.GetMethod5
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 282.96017358630957 < 322.80750397888056.
IsChangePoint: Marked as a change because one of 1/10/2023 11:45:30 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 39.93173928839287 (T) = (0 -288984.3504380423) / Math.Sqrt((25785495.212519288 / (299)) + (39219608.89238492 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14856978590285908 = (339410.4950157097 - 288984.3504380423) / 339410.4950157097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod4
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 229.93329999999997 < 258.3684634123563.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 33.63955673950371 (T) = (0 -230260.22738672508) / Math.Sqrt((16599662.611940304 / (299)) + (38055787.02193834 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.15025749243557995 = (270976.4726807771 - 230260.22738672508) / 270976.4726807771 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod12
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 681.0597710597827 < 767.419036875.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 30.793680041163977 (T) = (0 -692189.9606099102) / Math.Sqrt((119167329.38175933 / (299)) + (400317583.8767646 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14791137406310226 = (812345.0302470896 - 692189.9606099102) / 812345.0302470896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod15
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 852.4233414351853 < 0.9603071370798277.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 36.8137233599367 (T) = (0 -863032.387355685) / Math.Sqrt((379034492.3654142 / (299)) + (430880438.14668614 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.15041299615285056 = (1015825.7876446454 - 863032.387355685) / 1015825.7876446454 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod10
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 590.3991634837963 < 642.8001487165178.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.73536133492427 (T) = (0 -574403.1719167222) / Math.Sqrt((83803956.90242977 / (299)) + (228594503.65992257 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1517124701355716 = (677132.6368649108 - 574403.1719167222) / 677132.6368649108 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.65384163533835 < 126.94810691976895.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 31.8361311584326 (T) = (0 -114157.19069365691) / Math.Sqrt((7280751.557105385 / (299)) + (10995385.760398693 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1548420749169923 = (135072.02299788516 - 114157.19069365691) / 135072.02299788516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod3
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.7099157706093 < 191.00305519097222.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 34.67769288758244 (T) = (0 -172735.33068198225) / Math.Sqrt((12821908.683478957 / (299)) + (19311322.218814798 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14879001525727842 = (202929.16410536648 - 172735.33068198225) / 202929.16410536648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | d87f7b2b81647917528286d91ed91db2a34b0985 |
Diff | Diff |
Improvements in System.Collections.TryGetValueFalse<String, String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ImmutableDictionary - Duration of single invocation | 29.29 μs | 26.40 μs | 0.90 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*'
Payloads
Histogram
System.Collections.TryGetValueFalse<String, String>.ImmutableDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.401831864811694 < 27.997907651297957.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 17.50591089413123 (T) = (0 -26627.246989274125) / Math.Sqrt((392821.2902807946 / (299)) + (233599.73995820404 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.061589089427357586 = (28374.82673024922 - 26627.246989274125) / 28374.82673024922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | 71bb0d481086b8b8a89a99a17ec80a861f32dc5d |
Compare | d87f7b2b81647917528286d91ed91db2a34b0985 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
DivisionOperatorBenchmark - Duration of single invocation | 38.26 ns | 27.82 ns | 0.73 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int16>*'
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<Int16>.DivisionOperatorBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.81631504960993 < 36.34201650013176.
IsChangePoint: Marked as a change because one of 11/29/2022 11:47:49 AM, 12/14/2022 5:20:21 PM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 53.49675945670231 (T) = (0 -27.83507677214827) / Math.Sqrt((9.972241150944487 / (299)) + (0.02454460678083089 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.2623966036267925 = (37.73718628332409 - 27.83507677214827) / 37.73718628332409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | alpine 3.15 |
Baseline | 8a2b72e3140b14a2ec915345b2747eecdecff943 |
Compare | d87f7b2b81647917528286d91ed91db2a34b0985 |
Diff | Diff |
Improvements in LinqBenchmarks
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count00ForX - Duration of single invocation | 102.49 ms | 92.78 ms | 0.91 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
Payloads
Histogram
LinqBenchmarks.Count00ForX
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.776992875 < 97.43527890124999.
IsChangePoint: Marked as a change because one of 11/4/2022 9:11:54 AM, 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsImprovementStdDev: Marked as improvement because 33.21471237125669 (T) = (0 -91129389.61296298) / Math.Sqrt((8037136133906.638 / (299)) + (1052521242897.7266 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08553748234368011 = (99653499.02642144 - 91129389.61296298) / 99653499.02642144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository