-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | x86 |
OS | Windows 10.0.18362 |
Queue | TigerWindows |
Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
Diff | Diff |
Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Enumerate - Duration of single invocation | 72.25 ns | 66.07 ns | 0.91 | 0.07 | False | |||||
Enumerate - Duration of single invocation | 5.33 μs | 5.03 μs | 0.94 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<String, String>*'
Payloads
Histogram
System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 10)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.06929891533278 < 68.84244626331457.
IsChangePoint: Marked as a change because one of 2/16/2023 6:27:39 AM, 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 14.816346945622954 (T) = (0 -69.68331874655081) / Math.Sqrt((8.801395619267405 / (299)) + (4.074190058635158 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.07857734980304833 = (75.62579314894873 - 69.68331874655081) / 75.62579314894873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 1000)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.0253943045536005 < 5.058775692078364.
IsChangePoint: Marked as a change because one of 2/16/2023 6:27:39 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 17.35446849669152 (T) = (0 -4848.607041602536) / Math.Sqrt((36955.93201570946 / (299)) + (59792.517214156236 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.139527882649319 = (5634.821795889187 - 4848.607041602536) / 5634.821795889187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x86 |
OS | Windows 10.0.18362 |
Queue | TigerWindows |
Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
Diff | Diff |
Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Memory.Constructors<String>
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors<String>*'
Payloads
Histogram
System.Memory.Constructors<String>.MemoryFromArrayStartLength
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.464658577447053 < 10.337321741863775.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 36.92739728711494 (T) = (0 -7.602072535953529) / Math.Sqrt((0.2923615397566555 / (299)) + (0.22017707412426932 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.30393455473774095 = (10.921491057625005 - 7.602072535953529) / 10.921491057625005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.898758587034207 < 10.351093967920242.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 25.17592889800624 (T) = (0 -8.16245171571428) / Math.Sqrt((0.32604841472831825 / (299)) + (0.36892164888450946 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.260109545255561 = (11.031973264925579 - 8.16245171571428) / 11.031973264925579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.270090385494395 < 10.989494175691846.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 35.07790990073428 (T) = (0 -9.079952484312967) / Math.Sqrt((0.4900146499287883 / (299)) + (0.11944038991206501 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.2225702422139301 = (11.679450642808495 - 9.079952484312967) / 11.679450642808495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.0225717782844415 < 6.5090639968050406.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 41.07951408991123 (T) = (0 -4.200835425211485) / Math.Sqrt((0.16787900516095886 / (299)) + (0.13376519671412207 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.40577957817408455 = (7.06949016040747 - 4.200835425211485) / 7.06949016040747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.745473847253837 < 12.23429965254068.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 30.38014002524561 (T) = (0 -9.844102117476568) / Math.Sqrt((0.35435595866784014 / (299)) + (0.34465046723428816 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.2550146080815589 = (13.21381898794906 - 9.844102117476568) / 13.21381898794906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemoryStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.392449949976797 < 12.673022751033132.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 31.848761401556086 (T) = (0 -10.715938098792579) / Math.Sqrt((0.4664348905101382 / (299)) + (0.23297598297068073 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.2206588475915116 = (13.749996475453493 - 10.715938098792579) / 13.749996475453493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.872910297683812 < 9.679421967303684.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 48.63203574600667 (T) = (0 -6.87468138466332) / Math.Sqrt((0.24382977783241452 / (299)) + (0.13871717569401124 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.33972736162765954 = (10.411882887666405 - 6.87468138466332) / 10.411882887666405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.8816045389672915 < 7.85937489084528.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 40.38522906366058 (T) = (0 -5.988089406546664) / Math.Sqrt((0.1728985087891855 / (299)) + (0.09869728885578552 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.29261748089792855 = (8.465136252091375 - 5.988089406546664) / 8.465136252091375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.3139719323614845 < 9.301680708704053.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 59.87167744058189 (T) = (0 -5.451567591197716) / Math.Sqrt((0.16479436715185267 / (299)) + (0.14887674947170224 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.4455443349349226 = (9.832287655601565 - 5.451567591197716) / 9.832287655601565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.329280600954462 < 5.49883008095359.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 59.66632992297852 (T) = (0 -3.4021030725009975) / Math.Sqrt((0.08464137917711921 / (299)) + (0.04532110945848856 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.4228435433866792 = (5.894594149503407 - 3.4021030725009975) / 5.894594149503407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.87524135818257 < 7.731878194410816.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 48.7514885444022 (T) = (0 -5.45927150698699) / Math.Sqrt((0.18148643794486313 / (299)) + (0.06492352898639593 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.31699993385537906 = (7.99307610291069 - 5.45927150698699) / 7.99307610291069 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x86 |
OS | Windows 10.0.18362 |
Queue | TigerWindows |
Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
Diff | Diff |
Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrue<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ICollection - Duration of single invocation | 600.63 μs | 548.80 μs | 0.91 | 0.02 | False | |||||
Queue - Duration of single invocation | 599.34 μs | 547.07 μs | 0.91 | 0.04 | False | |||||
Stack - Duration of single invocation | 606.28 μs | 553.61 μs | 0.91 | 0.02 | False | |||||
ImmutableArray - Duration of single invocation | 612.22 μs | 560.58 μs | 0.92 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'
Payloads
Histogram
System.Collections.ContainsTrue<String>.ICollection(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 548.8040178571429 < 568.7767723765432.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 100.40448017994339 (T) = (0 -548469.2505345277) / Math.Sqrt((50433023.39002918 / (299)) + (3605118.281068519 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08902353517653215 = (602067.4207437532 - 548469.2505345277) / 602067.4207437532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsTrue<String>.Queue(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 547.0740550397878 < 568.9058321519559.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 9.067303240632219 (T) = (0 -568414.9260795423) / Math.Sqrt((7035695.13831966 / (299)) + (387197959.8516066 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.053415400538165264 = (600490.3591318782 - 568414.9260795423) / 600490.3591318782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsTrue<String>.Stack(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 553.6101447044334 < 572.9014137477106.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 94.28035527367236 (T) = (0 -554356.6666959524) / Math.Sqrt((47141437.50701512 / (299)) + (4044002.686204691 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08365186882982413 = (604962.947856989 - 554356.6666959524) / 604962.947856989 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 560.5840104166666 < 578.9484778044872.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 103.27022211178452 (T) = (0 -555224.6361895545) / Math.Sqrt((45562216.95799994 / (299)) + (3887734.1202031365 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08927954095791732 = (609654.2914755126 - 555224.6361895545) / 609654.2914755126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x86 |
OS | Windows 10.0.18362 |
Queue | TigerWindows |
Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
Diff | Diff |
Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Collections.IterateFor<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IList - Duration of single invocation | 3.08 μs | 2.33 μs | 0.76 | 0.14 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor<String>*'
Payloads
Histogram
System.Collections.IterateFor<String>.IList(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3293441899219025 < 2.923672335386921.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 77.29892748117976 (T) = (0 -2336.3416574794396) / Math.Sqrt((12316.769956228858 / (299)) + (859.0598486089181 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.21546402766821934 = (2977.991755477338 - 2336.3416574794396) / 2977.991755477338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository