Skip to content

[Perf] Windows/x86: 47 Improvements on 3/15/2023 12:03:45 AM #14244

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 72.25 ns 66.07 ns 0.91 0.07 False
Enumerate - Duration of single invocation 5.33 μs 5.03 μs 0.94 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 10)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.06929891533278 < 68.84244626331457.
IsChangePoint: Marked as a change because one of 2/16/2023 6:27:39 AM, 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 14.816346945622954 (T) = (0 -69.68331874655081) / Math.Sqrt((8.801395619267405 / (299)) + (4.074190058635158 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.07857734980304833 = (75.62579314894873 - 69.68331874655081) / 75.62579314894873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;.Enumerate(Size: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.0253943045536005 < 5.058775692078364.
IsChangePoint: Marked as a change because one of 2/16/2023 6:27:39 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 17.35446849669152 (T) = (0 -4848.607041602536) / Math.Sqrt((36955.93201570946 / (299)) + (59792.517214156236 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.139527882649319 = (5634.821795889187 - 4848.607041602536) / 5634.821795889187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Improvements in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MemoryFromArrayStartLength - Duration of single invocation 10.88 ns 7.46 ns 0.69 0.32 False
ArrayAsSpan - Duration of single invocation 10.87 ns 7.90 ns 0.73 0.34 False
ArrayAsSpanStartLength - Duration of single invocation 11.88 ns 9.27 ns 0.78 0.39 False
SpanImplicitCastFromArray - Duration of single invocation 6.85 ns 4.02 ns 0.59 0.32 False
ArrayAsMemory - Duration of single invocation 12.87 ns 9.75 ns 0.76 0.33 False
ArrayAsMemoryStartLength - Duration of single invocation 13.36 ns 10.39 ns 0.78 0.36 False
MemoryFromArray - Duration of single invocation 10.18 ns 6.87 ns 0.68 0.27 False
SpanFromArrayStartLength - Duration of single invocation 8.27 ns 5.88 ns 0.71 0.34 False
SpanImplicitCastFromArraySegment - Duration of single invocation 10.17 ns 5.31 ns 0.52 0.28 False
ReadOnlySpanImplicitCastFromArraySegment - Duration of single invocation 5.74 ns 3.33 ns 0.58 0.26 False
SpanFromArray - Duration of single invocation 8.14 ns 5.88 ns 0.72 0.31 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Constructors<String>.MemoryFromArrayStartLength


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.464658577447053 < 10.337321741863775.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 36.92739728711494 (T) = (0 -7.602072535953529) / Math.Sqrt((0.2923615397566555 / (299)) + (0.22017707412426932 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.30393455473774095 = (10.921491057625005 - 7.602072535953529) / 10.921491057625005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.ArrayAsSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.898758587034207 < 10.351093967920242.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 25.17592889800624 (T) = (0 -8.16245171571428) / Math.Sqrt((0.32604841472831825 / (299)) + (0.36892164888450946 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.260109545255561 = (11.031973264925579 - 8.16245171571428) / 11.031973264925579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.ArrayAsSpanStartLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.270090385494395 < 10.989494175691846.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 35.07790990073428 (T) = (0 -9.079952484312967) / Math.Sqrt((0.4900146499287883 / (299)) + (0.11944038991206501 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.2225702422139301 = (11.679450642808495 - 9.079952484312967) / 11.679450642808495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.SpanImplicitCastFromArray

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.0225717782844415 < 6.5090639968050406.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 41.07951408991123 (T) = (0 -4.200835425211485) / Math.Sqrt((0.16787900516095886 / (299)) + (0.13376519671412207 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.40577957817408455 = (7.06949016040747 - 4.200835425211485) / 7.06949016040747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.ArrayAsMemory

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.745473847253837 < 12.23429965254068.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 30.38014002524561 (T) = (0 -9.844102117476568) / Math.Sqrt((0.35435595866784014 / (299)) + (0.34465046723428816 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.2550146080815589 = (13.21381898794906 - 9.844102117476568) / 13.21381898794906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.ArrayAsMemoryStartLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.392449949976797 < 12.673022751033132.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 31.848761401556086 (T) = (0 -10.715938098792579) / Math.Sqrt((0.4664348905101382 / (299)) + (0.23297598297068073 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.2206588475915116 = (13.749996475453493 - 10.715938098792579) / 13.749996475453493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.MemoryFromArray

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.872910297683812 < 9.679421967303684.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 48.63203574600667 (T) = (0 -6.87468138466332) / Math.Sqrt((0.24382977783241452 / (299)) + (0.13871717569401124 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.33972736162765954 = (10.411882887666405 - 6.87468138466332) / 10.411882887666405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.SpanFromArrayStartLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.8816045389672915 < 7.85937489084528.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 40.38522906366058 (T) = (0 -5.988089406546664) / Math.Sqrt((0.1728985087891855 / (299)) + (0.09869728885578552 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.29261748089792855 = (8.465136252091375 - 5.988089406546664) / 8.465136252091375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.SpanImplicitCastFromArraySegment

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.3139719323614845 < 9.301680708704053.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 59.87167744058189 (T) = (0 -5.451567591197716) / Math.Sqrt((0.16479436715185267 / (299)) + (0.14887674947170224 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.4455443349349226 = (9.832287655601565 - 5.451567591197716) / 9.832287655601565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.ReadOnlySpanImplicitCastFromArraySegment

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.329280600954462 < 5.49883008095359.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 59.66632992297852 (T) = (0 -3.4021030725009975) / Math.Sqrt((0.08464137917711921 / (299)) + (0.04532110945848856 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.4228435433866792 = (5.894594149503407 - 3.4021030725009975) / 5.894594149503407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;String&gt;.SpanFromArray

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.87524135818257 < 7.731878194410816.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 48.7514885444022 (T) = (0 -5.45927150698699) / Math.Sqrt((0.18148643794486313 / (299)) + (0.06492352898639593 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.31699993385537906 = (7.99307610291069 - 5.45927150698699) / 7.99307610291069 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 600.63 μs 548.80 μs 0.91 0.02 False
Queue - Duration of single invocation 599.34 μs 547.07 μs 0.91 0.04 False
Stack - Duration of single invocation 606.28 μs 553.61 μs 0.91 0.02 False
ImmutableArray - Duration of single invocation 612.22 μs 560.58 μs 0.92 0.02 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.ICollection(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 548.8040178571429 < 568.7767723765432.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 100.40448017994339 (T) = (0 -548469.2505345277) / Math.Sqrt((50433023.39002918 / (299)) + (3605118.281068519 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08902353517653215 = (602067.4207437532 - 548469.2505345277) / 602067.4207437532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 547.0740550397878 < 568.9058321519559.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 9.067303240632219 (T) = (0 -568414.9260795423) / Math.Sqrt((7035695.13831966 / (299)) + (387197959.8516066 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.053415400538165264 = (600490.3591318782 - 568414.9260795423) / 600490.3591318782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 553.6101447044334 < 572.9014137477106.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 94.28035527367236 (T) = (0 -554356.6666959524) / Math.Sqrt((47141437.50701512 / (299)) + (4044002.686204691 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08365186882982413 = (604962.947856989 - 554356.6666959524) / 604962.947856989 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 560.5840104166666 < 578.9484778044872.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 103.27022211178452 (T) = (0 -555224.6361895545) / Math.Sqrt((45562216.95799994 / (299)) + (3887734.1202031365 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08927954095791732 = (609654.2914755126 - 555224.6361895545) / 609654.2914755126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 3.08 μs 2.33 μs 0.76 0.14 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.IList(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3293441899219025 < 2.923672335386921.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 77.29892748117976 (T) = (0 -2336.3416574794396) / Math.Sqrt((12316.769956228858 / (299)) + (859.0598486089181 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.21546402766821934 = (2977.991755477338 - 2336.3416574794396) / 2977.991755477338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions