Skip to content

[Perf] Linux/arm64: 79 Improvements on 4/26/2023 12:55:36 AM #17402

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 62.05 ns 55.40 ns 0.89 0.00 True
StartsWith - Duration of single invocation 31.44 ns 27.87 ns 0.89 0.02 False
EndsWith - Duration of single invocation 31.21 ns 28.78 ns 0.92 0.03 False
SequenceCompareTo - Duration of single invocation 8.04 ns 6.76 ns 0.84 0.06 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.SequenceEqual(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.398169266783306 < 59.174361613352545.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 85.16435726760126 (T) = (0 -54.53853341213208) / Math.Sqrt((0.4122321390792662 / (299)) + (0.1639987872153804 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12041784788327697 = (62.00504782968205 - 54.53853341213208) / 62.00504782968205 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.StartsWith(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.866047561099833 < 29.901031561446338.
IsChangePoint: Marked as a change because one of 1/18/2023 3:22:13 AM, 2/9/2023 1:41:04 PM, 4/6/2023 11:23:56 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 34.49404035088565 (T) = (0 -28.362601310738235) / Math.Sqrt((0.3030202934695391 / (299)) + (0.09192967450304583 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07581006618607644 = (30.689147623251177 - 28.362601310738235) / 30.689147623251177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.EndsWith(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.78483216169964 < 30.069914191240173.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/12/2023 6:21:10 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 30.169069018168457 (T) = (0 -28.721316700869338) / Math.Sqrt((0.48226247589147087 / (299)) + (0.14534914678809002 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08185300364490601 = (31.281828307328414 - 28.721316700869338) / 31.281828307328414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareTo(Size: 33)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.756787090746246 < 7.637393351393344.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/3/2023 4:19:26 AM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 3.1056825312679637 (T) = (0 -6.418675658720651) / Math.Sqrt((0.5331228313696631 / (299)) + (0.7435400398286169 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.09607530335010393 = (7.100896438065463 - 6.418675658720651) / 7.100896438065463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 6.56 μs 4.62 μs 0.70 0.05 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.617858292312526 < 6.282917761154753.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 126.75146662466908 (T) = (0 -4743.175428865346) / Math.Sqrt((50078.98933926697 / (299)) + (1488.4809656370242 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2860252891242838 = (6643.338141553595 - 4743.175428865346) / 6643.338141553595 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 10.15 ns 8.80 ns 0.87 0.06 False
TryParseHex - Duration of single invocation 5.21 ns 3.49 ns 0.67 0.38 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "3039")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.804927621511748 < 9.616312257044731.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/18/2023 2:18:01 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 51.35529418286189 (T) = (0 -8.620429400086374) / Math.Sqrt((0.20441759384668773 / (299)) + (0.05249249668415315 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.23646929824940052 = (11.290219738802541 - 8.620429400086374) / 11.290219738802541 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "0")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.4901924579388233 < 5.161276260628427.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 69.57571002101743 (T) = (0 -3.363165905414924) / Math.Sqrt((0.16024279481020443 / (299)) + (0.01093855243156166 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.39019144008881984 = (5.515117573791971 - 3.363165905414924) / 5.515117573791971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 623.10 ns 560.17 ns 0.90 0.00 True
LastIndexOf_Word_NotFound - Duration of single invocation 622.32 ns 561.06 ns 0.90 0.00 True
LastIndexOf_Word_NotFound - Duration of single invocation 622.54 ns 560.03 ns 0.90 0.01 True
LastIndexOf_Word_NotFound - Duration of single invocation 997.00 ns 801.37 ns 0.80 0.00 True
IndexOf_Word_NotFound - Duration of single invocation 629.88 ns 561.28 ns 0.89 0.01 True
IndexOf_Word_NotFound - Duration of single invocation 630.36 ns 558.52 ns 0.89 0.00 True
IndexOf_Word_NotFound - Duration of single invocation 631.05 ns 560.84 ns 0.89 0.00 True
IndexOf_Word_NotFound - Duration of single invocation 978.00 ns 812.63 ns 0.83 0.01 True
LastIndexOf_Word_NotFound - Duration of single invocation 994.03 ns 799.39 ns 0.80 0.00 True
IsSuffix_DifferentLastChar - Duration of single invocation 32.33 ns 29.64 ns 0.92 0.03 False
IndexOf_Word_NotFound - Duration of single invocation 978.49 ns 812.86 ns 0.83 0.00 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 560.1702058741884 < 591.9553184834358.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 432.3102412709109 (T) = (0 -560.1189326623872) / Math.Sqrt((3.3882555723485064 / (299)) + (0.26633817632971674 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10182597067670396 = (623.619604192289 - 560.1189326623872) / 623.619604192289 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 561.0553082129635 < 591.9260754269544.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 458.44721100014027 (T) = (0 -560.1417549986801) / Math.Sqrt((2.097931095122009 / (299)) + (0.31418773514994447 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10161919730214593 = (623.5014743375683 - 560.1417549986801) / 623.5014743375683 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 560.0284313370724 < 591.3386391123387.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 47.316453682157025 (T) = (0 -561.7641494528842) / Math.Sqrt((1.6620068541949131 / (299)) + (45.29919191399933 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10019723285964711 = (624.319206350317 - 561.7641494528842) / 624.319206350317 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 801.3707604910176 < 946.4875267188866.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 95.06352102342792 (T) = (0 -802.890709894433) / Math.Sqrt((98.35288776415791 / (299)) + (101.68545582543862 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19601313354350283 = (998.6365989199608 - 802.890709894433) / 998.6365989199608 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 561.2760293336712 < 598.6726842956258.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 56.43543377677947 (T) = (0 -561.8355676163272) / Math.Sqrt((3.067647567245063 / (299)) + (38.14005969322872 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10880142322654862 = (630.4269129899537 - 561.8355676163272) / 630.4269129899537 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 558.5247962308201 < 598.8831670367961.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 431.1812020243423 (T) = (0 -560.1239990775168) / Math.Sqrt((2.549714317952406 / (299)) + (0.48268709112103736 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11245745329697061 = (631.0953780843744 - 560.1239990775168) / 631.0953780843744 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 560.8404666002897 < 598.6711149153454.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 189.0153900769059 (T) = (0 -561.046493064931) / Math.Sqrt((3.2880491914222154 / (299)) + (3.212237277365591 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10998387466755176 = (630.3778966424491 - 561.046493064931) / 630.3778966424491 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 812.6305518064785 < 930.0821570736044.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 64.96503065029812 (T) = (0 -814.0476751698677) / Math.Sqrt((75.14136434811185 / (299)) + (168.2105250388471 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1714258598050678 = (982.4681168282093 - 814.0476751698677) / 982.4681168282093 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 799.3894580208604 < 948.6983088989396.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 173.99101743177312 (T) = (0 -798.5101781355077) / Math.Sqrt((107.78070528121486 / (299)) + (25.090298380718046 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20055071599758575 = (998.8253090149697 - 798.5101781355077) / 998.8253090149697 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.636833067685913 < 30.701894042011457.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 25.91177786734086 (T) = (0 -29.026173064383293) / Math.Sqrt((0.9685701534905444 / (299)) + (0.21098273983481566 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08686019365344266 = (31.78721687812085 - 29.026173064383293) / 31.78721687812085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False))


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 812.8583326837127 < 929.9904607482273.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 116.52418879826712 (T) = (0 -812.1715310956145) / Math.Sqrt((73.77114370968978 / (299)) + (49.28079835037716 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17354483973785198 = (982.7169943956756 - 812.1715310956145) / 982.7169943956756 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SelectToList - Duration of single invocation 577.39 ns 464.48 ns 0.80 0.47 False
SelectToArray - Duration of single invocation 347.44 ns 299.00 ns 0.86 0.31 False
SelectToList - Duration of single invocation 461.18 ns 317.85 ns 0.69 0.35 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 464.4830439050207 < 551.8110008502285.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 10.85061498012346 (T) = (0 -501.04028233209766) / Math.Sqrt((1425.3683662752633 / (299)) + (2164.4095298882335 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16886132398135298 = (602.8359608196798 - 501.04028233209766) / 602.8359608196798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 298.9978070635087 < 331.17032138190405.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 6.216507154178892 (T) = (0 -312.0545381220496) / Math.Sqrt((2510.128435524678 / (299)) + (2064.4115462820555 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1572988981848869 = (370.30275319435117 - 312.0545381220496) / 370.30275319435117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 317.85269350276656 < 436.68064611304965.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 24.03202802545499 (T) = (0 -314.14727853112134) / Math.Sqrt((1883.483152770094 / (299)) + (737.5700925862083 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.32558442887611994 = (465.8066806014139 - 314.14727853112134) / 465.8066806014139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 29.92 μs 27.96 μs 0.93 0.00 True
Dictionary - Duration of single invocation 6.26 μs 4.50 μs 0.72 0.05 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.964219335122245 < 28.463561547333967.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 98.65194564468185 (T) = (0 -28117.878817687717) / Math.Sqrt((28488.508830799183 / (299)) + (6676.495436520614 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06176570742289892 = (29968.931044350076 - 28117.878817687717) / 29968.931044350076 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.495619232634845 < 5.908833589567411.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/4/2023 10:11:10 PM, 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsImprovementStdDev: Marked as improvement because 38.273551146140264 (T) = (0 -4625.465561142535) / Math.Sqrt((115856.92537259196 / (299)) + (2169.8403735026122 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15223071251511078 = (5456.04285201825 - 4625.465561142535) / 5456.04285201825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions