Skip to content

[Perf] Windows/arm64: 51 Improvements on 6/29/2023 8:27:03 AM #19583

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
LinqOrderByExtension - Duration of single invocation 41.34 μs 26.27 μs 0.64 0.08 False
Array_ComparerClass - Duration of single invocation 33.94 μs 18.43 μs 0.54 0.12 False
Array - Duration of single invocation 27.24 μs 9.37 μs 0.34 0.01 True
Array_Comparison - Duration of single invocation 35.13 μs 21.72 μs 0.62 0.03 False
List - Duration of single invocation 26.94 μs 9.01 μs 0.33 0.01 True
LinqQuery - Duration of single invocation 40.21 μs 26.39 μs 0.66 0.09 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'

Payloads

Baseline
Compare

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.26709714285714 < 39.833389483333335.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 89.75131527965284 (T) = (0 -26563.17750746689) / Math.Sqrt((19517240.021517254 / (299)) + (238952.58444156343 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4942491974898899 = (52522.26467191001 - 26563.17750746689) / 52522.26467191001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.431440000000002 < 32.234971821428566.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 56.98358697688732 (T) = (0 -17533.402395040852) / Math.Sqrt((18933216.064761925 / (299)) + (1240067.7432906446 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5642238287023823 = (40234.88100056356 - 17533.402395040852) / 40234.88100056356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array(Size: 512)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.369751999999998 < 25.8387912032967.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 396.5060045737119 (T) = (0 -9324.466074387152) / Math.Sqrt((135867.64590530362 / (299)) + (21497.651998935347 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6612923817122311 = (27529.543390621377 - 9324.466074387152) / 27529.543390621377 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.723868000000003 < 33.370398554487174.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 60.90721966612799 (T) = (0 -20498.93489912652) / Math.Sqrt((10904056.824100025 / (299)) + (858985.2550601526 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4875646005691295 = (40002.96412366005 - 20498.93489912652) / 40002.96412366005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.List(Size: 512)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.012182857142857 < 25.596059678571425.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 622.7407766390801 (T) = (0 -9006.411293650794) / Math.Sqrt((173530.00652023221 / (299)) + (3505.0383369907413 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.6713050626106389 = (27400.517224827534 - 9006.411293650794) / 27400.517224827534 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.387489230769237 < 39.5133699047619.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 99.88204447051368 (T) = (0 -26339.39125528317) / Math.Sqrt((20273865.019542947 / (299)) + (8668.174462197554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.49806544124711166 = (52475.747676602085 - 26339.39125528317) / 52475.747676602085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
CreateFromScalars - Duration of single invocation 2.81 ns 0.98 ns 0.35 0.56 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix3x2.CreateFromScalars

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9754693838340621 < 2.6503504628132006.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 48.046016389519636 (T) = (0 -1.0183994824155538) / Math.Sqrt((0.0763748358044497 / (299)) + (0.01380065022355191 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6312856341055744 = (2.762028216462695 - 1.0183994824155538) / 2.762028216462695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
InterpolateIntoSpan_Flags - Duration of single invocation 104.13 ns 98.42 ns 0.95 0.01 False
InterpolateIntoSpan_Flags - Duration of single invocation 107.03 ns 101.11 ns 0.94 0.01 False
InterpolateIntoSpan_NonFlags - Duration of single invocation 115.90 ns 108.35 ns 0.93 0.01 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 98.41798350571605 < 99.28318037216505.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 43.54347480750639 (T) = (0 -98.18687078209283) / Math.Sqrt((6.493872148777041 / (299)) + (0.033996887791657826 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06470294241787697 = (104.97934317886123 - 98.18687078209283) / 104.97934317886123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 101.11337699934695 < 102.17714822436142.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 36.95355889879681 (T) = (0 -101.44614728034477) / Math.Sqrt((11.577506730308407 / (299)) + (0.025827826449929824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06846332124548722 = (108.90193547287987 - 101.44614728034477) / 108.90193547287987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 108.3509934076004 < 110.29474608505623.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 34.295799530776115 (T) = (0 -109.37662401161118) / Math.Sqrt((11.543642502622902 / (299)) + (0.15770647160518428 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06597010127206568 = (117.1018445561244 - 109.37662401161118) / 117.1018445561244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
ReadStructAndReverseBE - Duration of single invocation 10.66 ns 2.31 ns 0.22 0.29 True
ReadStructFieldByFieldBE - Duration of single invocation 8.01 ns 5.17 ns 0.65 0.43 False
ReadStructFieldByFieldUsingBitConverterBE - Duration of single invocation 17.70 ns 1.89 ns 0.11 0.27 True
ReadStructFieldByFieldUsingBitConverterLE - Duration of single invocation 8.56 ns 1.83 ns 0.21 0.29 True
ReadStructFieldByFieldLE - Duration of single invocation 8.45 ns 4.92 ns 0.58 0.40 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.312754603627724 < 10.30938151564364.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 145.8343661594201 (T) = (0 -2.1533839438226283) / Math.Sqrt((0.3082248656647507 / (299)) + (0.032977762500796017 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8017869319489568 = (10.863985735128702 - 2.1533839438226283) / 10.863985735128702 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.172164245984457 < 8.002706461427184.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 43.89746022183775 (T) = (0 -5.4823373038374195) / Math.Sqrt((0.5787472320692092 / (299)) + (0.04135782872158954 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36417927133319483 = (8.622457646721955 - 5.4823373038374195) / 8.622457646721955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8946947351082222 < 16.622688729792365.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 168.58303008476727 (T) = (0 -1.767253582880384) / Math.Sqrt((0.8576843283366887 / (299)) + (0.07654852749819531 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.899261210291919 = (17.542930464039703 - 1.767253582880384) / 17.542930464039703 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8282319193156682 < 8.081330273244793.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 120.81517172644624 (T) = (0 -1.8209277687970455) / Math.Sqrt((0.26312912574431613 / (299)) + (0.025671723287621774 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.784748700861126 = (8.459543687224093 - 1.8209277687970455) / 8.459543687224093 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.922566858589282 < 8.150515208272855.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 53.411450446055305 (T) = (0 -5.338215025746046) / Math.Sqrt((0.18780330330053804 / (299)) + (0.03503415281139085 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36579390958879504 = (8.417161402983165 - 5.338215025746046) / 8.417161402983165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
IterateTryGetMemory - Duration of single invocation 17.11 ns 13.96 ns 0.82 0.24 False
IterateForEachArray - Duration of single invocation 16.43 ns 4.89 ns 0.30 0.36 True
IterateForEachSingleSegment - Duration of single invocation 21.68 ns 9.38 ns 0.43 0.30 True
IterateForEachMemory - Duration of single invocation 27.84 ns 13.49 ns 0.48 0.17 True
IterateGetPositionMemory - Duration of single invocation 29.24 ns 25.44 ns 0.87 0.22 False
IterateForEachTenSegments - Duration of single invocation 68.37 ns 42.52 ns 0.62 0.02 True

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetMemory

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.96257742760995 < 16.315278464226775.
IsChangePoint: Marked as a change because one of 3/3/2023 4:19:26 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 45.01471080749216 (T) = (0 -13.775694580559946) / Math.Sqrt((2.2315571540211305 / (299)) + (0.012816646751738096 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2309859404826928 = (17.913449578810873 - 13.775694580559946) / 17.913449578810873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachArray

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.888441455507997 < 15.619858550934177.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 142.21910030965572 (T) = (0 -4.804568820779855) / Math.Sqrt((0.6013143377002923 / (299)) + (0.07418062507146722 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7222510817665462 = (17.298244944887653 - 4.804568820779855) / 17.298244944887653 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.377807913782599 < 20.603843577420903.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 272.9118387060244 (T) = (0 -9.440652788158964) / Math.Sqrt((0.47008057741447407 / (299)) + (0.008008612660880158 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.574880266643423 = (22.207044386341018 - 9.440652788158964) / 22.207044386341018 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.489483054461344 < 26.409770014132768.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 155.5800529623684 (T) = (0 -13.54868005270485) / Math.Sqrt((2.992372100978861 / (299)) + (0.004500242017892639 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5388428793823148 = (29.379748131303735 - 13.54868005270485) / 29.379748131303735 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionMemory

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.438942622763435 < 28.08502921127492.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 31.15103759993712 (T) = (0 -25.296803301284033) / Math.Sqrt((3.1032912010264546 / (299)) + (0.2172677205735831 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16852844557735422 = (30.424135578335612 - 25.296803301284033) / 30.424135578335612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 42.51993148290257 < 63.98875983800608.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 133.25926676656985 (T) = (0 -42.41867855150144) / Math.Sqrt((1.169185148922112 / (299)) + (0.4228066267711737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37486071875567667 = (67.85476424880576 - 42.41867855150144) / 67.85476424880576 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc
Compare bba7a9c67e5b6728833bedbb368f732fb499bcc1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
MatchesBoundary - Duration of single invocation 135.63 μs 125.08 μs 0.92 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None)

ETL Files

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.07996666666666 < 129.20057531055897.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 54.350834871715705 (T) = (0 -124114.17538222257) / Math.Sqrt((49198614.7274698 / (299)) + (236477.75699934724 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1576786422497727 = (147347.77201152963 - 124114.17538222257) / 147347.77201152963 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions