-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.Sort<BigStruct>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
LinqOrderByExtension - Duration of single invocation | 41.34 μs | 26.27 μs | 0.64 | 0.08 | False | |||
Array_ComparerClass - Duration of single invocation | 33.94 μs | 18.43 μs | 0.54 | 0.12 | False | |||
Array - Duration of single invocation | 27.24 μs | 9.37 μs | 0.34 | 0.01 | True | |||
Array_Comparison - Duration of single invocation | 35.13 μs | 21.72 μs | 0.62 | 0.03 | False | |||
List - Duration of single invocation | 26.94 μs | 9.01 μs | 0.33 | 0.01 | True | |||
LinqQuery - Duration of single invocation | 40.21 μs | 26.39 μs | 0.66 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<BigStruct>*'
Payloads
System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.26709714285714 < 39.833389483333335.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 89.75131527965284 (T) = (0 -26563.17750746689) / Math.Sqrt((19517240.021517254 / (299)) + (238952.58444156343 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4942491974898899 = (52522.26467191001 - 26563.17750746689) / 52522.26467191001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.431440000000002 < 32.234971821428566.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 56.98358697688732 (T) = (0 -17533.402395040852) / Math.Sqrt((18933216.064761925 / (299)) + (1240067.7432906446 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5642238287023823 = (40234.88100056356 - 17533.402395040852) / 40234.88100056356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.Array(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.369751999999998 < 25.8387912032967.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 396.5060045737119 (T) = (0 -9324.466074387152) / Math.Sqrt((135867.64590530362 / (299)) + (21497.651998935347 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6612923817122311 = (27529.543390621377 - 9324.466074387152) / 27529.543390621377 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.723868000000003 < 33.370398554487174.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 60.90721966612799 (T) = (0 -20498.93489912652) / Math.Sqrt((10904056.824100025 / (299)) + (858985.2550601526 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4875646005691295 = (40002.96412366005 - 20498.93489912652) / 40002.96412366005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.012182857142857 < 25.596059678571425.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 622.7407766390801 (T) = (0 -9006.411293650794) / Math.Sqrt((173530.00652023221 / (299)) + (3505.0383369907413 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.6713050626106389 = (27400.517224827534 - 9006.411293650794) / 27400.517224827534 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.387489230769237 < 39.5133699047619.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 99.88204447051368 (T) = (0 -26339.39125528317) / Math.Sqrt((20273865.019542947 / (299)) + (8668.174462197554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.49806544124711166 = (52475.747676602085 - 26339.39125528317) / 52475.747676602085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Matrix3x2
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
CreateFromScalars - Duration of single invocation | 2.81 ns | 0.98 ns | 0.35 | 0.56 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'
Payloads
System.Numerics.Tests.Perf_Matrix3x2.CreateFromScalars
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9754693838340621 < 2.6503504628132006.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 48.046016389519636 (T) = (0 -1.0183994824155538) / Math.Sqrt((0.0763748358044497 / (299)) + (0.01380065022355191 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6312856341055744 = (2.762028216462695 - 1.0183994824155538) / 2.762028216462695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Enum
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
InterpolateIntoSpan_Flags - Duration of single invocation | 104.13 ns | 98.42 ns | 0.95 | 0.01 | False | |||
InterpolateIntoSpan_Flags - Duration of single invocation | 107.03 ns | 101.11 ns | 0.94 | 0.01 | False | |||
InterpolateIntoSpan_NonFlags - Duration of single invocation | 115.90 ns | 108.35 ns | 0.93 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Payloads
System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 98.41798350571605 < 99.28318037216505.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 43.54347480750639 (T) = (0 -98.18687078209283) / Math.Sqrt((6.493872148777041 / (299)) + (0.033996887791657826 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06470294241787697 = (104.97934317886123 - 98.18687078209283) / 104.97934317886123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 101.11337699934695 < 102.17714822436142.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 36.95355889879681 (T) = (0 -101.44614728034477) / Math.Sqrt((11.577506730308407 / (299)) + (0.025827826449929824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06846332124548722 = (108.90193547287987 - 101.44614728034477) / 108.90193547287987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 108.3509934076004 < 110.29474608505623.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 34.295799530776115 (T) = (0 -109.37662401161118) / Math.Sqrt((11.543642502622902 / (299)) + (0.15770647160518428 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06597010127206568 = (117.1018445561244 - 109.37662401161118) / 117.1018445561244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
ReadStructAndReverseBE - Duration of single invocation | 10.66 ns | 2.31 ns | 0.22 | 0.29 | True | |||
ReadStructFieldByFieldBE - Duration of single invocation | 8.01 ns | 5.17 ns | 0.65 | 0.43 | False | |||
ReadStructFieldByFieldUsingBitConverterBE - Duration of single invocation | 17.70 ns | 1.89 ns | 0.11 | 0.27 | True | |||
ReadStructFieldByFieldUsingBitConverterLE - Duration of single invocation | 8.56 ns | 1.83 ns | 0.21 | 0.29 | True | |||
ReadStructFieldByFieldLE - Duration of single invocation | 8.45 ns | 4.92 ns | 0.58 | 0.40 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'
Payloads
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.312754603627724 < 10.30938151564364.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 145.8343661594201 (T) = (0 -2.1533839438226283) / Math.Sqrt((0.3082248656647507 / (299)) + (0.032977762500796017 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8017869319489568 = (10.863985735128702 - 2.1533839438226283) / 10.863985735128702 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.172164245984457 < 8.002706461427184.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 43.89746022183775 (T) = (0 -5.4823373038374195) / Math.Sqrt((0.5787472320692092 / (299)) + (0.04135782872158954 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36417927133319483 = (8.622457646721955 - 5.4823373038374195) / 8.622457646721955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8946947351082222 < 16.622688729792365.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 168.58303008476727 (T) = (0 -1.767253582880384) / Math.Sqrt((0.8576843283366887 / (299)) + (0.07654852749819531 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.899261210291919 = (17.542930464039703 - 1.767253582880384) / 17.542930464039703 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8282319193156682 < 8.081330273244793.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 120.81517172644624 (T) = (0 -1.8209277687970455) / Math.Sqrt((0.26312912574431613 / (299)) + (0.025671723287621774 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.784748700861126 = (8.459543687224093 - 1.8209277687970455) / 8.459543687224093 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.922566858589282 < 8.150515208272855.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 53.411450446055305 (T) = (0 -5.338215025746046) / Math.Sqrt((0.18780330330053804 / (299)) + (0.03503415281139085 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36579390958879504 = (8.417161402983165 - 5.338215025746046) / 8.417161402983165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
IterateTryGetMemory - Duration of single invocation | 17.11 ns | 13.96 ns | 0.82 | 0.24 | False | |||
IterateForEachArray - Duration of single invocation | 16.43 ns | 4.89 ns | 0.30 | 0.36 | True | |||
IterateForEachSingleSegment - Duration of single invocation | 21.68 ns | 9.38 ns | 0.43 | 0.30 | True | |||
IterateForEachMemory - Duration of single invocation | 27.84 ns | 13.49 ns | 0.48 | 0.17 | True | |||
IterateGetPositionMemory - Duration of single invocation | 29.24 ns | 25.44 ns | 0.87 | 0.22 | False | |||
IterateForEachTenSegments - Duration of single invocation | 68.37 ns | 42.52 ns | 0.62 | 0.02 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'
Payloads
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.96257742760995 < 16.315278464226775.
IsChangePoint: Marked as a change because one of 3/3/2023 4:19:26 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 45.01471080749216 (T) = (0 -13.775694580559946) / Math.Sqrt((2.2315571540211305 / (299)) + (0.012816646751738096 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2309859404826928 = (17.913449578810873 - 13.775694580559946) / 17.913449578810873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachArray
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.888441455507997 < 15.619858550934177.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 142.21910030965572 (T) = (0 -4.804568820779855) / Math.Sqrt((0.6013143377002923 / (299)) + (0.07418062507146722 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7222510817665462 = (17.298244944887653 - 4.804568820779855) / 17.298244944887653 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.377807913782599 < 20.603843577420903.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 272.9118387060244 (T) = (0 -9.440652788158964) / Math.Sqrt((0.47008057741447407 / (299)) + (0.008008612660880158 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.574880266643423 = (22.207044386341018 - 9.440652788158964) / 22.207044386341018 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.489483054461344 < 26.409770014132768.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 155.5800529623684 (T) = (0 -13.54868005270485) / Math.Sqrt((2.992372100978861 / (299)) + (0.004500242017892639 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5388428793823148 = (29.379748131303735 - 13.54868005270485) / 29.379748131303735 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.438942622763435 < 28.08502921127492.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 31.15103759993712 (T) = (0 -25.296803301284033) / Math.Sqrt((3.1032912010264546 / (299)) + (0.2172677205735831 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16852844557735422 = (30.424135578335612 - 25.296803301284033) / 30.424135578335612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 42.51993148290257 < 63.98875983800608.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 133.25926676656985 (T) = (0 -42.41867855150144) / Math.Sqrt((1.169185148922112 / (299)) + (0.4228066267711737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37486071875567667 = (67.85476424880576 - 42.41867855150144) / 67.85476424880576 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
MatchesBoundary - Duration of single invocation | 135.63 μs | 125.08 μs | 0.92 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
Payloads
System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.07996666666666 < 129.20057531055897.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 54.350834871715705 (T) = (0 -124114.17538222257) / Math.Sqrt((49198614.7274698 / (299)) + (236477.75699934724 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1576786422497727 = (147347.77201152963 - 124114.17538222257) / 147347.77201152963 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository