Skip to content

[Perf] Windows/x86: 13 Improvements on 5/13/2023 6:35:24 AM #17798

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToLower - Duration of single invocation 38.17 ns 33.06 ns 0.87 0.36 False 412.28884650853956 391.69855287331893 0.9500585722616822)
ToLower - Duration of single invocation 37.68 ns 32.98 ns 0.88 0.37 False 412.2597811940613 392.2727360552085 0.9515183240020099)

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToLower(s: "TEST")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.060066470196745 < 36.643513898009125.
IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 18.130122719847193 (T) = (0 -33.075654787806464) / Math.Sqrt((4.537564116414962 / (299)) + (0.3862797389351696 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12693911843018293 = (37.88470596498884 - 33.075654787806464) / 37.88470596498884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_String.ToLower(s: "TeSt")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.97950108190515 < 35.63814101873775.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 27.78263121535 (T) = (0 -32.643121011465574) / Math.Sqrt((4.062393914526268 / (299)) + (0.1484953947613509 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1370171469343308 = (37.82592075323839 - 32.643121011465574) / 37.82592075323839 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 447.18 μs 407.49 μs 0.91 0.19 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 407.4918696581197 < 421.3041417824074.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 20.77674092478743 (T) = (0 -403773.64621489623) / Math.Sqrt((200381358.3989649 / (299)) + (18353267.246957343 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08539049005654156 = (441471.0779028065 - 403773.64621489623) / 441471.0779028065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_ThreadStatic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetThreadStatic - Duration of single invocation 2.42 ns 0.00 ns 0.00 0.44 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_ThreadStatic*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_ThreadStatic.GetThreadStatic


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 2.189268533088456.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 29.793107303505035 (T) = (0 -0.06679310889154648) / Math.Sqrt((0.08154669045265547 / (299)) + (0.031229235767785895 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.9684445653393288 = (2.1166911376693363 - 0.06679310889154648) / 2.1166911376693363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_ComparerClass - Duration of single invocation 415.14 μs 366.97 μs 0.88 0.24 False 3000173.3333333335 2902373.3333333335 0.9674018834467342) Trace Trace
List - Duration of single invocation 379.41 μs 342.93 μs 0.90 0.25 False 2942031.3725490198 2843693.333333333 0.9665747822632886) Trace Trace

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<String>.Array_ComparerClass(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 366.9665493333333 < 373.7856093809524.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 13.869725976109729 (T) = (0 -361697.0304353741) / Math.Sqrt((247485263.58170983 / (299)) + (24983475.401666973 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.07442229888722601 = (390779.7584152304 - 361697.0304353741) / 390779.7584152304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Collections.Sort<String>.List(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 342.9255371428571 < 352.87371085714284.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 14.566413073072889 (T) = (0 -345804.2028644689) / Math.Sqrt((173309114.88252562 / (299)) + (33444603.8842842 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08883751933893279 = (379519.8004790329 - 345804.2028644689) / 379519.8004790329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Lock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReaderWriterLockSlimPerf - Duration of single invocation 28.81 ns 23.56 ns 0.82 0.04 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Lock*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Lock.ReaderWriterLockSlimPerf


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.56087104369602 < 26.74311394760398.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 30.148663034321054 (T) = (0 -23.916114931256157) / Math.Sqrt((0.6455074536757064 / (299)) + (0.056716366198639595 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11323613476929918 = (26.970105423763666 - 23.916114931256157) / 26.970105423763666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateAndAwait_FromResult_ConfigureAwait - Duration of single invocation 14.15 ns 10.39 ns 0.73 0.38 False 134.06709131788082 114.40923520739733 0.8533729946905942) Trace Trace
CreateAndAwait_FromDelayedTCS - Duration of single invocation 192.75 ns 174.35 ns 0.90 0.09 False 1304.250662721164 1167.2658963411511 0.8949705219284992) Trace Trace
CreateAndAwait_FromResult - Duration of single invocation 13.19 ns 10.36 ns 0.78 0.40 False Trace Trace

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult_ConfigureAwait


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.389713634360824 < 12.724327267655568.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 29.879519105017387 (T) = (0 -10.507058408078567) / Math.Sqrt((1.376315617223371 / (299)) + (0.05576739201154755 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.241751325065907 = (13.857008598124937 - 10.507058408078567) / 13.857008598124937 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromDelayedTCS


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 174.3497629682349 < 183.68725208354934.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 26.0877130320443 (T) = (0 -174.4607944130299) / Math.Sqrt((47.95478962239706 / (299)) + (0.8062838094633422 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.07278353559053191 = (188.15541042419008 - 174.4607944130299) / 188.15541042419008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.355198108805123 < 12.739832325925398.
IsChangePoint: Marked as a change because one of 3/22/2023 4:29:55 AM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 28.266293780777243 (T) = (0 -10.45913716496663) / Math.Sqrt((1.3265660922083027 / (299)) + (0.07428620633450043 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24898591692054262 = (13.926685798061197 - 10.45913716496663) / 13.926685798061197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 35cbe111b86cdb0581c257f85490cdcca735cedf
Compare aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 17.27 ns 14.50 ns 0.84 0.33 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: -128)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.495912968831854 < 16.349643581188545.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 8:51:41 AM, 5/13/2023 1:49:08 AM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 16.805361322026467 (T) = (0 -14.636067654203185) / Math.Sqrt((1.0718669132560403 / (299)) + (0.07236648153431163 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11931838160330692 = (16.61902252581197 - 14.636067654203185) / 16.61902252581197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions