[Perf] Windows/arm64: 133 Improvements on 5/13/2023 2:36:22 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	937e25a340484327fcb87cfc5f20905c10476308
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	13.72 ns	10.67 ns	0.78	0.11	False
Parse - Duration of single invocation	9.87 ns	7.19 ns	0.73	0.31	False
TryParse - Duration of single invocation	13.69 ns	10.94 ns	0.80	0.08	False
TryParse - Duration of single invocation	10.17 ns	7.77 ns	0.76	0.35	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.Parse(value: "255")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.671812978147706 < 13.005944637581637.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 24.91699700063721 (T) = (0 -10.440817391070382) / Math.Sqrt((0.5052967685407043 / (299)) + (0.03577598490124334 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1476355759782734 = (12.249241165893906 - 10.440817391070382) / 12.249241165893906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.192700720911371 < 9.8189588358402.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 6.606544549703173 (T) = (0 -7.769924459371771) / Math.Sqrt((0.8569441948899227 / (299)) + (0.6067824529657947 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17000189259945694 = (9.36137611651461 - 7.769924459371771) / 9.36137611651461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "255")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.938867282216657 < 12.986995342135273.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 23.035171612286277 (T) = (0 -10.514466498098232) / Math.Sqrt((0.5531824264038403 / (299)) + (0.04398782583373275 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14350581609521448 = (12.276168006375045 - 10.514466498098232) / 12.276168006375045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.774458026650582 < 9.910567488992319.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 25.423502373522023 (T) = (0 -7.779598236580259) / Math.Sqrt((0.7343734408535995 / (299)) + (0.02538663590732934 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.184046765877201 = (9.534367793693246 - 7.779598236580259) / 9.534367793693246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ParseSpan - Duration of single invocation	11.88 ns	7.83 ns	0.66	0.19	False
TryParseSpan - Duration of single invocation	15.31 ns	10.81 ns	0.71	0.05	False
TryParse - Duration of single invocation	23.48 ns	19.07 ns	0.81	0.03	False
ParseHex - Duration of single invocation	11.05 ns	7.61 ns	0.69	0.22	False
ToString - Duration of single invocation	31.28 ns	27.20 ns	0.87	0.04	False
TryParse - Duration of single invocation	15.34 ns	11.15 ns	0.73	0.03	False
Parse - Duration of single invocation	16.49 ns	12.44 ns	0.75	0.06	False
ParseSpan - Duration of single invocation	23.81 ns	19.42 ns	0.82	0.02	False
Parse - Duration of single invocation	11.87 ns	8.04 ns	0.68	0.17	False
TryParseSpan - Duration of single invocation	10.11 ns	6.93 ns	0.69	0.35	False
Parse - Duration of single invocation	24.99 ns	20.48 ns	0.82	0.05	False
TryParse - Duration of single invocation	22.57 ns	18.31 ns	0.81	0.01	False
Parse - Duration of single invocation	23.85 ns	19.46 ns	0.82	0.04	False
ParseHex - Duration of single invocation	22.21 ns	18.34 ns	0.83	0.05	False
TryParse - Duration of single invocation	10.50 ns	6.61 ns	0.63	0.31	False
ParseSpan - Duration of single invocation	25.01 ns	20.50 ns	0.82	0.02	False
ParseHex - Duration of single invocation	15.72 ns	11.83 ns	0.75	0.03	False
ParseSpan - Duration of single invocation	16.58 ns	12.67 ns	0.76	0.05	False
TryParseSpan - Duration of single invocation	22.53 ns	18.31 ns	0.81	0.02	False
ParseHex - Duration of single invocation	23.37 ns	18.29 ns	0.78	0.09	False
TryParseSpan - Duration of single invocation	23.67 ns	19.35 ns	0.82	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.833814267534509 < 11.260372740326353.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 47.140669245368585 (T) = (0 -8.037512599101506) / Math.Sqrt((0.7889661862279099 / (299)) + (0.010005325517635536 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.258911677721236 = (10.845552894946517 - 8.037512599101506) / 10.845552894946517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.81435031786245 < 14.53324565490326.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 46.135566851693376 (T) = (0 -10.946070352294988) / Math.Sqrt((0.2823973739313572 / (299)) + (0.046340011644749796 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.23235215161826778 = (14.259234068551416 - 10.946070352294988) / 14.259234068551416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.074550621013632 < 22.187994979388435.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 45.326025762651355 (T) = (0 -19.188919037519323) / Math.Sqrt((0.3523215954486938 / (299)) + (0.03914061982668859 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13983362880821293 = (22.308380890236947 - 19.188919037519323) / 22.308380890236947 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "4")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.608549425815084 < 10.220012510288035.
IsChangePoint: Marked as a change because one of 4/24/2023 7:54:29 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 32.3589149844708 (T) = (0 -7.472331738775517) / Math.Sqrt((0.9877948827323542 / (299)) + (0.027599961645350824 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.24821252238777392 = (9.939420329942985 - 7.472331738775517) / 9.939420329942985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToString(value: -2147483648)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.201280708804003 < 29.181393117434478.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 17.79663614953211 (T) = (0 -27.183220361209145) / Math.Sqrt((0.8696924263078313 / (299)) + (0.11085520364145555 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06942755354843885 = (29.21128866953198 - 27.183220361209145) / 29.21128866953198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.15495055282128 < 14.545005210353954.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 81.39285479853484 (T) = (0 -11.190433503057179) / Math.Sqrt((0.31668554714397706 / (299)) + (0.0038012476008723157 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.21421020870127924 = (14.241001380995412 - 11.190433503057179) / 14.241001380995412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.439432174066894 < 15.747896919528499.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 66.35629929858187 (T) = (0 -12.568507660256616) / Math.Sqrt((0.3814211461656128 / (299)) + (0.008844918979717224 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1970326046014402 = (15.652575350233404 - 12.568507660256616) / 15.652575350233404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.4197503235874 < 22.539765289450003.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 55.4610586948543 (T) = (0 -19.434634550228783) / Math.Sqrt((0.43817834638437975 / (299)) + (0.022390105817779248 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14445915824642375 = (22.716197289183995 - 19.434634550228783) / 22.716197289183995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "4")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.038381563685173 < 11.254837364556115.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/13/2023 2:36:22 AM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 9.257816575795479 (T) = (0 -8.260969220441677) / Math.Sqrt((0.7792974080451075 / (299)) + (0.7086664732241207 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.2329921310783928 = (10.77038392325281 - 8.260969220441677) / 10.77038392325281 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.934109369591293 < 9.947743408973276.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 33.682994719122206 (T) = (0 -6.967737801237536) / Math.Sqrt((0.8977245597449506 / (299)) + (0.03987988953300557 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2824109905307026 = (9.709928258782309 - 6.967737801237536) / 9.709928258782309 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.482952553942166 < 23.646362684930825.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 60.639236788913706 (T) = (0 -20.452692057327457) / Math.Sqrt((0.5253441565597482 / (299)) + (0.009069082574075835 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13091743361886274 = (23.53365819141044 - 20.452692057327457) / 23.53365819141044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.314481881546637 < 21.43254628341127.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 45.732464686972925 (T) = (0 -18.221466962442776) / Math.Sqrt((0.4357109157826383 / (299)) + (0.03487461274787214 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1458353909813579 = (21.332500515770132 - 18.221466962442776) / 21.332500515770132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.456331467177264 < 22.590529023626818.
IsChangePoint: Marked as a change because one of 4/19/2023 10:34:24 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 30.232079363568523 (T) = (0 -19.540349983537556) / Math.Sqrt((0.5167896614057079 / (299)) + (0.09861995914246847 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13792627813249678 = (22.666680920522037 - 19.540349983537556) / 22.666680920522037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.336723122321462 < 21.16712807962659.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 38.47775841929523 (T) = (0 -18.295496732266027) / Math.Sqrt((0.8280201443146081 / (299)) + (0.00482207554481668 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10643543642821589 = (20.47473397908115 - 18.295496732266027) / 20.47473397908115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "4")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.614786146660132 < 10.081709387426008.
IsChangePoint: Marked as a change because one of 4/24/2023 7:54:29 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 33.43236035610447 (T) = (0 -6.856168835625408) / Math.Sqrt((0.581032225911525 / (299)) + (0.046321070679843264 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.283363628544553 = (9.567151638844294 - 6.856168835625408) / 9.567151638844294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.499555989949997 < 23.652476225329508.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 51.32607039085127 (T) = (0 -20.466658440295305) / Math.Sqrt((0.4215360526108203 / (299)) + (0.025157970311464932 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13230532247769755 = (23.587396546833432 - 20.466658440295305) / 23.587396546833432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.827673395020716 < 14.974069651918867.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 40.729715197358246 (T) = (0 -11.86415198213645) / Math.Sqrt((0.4556584020561138 / (299)) + (0.028148127057640703 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1798975063528134 = (14.466669805348117 - 11.86415198213645) / 14.466669805348117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.670442910351142 < 15.820650988024965.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 55.75081165988025 (T) = (0 -12.57997032409122) / Math.Sqrt((0.3242800877092443 / (299)) + (0.022139197683962785 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19784121347192807 = (15.682643555573618 - 12.57997032409122) / 15.682643555573618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.305529833495086 < 21.351982869486086.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 75.99316575874028 (T) = (0 -18.25413727386574) / Math.Sqrt((0.3894172529211899 / (299)) + (0.0037220591025388635 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14429772385357617 = (21.332346287626535 - 18.25413727386574) / 21.332346287626535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "80000000")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.286183793767446 < 21.276183181782827.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 36.28141640931502 (T) = (0 -18.24699050595397) / Math.Sqrt((0.9115464163212782 / (299)) + (0.006726704458563844 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10817372791267536 = (20.46025226779514 - 18.24699050595397) / 20.46025226779514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.352732948276316 < 22.226465239756866.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.84041432486404 (T) = (0 -19.03607943023168) / Math.Sqrt((0.375769881086755 / (299)) + (0.3967604559671293 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14596708817661783 = (22.289632128566524 - 19.03607943023168) / 22.289632128566524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	937e25a340484327fcb87cfc5f20905c10476308
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	315.66 μs	293.67 μs	0.93	0.07	False
Array_ComparerClass - Duration of single invocation	329.77 μs	307.44 μs	0.93	0.02	False
Array_ComparerStruct - Duration of single invocation	331.99 μs	312.47 μs	0.94	0.07	False
List - Duration of single invocation	315.50 μs	295.68 μs	0.94	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<String>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 293.6669253333333 < 299.87536709615387.
IsChangePoint: Marked as a change because one of 3/15/2023 2:24:48 AM, 3/15/2023 9:07:40 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 7.692692518899324 (T) = (0 -296962.6203849484) / Math.Sqrt((65243479.534937225 / (299)) + (74717402.26298454 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06418688569156503 = (317331.1164851579 - 296962.6203849484) / 317331.1164851579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<String>.Array_ComparerClass(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 307.44456 < 312.05370616758245.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 9.287286339424208 (T) = (0 -310842.6984029304) / Math.Sqrt((22664955.663297083 / (299)) + (56308044.57239097 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0637555933921122 = (332010.20610541885 - 310842.6984029304) / 332010.20610541885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<String>.Array_ComparerStruct(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 312.46698266666664 < 321.49555625.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.041405156119872 (T) = (0 -312104.020989011) / Math.Sqrt((48053715.180840425 / (299)) + (19783564.864260435 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06991619005106472 = (335565.48092816124 - 312104.020989011) / 335565.48092816124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<String>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 295.6757846153846 < 300.3120642166666.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 45.06412727899171 (T) = (0 -295426.22681185475) / Math.Sqrt((51442836.71772181 / (299)) + (663616.6117074635 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06849589908875008 = (317149.67923689453 - 295426.22681185475) / 317149.67923689453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ParseSpan - Duration of single invocation	38.76 ns	35.75 ns	0.92	0.09	False
TryParseSpan - Duration of single invocation	16.42 ns	11.81 ns	0.72	0.08	False
Parse - Duration of single invocation	39.84 ns	35.93 ns	0.90	0.08	False
TryParseSpan - Duration of single invocation	38.29 ns	34.23 ns	0.89	0.05	False
TryParseSpan - Duration of single invocation	37.09 ns	33.62 ns	0.91	0.07	False
Parse - Duration of single invocation	17.63 ns	13.64 ns	0.77	0.09	False
TryParse - Duration of single invocation	38.72 ns	33.58 ns	0.87	0.06	False
Parse - Duration of single invocation	38.30 ns	35.81 ns	0.93	0.07	False
ParseSpan - Duration of single invocation	17.56 ns	13.48 ns	0.77	0.10	False
TryParse - Duration of single invocation	38.32 ns	34.72 ns	0.91	0.06	False
ParseSpan - Duration of single invocation	39.29 ns	32.36 ns	0.82	0.06	False
TryParse - Duration of single invocation	15.85 ns	12.23 ns	0.77	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.75178555492885 < 36.882378221089596.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/1/2023 10:17:01 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 14.420713072428601 (T) = (0 -35.43323575204482) / Math.Sqrt((1.0377245309807421 / (299)) + (0.16115261699095965 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.051939948344460446 = (37.37446345320628 - 35.43323575204482) / 37.37446345320628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.806520234146824 < 15.417188566013213.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 53.09489376417399 (T) = (0 -12.103204621501726) / Math.Sqrt((0.25928387490855065 / (299)) + (0.0256781383090054 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19885752717425043 = (15.107430990160726 - 12.103204621501726) / 15.107430990160726 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.92733570388294 < 37.47828263603495.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.087065020320246 (T) = (0 -35.84382670531152) / Math.Sqrt((0.988313198753162 / (299)) + (0.21760476779990942 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06384130551294 = (38.28819506392671 - 35.84382670531152) / 38.28819506392671 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.23452381873218 < 36.45727717312098.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 22.955813927103648 (T) = (0 -34.64908338793414) / Math.Sqrt((0.8661570495979389 / (299)) + (0.050941145397218404 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.05591295988514003 = (36.70115351198831 - 34.64908338793414) / 36.70115351198831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.620586339917836 < 34.97184234678203.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 30.675051455860434 (T) = (0 -33.26957926361136) / Math.Sqrt((0.72762868424096 / (299)) + (0.04884609687226261 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07101566333357077 = (35.81285275809498 - 33.26957926361136) / 35.81285275809498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.644574632341124 < 16.62315707383255.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 65.86808627968769 (T) = (0 -13.45762001189819) / Math.Sqrt((0.3858768681184381 / (299)) + (0.010555707427014739 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1884265418114714 = (16.582134218553982 - 13.45762001189819) / 16.582134218553982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.57665763276371 < 35.347088761139524.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 9.561623995949068 (T) = (0 -33.180961137014855) / Math.Sqrt((0.7146525421564726 / (299)) + (0.7405958141839251 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.07379670659184898 = (35.82470649064402 - 33.180961137014855) / 35.82470649064402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.80717708373515 < 36.99053037253573.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/17/2023 8:43:19 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 9.407080745103043 (T) = (0 -35.03241050177611) / Math.Sqrt((0.9013775816004198 / (299)) + (0.5815988979565807 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06227929377257614 = (37.35910945457971 - 35.03241050177611) / 37.35910945457971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.483927273865143 < 16.68261029378252.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 24.256223233307942 (T) = (0 -13.563672050778296) / Math.Sqrt((0.3665887925917611 / (299)) + (0.16172709741667948 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18414035210959814 = (16.625006624424163 - 13.563672050778296) / 16.625006624424163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.71817424009282 < 36.44960384730624.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 4:25:00 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 23.82174761642944 (T) = (0 -34.598485085347946) / Math.Sqrt((0.8335514474166558 / (299)) + (0.052564971109128375 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0565068172801508 = (36.67062541523551 - 34.598485085347946) / 36.67062541523551 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.362668560547924 < 37.49503943794331.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 4:25:00 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 9.914154075253807 (T) = (0 -35.241965093585186) / Math.Sqrt((0.9077809661036201 / (299)) + (0.9701926545343085 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07831745630540175 = (38.23655480369247 - 35.241965093585186) / 38.23655480369247 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.228804555268807 < 15.400050976210533.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 46.7713979998979 (T) = (0 -12.090071822447486) / Math.Sqrt((0.2545564083711774 / (299)) + (0.03503996841634146 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19729885165411534 = (15.06173480299777 - 12.090071822447486) / 15.06173480299777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Diagnostics.Perf_Activity

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ActivityAllocations - Duration of single invocation	372.21 ns	331.81 ns	0.89	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 331.8087064131368 < 335.3782372934018.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 10.790550093112337 (T) = (0 -334.49237570601247) / Math.Sqrt((54.44762429060378 / (299)) + (46.68971387080758 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06633849389300277 = (358.2587195874816 - 334.49237570601247) / 358.2587195874816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dequeue_And_Enqueue - Duration of single invocation	2.78 ms	2.60 ms	0.94	0.05	False
HeapSort - Duration of single invocation	3.21 μs	3.01 μs	0.94	0.03	False
HeapSort - Duration of single invocation	72.31 μs	68.21 μs	0.94	0.02	False
HeapSort - Duration of single invocation	1.27 ms	1.19 ms	0.94	0.03	False
K_Max_Elements - Duration of single invocation	10.38 μs	9.76 μs	0.94	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<String, String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<String, String>.Dequeue_And_Enqueue(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.596553525641026 < 2.6312660777243586.
IsChangePoint: Marked as a change because one of 1/31/2023 7:06:58 PM, 2/3/2023 4:47:55 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 38.38371628434633 (T) = (0 -2596856.4800997614) / Math.Sqrt((4646220255.216939 / (299)) + (48596567.999632105 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06194091743112621 = (2768329.339116117 - 2596856.4800997614) / 2768329.339116117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.006106838428507 < 3.0426658983839077.
IsChangePoint: Marked as a change because one of 1/26/2023 1:49:07 PM, 1/27/2023 7:12:54 PM, 3/2/2023 6:22:31 PM, 3/3/2023 12:05:09 PM, 4/12/2023 8:39:35 PM, 4/18/2023 2:18:01 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 27.45610325530387 (T) = (0 -3006.343983738635) / Math.Sqrt((4476.130543196051 / (299)) + (204.30988016703253 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0502368852616828 = (3165.361906654962 - 3006.343983738635) / 3165.361906654962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 68.21303751803751 < 68.97272789294175.
IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 2/4/2023 3:18:53 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 57.53965625430034 (T) = (0 -68102.78049413413) / Math.Sqrt((1090658.9541475957 / (299)) + (23917.034278976862 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06056267022769126 = (72493.15982646782 - 68102.78049413413) / 72493.15982646782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1863931760204083 < 1.2080677120535712.
IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 2/4/2023 3:18:53 AM, 3/1/2023 1:48:25 AM, 3/4/2023 1:07:20 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 32.492223477878554 (T) = (0 -1188965.4728872317) / Math.Sqrt((1141645129.5286038 / (299)) + (28279305.51982217 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06581542335243626 = (1272730.7885492828 - 1188965.4728872317) / 1272730.7885492828 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.K_Max_Elements(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.759354181643902 < 9.840642002361319.
IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 2/4/2023 3:18:53 AM, 2/28/2023 5:58:27 PM, 3/4/2023 1:07:20 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 43.048714320369704 (T) = (0 -9759.17996363443) / Math.Sqrt((46631.56283623045 / (299)) + (628.2540223607166 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0604937016205564 = (10387.562042391903 - 9759.17996363443) / 10387.562042391903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateAndAwait_FromCompletedTask_ConfigureAwait - Duration of single invocation	23.35 ns	20.17 ns	0.86	0.21	False
Await_FromCompletedTask - Duration of single invocation	32.28 ns	28.65 ns	0.89	0.22	False
Await_FromResult - Duration of single invocation	15.41 ns	12.28 ns	0.80	0.28	False
CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait - Duration of single invocation	29.39 ns	24.74 ns	0.84	0.03	True
CreateAndAwait_FromResult - Duration of single invocation	15.21 ns	12.32 ns	0.81	0.26	False
CreateAndAwait_FromResult_ConfigureAwait - Duration of single invocation	15.92 ns	11.38 ns	0.71	0.27	False
CreateAndAwait_FromDelayedTCS - Duration of single invocation	242.46 ns	220.04 ns	0.91	0.21	False
CreateAndAwait_FromCompletedValueTaskSource - Duration of single invocation	29.63 ns	25.22 ns	0.85	0.02	True
CreateAndAwait_FromCompletedTask - Duration of single invocation	23.42 ns	20.62 ns	0.88	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedTask_ConfigureAwait

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.174566710085767 < 22.084359122417442.
IsChangePoint: Marked as a change because one of 4/25/2023 1:14:10 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 12.145883492517804 (T) = (0 -20.551434724220453) / Math.Sqrt((1.3689573349064499 / (299)) + (1.0355999052637155 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15661215061016448 = (24.36771497133705 - 20.551434724220453) / 24.36771497133705 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.Await_FromCompletedTask

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.65132439900589 < 30.605830782306377.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 11.360463158684592 (T) = (0 -28.751520010510532) / Math.Sqrt((1.4672954628416013 / (299)) + (1.4665385308833383 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1280877594097693 = (32.97524529653068 - 28.751520010510532) / 32.97524529653068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.Await_FromResult

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.280509666185555 < 14.572392214737913.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 32.979817131247664 (T) = (0 -12.458735922680926) / Math.Sqrt((0.953553302160567 / (299)) + (0.16119579595326555 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.26122916011877145 = (16.864141422641836 - 12.458735922680926) / 16.864141422641836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.739164211141514 < 27.742108850510338.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 42.02688329435479 (T) = (0 -24.652762276793737) / Math.Sqrt((0.2055181715543141 / (299)) + (0.1350387214378233 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.16254884213199933 = (29.437850846794714 - 24.652762276793737) / 29.437850846794714 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.322618648049941 < 14.380047639463235.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.517797529158223 (T) = (0 -12.60381907066331) / Math.Sqrt((0.9901297405603895 / (299)) + (0.6551247512259131 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.24732722617654923 = (16.745416479777838 - 12.60381907066331) / 16.745416479777838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult_ConfigureAwait

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.380993105789425 < 14.861817930144394.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 39.499661388436024 (T) = (0 -12.124452535949763) / Math.Sqrt((0.9446238608444572 / (299)) + (0.11398106604794367 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.27474484509688296 = (16.717499288328952 - 12.124452535949763) / 16.717499288328952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromDelayedTCS

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 220.03966899780772 < 232.79167611014685.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 8.039677601320054 (T) = (0 -222.2496829286433) / Math.Sqrt((112.45172099031622 / (299)) + (44.230803835295845 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07050598916096781 = (239.108246354405 - 222.2496829286433) / 239.108246354405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.219630579883006 < 28.192147979751663.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 25.801085624814178 (T) = (0 -24.798600679039165) / Math.Sqrt((0.19786699774981786 / (299)) + (0.34314611108698845 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15661646902028126 = (29.40370515681259 - 24.798600679039165) / 29.40370515681259 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedTask

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.622299643887978 < 22.233242497323598.
IsChangePoint: Marked as a change because one of 4/20/2023 12:14:19 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 37.83363506141038 (T) = (0 -20.30158158450484) / Math.Sqrt((1.4327631647604486 / (299)) + (0.07608645170402348 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.16781291033718368 = (24.39545366262602 - 20.30158158450484) / 24.39545366262602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	17.88 ns	13.87 ns	0.78	0.06	False
Parse - Duration of single invocation	10.14 ns	7.21 ns	0.71	0.38	False
Parse - Duration of single invocation	18.58 ns	15.17 ns	0.82	0.08	False
Parse - Duration of single invocation	17.50 ns	11.92 ns	0.68	0.05	False
ToString - Duration of single invocation	25.77 ns	21.39 ns	0.83	0.06	False
TryParse - Duration of single invocation	18.77 ns	14.89 ns	0.79	0.10	False
TryParse - Duration of single invocation	10.25 ns	7.89 ns	0.77	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.TryParse(value: "32767")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.869840215279483 < 16.950276670911336.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 17.505631923596628 (T) = (0 -13.766084684213013) / Math.Sqrt((1.6070791453640907 / (299)) + (0.02674270073756194 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10240515471958252 = (15.336635183006596 - 13.766084684213013) / 15.336635183006596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.214816752115288 < 10.056588702680603.
IsChangePoint: Marked as a change because one of 4/25/2023 1:14:10 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 23.37745632952592 (T) = (0 -7.515023178574551) / Math.Sqrt((0.7756231831951722 / (299)) + (0.08897555856342784 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2432957515171344 = (9.931255432543956 - 7.515023178574551) / 9.931255432543956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "-32768")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.173971995990971 < 18.077718088015455.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 10.98152069074777 (T) = (0 -15.077258277594762) / Math.Sqrt((2.172010457042662 / (299)) + (0.048773168054010545 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07302431305536766 = (16.264998629349506 - 15.077258277594762) / 16.264998629349506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "32767")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.915572176528576 < 17.08175871625318.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 8.049900854850474 (T) = (0 -13.760286973283884) / Math.Sqrt((1.639776015837012 / (299)) + (0.40580506339657535 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1074824554291153 = (15.417385413862895 - 13.760286973283884) / 15.417385413862895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.ToString(value: -32768)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.38842152625813 < 24.17237430522043.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.916410812168518 (T) = (0 -21.300707189023868) / Math.Sqrt((0.3765362770615314 / (299)) + (0.506725713050478 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.14731572534550344 = (24.98076699919769 - 21.300707189023868) / 24.98076699919769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "-32768")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.886649668578897 < 17.92241842367771.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 14.868154876102823 (T) = (0 -14.804273789096188) / Math.Sqrt((1.956389751117432 / (299)) + (0.02651220729707074 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08678320293744117 = (16.211127343162566 - 14.804273789096188) / 16.211127343162566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.891591005291478 < 9.960813358301325.
IsChangePoint: Marked as a change because one of 4/25/2023 1:14:10 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 31.032361878356262 (T) = (0 -7.7940415079307845) / Math.Sqrt((0.5315970979590579 / (299)) + (0.0324791037335936 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.21497656968726456 = (9.928418957923098 - 7.7940415079307845) / 9.928418957923098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	16.00 ns	11.36 ns	0.71	0.07	False
Parse - Duration of single invocation	17.39 ns	13.20 ns	0.76	0.09	False
TryParse - Duration of single invocation	36.23 ns	32.60 ns	0.90	0.04	False
ParseSpan - Duration of single invocation	17.18 ns	13.22 ns	0.77	0.06	False
ParseSpan - Duration of single invocation	11.34 ns	8.51 ns	0.75	0.26	False
TryParse - Duration of single invocation	10.42 ns	6.89 ns	0.66	0.33	False
ParseSpan - Duration of single invocation	37.87 ns	34.37 ns	0.91	0.14	False
Parse - Duration of single invocation	37.97 ns	34.53 ns	0.91	0.09	False
Parse - Duration of single invocation	11.48 ns	8.60 ns	0.75	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.356945141100113 < 15.15617097280254.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 41.19227420355002 (T) = (0 -11.636644260084324) / Math.Sqrt((0.40944816377010745 / (299)) + (0.04260570154714493 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.20986577438686282 = (14.727427167269449 - 11.636644260084324) / 14.727427167269449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.2036409468859 < 16.34711606201581.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 62.014481361807164 (T) = (0 -13.153317328335197) / Math.Sqrt((0.4303638472730388 / (299)) + (0.009678048988614957 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18503807846530673 = (16.139793750811773 - 13.153317328335197) / 16.139793750811773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.60264993619963 < 34.16270329813288.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 20.340171739068225 (T) = (0 -32.484901084979896) / Math.Sqrt((0.3903872970661628 / (299)) + (0.17388111951494126 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07570861009363648 = (35.14573590074318 - 32.484901084979896) / 35.14573590074318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.217617475578372 < 16.35552605685893.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 55.00876357968174 (T) = (0 -13.155773854538042) / Math.Sqrt((0.42728665063227583 / (299)) + (0.015862816137244856 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1830387731094149 = (16.103302606673136 - 13.155773854538042) / 16.103302606673136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.513292735362421 < 10.837546780542015.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/23/2023 8:38:14 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 26.839184900346083 (T) = (0 -8.494942472397598) / Math.Sqrt((1.07477213675833 / (299)) + (0.030515392323325806 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20136426767462173 = (10.636817423211173 - 8.494942472397598) / 10.636817423211173 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.891178229879653 < 9.888107485046476.
IsChangePoint: Marked as a change because one of 4/24/2023 8:06:00 AM, 5/13/2023 2:36:22 AM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 23.68535962821919 (T) = (0 -7.211859326718328) / Math.Sqrt((0.9308633109038095 / (299)) + (0.06675891258064708 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.24524968271171085 = (9.555291546785321 - 7.211859326718328) / 9.555291546785321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.3697824255896 < 35.73211594817747.
IsChangePoint: Marked as a change because one of 4/24/2023 7:54:29 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 27.464654441858123 (T) = (0 -34.26557426515773) / Math.Sqrt((0.9095745112408192 / (299)) + (0.06487559619907148 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.0704458798027624 = (36.86237683276266 - 34.26557426515773) / 36.86237683276266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.53300973240322 < 35.77594136074848.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 25.9300800971395 (T) = (0 -34.35766576870049) / Math.Sqrt((0.7284434037831707 / (299)) + (0.07558801713844723 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06787069629807485 = (36.8593344638453 - 34.35766576870049) / 36.8593344638453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.598974244053775 < 10.924505243351907.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/24/2023 1:33:55 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 12.15868252748645 (T) = (0 -8.601936303872456) / Math.Sqrt((0.8368293103581739 / (299)) + (0.25307517072625096 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1850462153237079 = (10.555121610103612 - 8.601936303872456) / 10.555121610103612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	14.27 ns	10.90 ns	0.76	0.09	False
Parse - Duration of single invocation	15.19 ns	11.75 ns	0.77	0.06	False
ToString - Duration of single invocation	23.61 ns	19.38 ns	0.82	0.07	False
TryParse - Duration of single invocation	14.30 ns	10.81 ns	0.76	0.10	False
TryParse - Duration of single invocation	15.40 ns	11.29 ns	0.73	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.Parse(value: "127")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.896206659431694 < 13.34392844466415.
IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 22.3098319343113 (T) = (0 -10.90672343785299) / Math.Sqrt((0.8839186652626748 / (299)) + (0.012474344833238244 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11568700566773116 = (12.33355554849501 - 10.90672343785299) / 12.33355554849501 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.754130993184825 < 14.425373478705724.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.7516831750983 (T) = (0 -11.72105090594174) / Math.Sqrt((1.1328880920134363 / (299)) + (0.025640869832293363 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10056237509871184 = (13.031532794981871 - 11.72105090594174) / 13.031532794981871 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.ToString(value: -128)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.37734087269667 < 22.456543344499725.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 22.02399245163783 (T) = (0 -19.35480259907698) / Math.Sqrt((0.2596853707762637 / (299)) + (0.2682267046751103 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1592085379961328 = (23.019742080810946 - 19.35480259907698) / 23.019742080810946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "127")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.812392110726767 < 13.445369483500178.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 17.954440793318604 (T) = (0 -10.761312411319212) / Math.Sqrt((0.9213382932600884 / (299)) + (0.052783311854749014 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12899869136095532 = (12.355104756540443 - 10.761312411319212) / 12.355104756540443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.288554332440631 < 14.474346833973009.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 18.35047846186695 (T) = (0 -11.629730440605634) / Math.Sqrt((1.086755756574139 / (299)) + (0.030222655416637767 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.111944168733155 = (13.095719921139853 - 11.629730440605634) / 13.095719921139853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	937e25a340484327fcb87cfc5f20905c10476308
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	66.08 ns	61.90 ns	0.94	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.900432957344066 < 62.58333092497641.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 41.902078433800625 (T) = (0 -61.93454743446859) / Math.Sqrt((0.5892251425112784 / (299)) + (0.12169676669299663 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07170161965742398 = (66.71836205467953 - 61.93454743446859) / 66.71836205467953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FilteredByLevel_InsideScope - Duration of single invocation	102.11 ns	86.00 ns	0.84	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 86.00430495383073 < 95.76751525102225.
IsChangePoint: Marked as a change because one of 2/11/2023 4:30:46 AM, 2/16/2023 7:04:07 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 15.896280239750912 (T) = (0 -88.69704832182927) / Math.Sqrt((77.68823665135233 / (299)) + (8.249125994032937 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1526079340776387 = (104.67061456999264 - 88.69704832182927) / 104.67061456999264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	341.17 μs	317.25 μs	0.93	0.00	False
SortedDictionary - Duration of single invocation	354.36 μs	330.65 μs	0.93	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 317.25098214285714 < 323.3392520947484.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 58.54067992130407 (T) = (0 -318421.945175955) / Math.Sqrt((5958176.657238175 / (299)) + (1035586.9966979567 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.058462902612756884 = (338193.73241858766 - 318421.945175955) / 338193.73241858766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<String, String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 330.65480769230766 < 337.0072872305646.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 50.2762464184924 (T) = (0 -332200.65765905194) / Math.Sqrt((62238894.246273875 / (299)) + (584979.69205264 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07181224209326337 = (357902.43388712214 - 332200.65765905194) / 357902.43388712214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EmptyAsyncMethodInvocation - Duration of single invocation	18.61 ns	15.93 ns	0.86	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.Tests.Perf_AsyncMethods.EmptyAsyncMethodInvocation

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.92547307692308 < 17.700755747354496.
IsChangePoint: Marked as a change because one of 3/27/2023 2:09:01 AM, 3/27/2023 9:31:31 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 20.303864988147797 (T) = (0 -15.991091440330507) / Math.Sqrt((2.2674755257684005 / (299)) + (0.26752761616732756 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1848648401775945 = (19.617717684775744 - 15.991091440330507) / 19.617717684775744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Thread

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CurrentThread - Duration of single invocation	5.51 ns	1.00 ns	0.18	0.71	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Thread*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Thread.CurrentThread

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9990990610885883 < 4.560680610089777.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 59.187847259231035 (T) = (0 -1.279070679363623) / Math.Sqrt((0.5883981312657196 / (299)) + (0.01980295183721826 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.7396206034780621 = (4.912334449073266 - 1.279070679363623) / 4.912334449073266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToUpper - Duration of single invocation	33.41 ns	29.02 ns	0.87	0.04	False
ToUpper - Duration of single invocation	33.26 ns	28.97 ns	0.87	0.05	False
ToLower - Duration of single invocation	32.77 ns	28.79 ns	0.88	0.05	False
ToLower - Duration of single invocation	18.33 ns	14.55 ns	0.79	0.13	False
ToLower - Duration of single invocation	32.92 ns	28.85 ns	0.88	0.05	False
ToUpper - Duration of single invocation	18.44 ns	14.78 ns	0.80	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpper(s: "TeSt")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.02240899860086 < 31.032922793214933.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 41.58902314916873 (T) = (0 -28.902112739485645) / Math.Sqrt((0.4392589516731492 / (299)) + (0.11931801493457198 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13770319611617896 = (33.517592329357264 - 28.902112739485645) / 33.517592329357264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.969790067510353 < 31.17336720429699.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 33.47255482966644 (T) = (0 -29.356000107530186) / Math.Sqrt((0.448346845561942 / (299)) + (0.13903169589660003 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.12396804604620029 = (33.51019329265056 - 29.356000107530186) / 33.51019329265056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TEST")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.791766305324565 < 31.34471564125857.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 30.862778661325223 (T) = (0 -28.67889728814159) / Math.Sqrt((0.6486483072298569 / (299)) + (0.31611694012221275 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15909299308926633 = (34.104719133570015 - 28.67889728814159) / 34.104719133570015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.553567309705436 < 17.2944307682803.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 66.30356277021264 (T) = (0 -14.815664311529753) / Math.Sqrt((0.4116250187892966 / (299)) + (0.030628773962020697 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.22969352213962227 = (19.23346711646786 - 14.815664311529753) / 19.23346711646786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TeSt")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.848130039709634 < 31.108156380293725.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 28.552009084377023 (T) = (0 -29.34964890673109) / Math.Sqrt((0.6983985885257695 / (299)) + (0.24512887119862561 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.13748676840312954 = (34.02805641878988 - 29.34964890673109) / 34.02805641878988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "TEST")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.775811228831518 < 17.403200020717797.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 40.85133881283647 (T) = (0 -14.68739127134981) / Math.Sqrt((0.21086588393536057 / (299)) + (0.09061611569034331 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2082530007432988 = (18.55061185598235 - 14.68739127134981) / 18.55061185598235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableSortedDictionary - Duration of single invocation	341.88 μs	322.21 μs	0.94	0.01	False
SortedList - Duration of single invocation	339.52 μs	319.01 μs	0.94	0.01	False
SortedDictionary - Duration of single invocation	354.82 μs	332.80 μs	0.94	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.2122813411078 < 323.47314222631985.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 58.0245352277768 (T) = (0 -321852.70524543483) / Math.Sqrt((9356232.021914076 / (299)) + (870288.5566812012 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.056518412220471916 = (341133.0008070545 - 321852.70524543483) / 341133.0008070545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyFalse<String, String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 319.0052295918367 < 322.5250683795937.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/4/2023 7:16:27 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 40.82046487871538 (T) = (0 -317413.442701856) / Math.Sqrt((50190229.0452643 / (299)) + (1236774.6802968532 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06374607666272733 = (339024.9533699547 - 317413.442701856) / 339024.9533699547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyFalse<String, String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 332.79517730496457 < 335.9907969907407.
IsChangePoint: Marked as a change because one of 3/17/2023 7:47:41 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 49.68159896875084 (T) = (0 -332697.248594712) / Math.Sqrt((48279003.422074586 / (299)) + (838707.6555800937 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06887146053859805 = (357305.3928592458 - 332697.248594712) / 357305.3928592458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	392.22 μs	366.97 μs	0.94	0.03	False
SortedSet - Duration of single invocation	360.53 μs	337.62 μs	0.94	0.03	False
ImmutableSortedSet - Duration of single invocation	380.65 μs	351.09 μs	0.92	0.10	False
SortedDictionary - Duration of single invocation	350.02 μs	328.55 μs	0.94	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 366.9679263565892 < 372.5899360305059.
IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 43.682759637219824 (T) = (0 -367326.4848765575) / Math.Sqrt((102852389.22987644 / (299)) + (1881786.5484476709 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07863538608028649 = (398676.5709547489 - 367326.4848765575) / 398676.5709547489 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 337.6170307750759 < 343.4538860899275.
IsChangePoint: Marked as a change because one of 2/10/2023 11:47:22 AM, 2/12/2023 9:02:44 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 31.783837528898946 (T) = (0 -340313.7758597977) / Math.Sqrt((34904174.08089938 / (299)) + (4018705.8820720697 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06301971358155738 = (363202.70638844394 - 340313.7758597977) / 363202.70638844394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 351.09415674603173 < 357.17504365079367.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 25.2489762858842 (T) = (0 -352031.3164844415) / Math.Sqrt((59119944.94842768 / (299)) + (8073218.306336943 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06708102256578707 = (377343.93339561566 - 352031.3164844415) / 377343.93339561566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 328.5492287660257 < 332.3093971153846.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 21.197597287610233 (T) = (0 -329777.015880839) / Math.Sqrt((51087590.80050543 / (299)) + (9199305.205293667 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.060599193427344135 = (351050.3861328473 - 329777.015880839) / 351050.3861328473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	22.16 ns	17.64 ns	0.80	0.03	False
Parse - Duration of single invocation	11.56 ns	8.35 ns	0.72	0.21	False
TryParse - Duration of single invocation	15.52 ns	11.35 ns	0.73	0.08	False
ParseSpan - Duration of single invocation	16.61 ns	12.61 ns	0.76	0.10	False
ParseSpan - Duration of single invocation	23.53 ns	19.46 ns	0.83	0.04	False
Parse - Duration of single invocation	23.51 ns	19.41 ns	0.83	0.06	False
ParseSpan - Duration of single invocation	11.74 ns	9.22 ns	0.79	0.21	False
Parse - Duration of single invocation	16.67 ns	12.82 ns	0.77	0.10	False
TryParse - Duration of single invocation	9.86 ns	7.32 ns	0.74	0.43	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryParse(value: "4294967295")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.643840778002758 < 21.024178757981826.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 75.3404837196857 (T) = (0 -17.94992260641178) / Math.Sqrt((0.3497647030634955 / (299)) + (0.026160678636451544 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2000070584003754 = (22.43760122498086 - 17.94992260641178) / 22.43760122498086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.348164149412849 < 11.025031189937405.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 7.192741630624318 (T) = (0 -8.73110355151127) / Math.Sqrt((0.9768426057613145 / (299)) + (0.637363417588573 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1757624351197311 = (10.59294543652542 - 8.73110355151127) / 10.59294543652542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.348632753963011 < 14.707764074868395.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 55.63649063358901 (T) = (0 -11.366167362255633) / Math.Sqrt((0.313567120318 / (299)) + (0.023251292084063853 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.21585241976900044 = (14.494933924182115 - 11.366167362255633) / 14.494933924182115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.611084411644331 < 15.883604826316917.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 55.143666087529766 (T) = (0 -12.85376544484571) / Math.Sqrt((0.19620757579802756 / (299)) + (0.024818170006722966 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1937578418503371 = (15.942809880278752 - 12.85376544484571) / 15.942809880278752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "4294967295")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.464389171269833 < 22.276741188346847.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 74.04019196370909 (T) = (0 -19.338201202686495) / Math.Sqrt((0.3587189004862076 / (299)) + (0.024398419439119332 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18289455821875508 = (23.66671449468049 - 19.338201202686495) / 23.66671449468049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.40963903977307 < 22.312712303231514.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 39.060452520514275 (T) = (0 -19.42850372186551) / Math.Sqrt((0.364962901330944 / (299)) + (0.11724662802458949 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.17974064265965314 = (23.685805651594805 - 19.42850372186551) / 23.685805651594805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.219330636057835 < 11.042079314268033.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 22.69018641407821 (T) = (0 -8.504984692492293) / Math.Sqrt((0.8074236465357894 / (299)) + (0.061351914303164834 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.195320023649089 = (10.569400186967464 - 8.504984692492293) / 10.569400186967464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.820712877728207 < 15.846550643235469.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 84.72623042433003 (T) = (0 -12.842757822589318) / Math.Sqrt((0.374165892198346 / (299)) + (0.0006377448586407309 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1927156358369813 = (15.908592303660573 - 12.842757822589318) / 15.908592303660573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.315573737463702 < 9.816720019439895.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 4.363826842391926 (T) = (0 -7.74698420870467) / Math.Sqrt((0.9079524784855543 / (299)) + (1.5448521039781054 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1827165657727853 = (9.478944371397738 - 7.74698420870467) / 9.478944371397738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteAsyncThenReadAsync - Duration of single invocation	72.02 ns	65.88 ns	0.91	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 65.8771694521901 < 68.4148525906287.
IsChangePoint: Marked as a change because one of 2/8/2023 4:01:29 AM, 4/8/2023 1:46:22 AM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 61.623602547634555 (T) = (0 -65.99028472310862) / Math.Sqrt((4.374371906985948 / (299)) + (0.05147360968221532 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.11608162614124695 = (74.65653693228197 - 65.99028472310862) / 74.65653693228197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	18.82 ns	14.65 ns	0.78	0.08	False
TryParse - Duration of single invocation	11.66 ns	8.30 ns	0.71	0.32	False
TryParse - Duration of single invocation	16.19 ns	14.17 ns	0.87	0.14	False
TryParse - Duration of single invocation	18.65 ns	14.04 ns	0.75	0.10	False
Parse - Duration of single invocation	11.78 ns	8.28 ns	0.70	0.33	False
Parse - Duration of single invocation	18.07 ns	13.96 ns	0.77	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.654890071773526 < 17.69499607973247.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 10.711636750990962 (T) = (0 -14.468294449884167) / Math.Sqrt((1.82022369987674 / (299)) + (0.051850024207251454 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07144701241061971 = (15.581549618881049 - 14.468294449884167) / 15.581549618881049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.295693313463246 < 11.094842398075707.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 21.42117407407701 (T) = (0 -7.739522369311266) / Math.Sqrt((1.3023803898823323 / (299)) + (0.054630315657934345 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.21087827607546092 = (9.807767464340355 - 7.739522369311266) / 9.807767464340355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.165524091545402 < 17.309132927584812.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 15.406657274864088 (T) = (0 -14.1775727015558) / Math.Sqrt((1.7820920981023178 / (299)) + (0.02313748768169286 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08890649671887263 = (15.56105125379339 - 14.1775727015558) / 15.56105125379339 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.044079568045808 < 17.679417606697047.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 17.29937782277148 (T) = (0 -13.928767863495757) / Math.Sqrt((1.6674467170620297 / (299)) + (0.02716269037752801 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1002390842534344 = (15.480521124812954 - 13.928767863495757) / 15.480521124812954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.277048761344641 < 11.207426918841335.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 15.502535954109613 (T) = (0 -8.110928174253239) / Math.Sqrt((1.1730694497651997 / (299)) + (0.06319278138288505 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.15820164648527527 = (9.635238819829032 - 8.110928174253239) / 9.635238819829032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.959820032065018 < 17.546790897960324.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 12.010156040763809 (T) = (0 -14.351310289489387) / Math.Sqrt((1.6962665303386708 / (299)) + (0.04916267832712863 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.0792895319248606 = (15.587213121941145 - 14.351310289489387) / 15.587213121941145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableSortedDictionary - Duration of single invocation	261.60 μs	246.57 μs	0.94	0.01	False
SortedDictionary - Duration of single invocation	272.76 μs	258.26 μs	0.95	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<String, String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<String, String>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 246.57044020432693 < 247.27412844986017.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 3/6/2023 6:03:21 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 61.410764220755794 (T) = (0 -246769.40013111886) / Math.Sqrt((8061459.243819224 / (299)) + (390919.2888155619 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05857055162893462 = (262122.0321481323 - 246769.40013111886) / 262122.0321481323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<String, String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.257962529274 < 259.23523641965073.
IsChangePoint: Marked as a change because one of 3/17/2023 5:40:49 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 38.51665280159348 (T) = (0 -257198.90849251888) / Math.Sqrt((52690055.3019856 / (299)) + (829216.051223141 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06986869335896315 = (276518.9244315792 - 257198.90849251888) / 276518.9244315792 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Get - Duration of single invocation	39.20 ms	36.66 ms	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 40)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.65797333333334 < 37.51255822756411.
IsChangePoint: Marked as a change because one of 3/27/2023 4:20:55 AM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 9.716880541942885 (T) = (0 -35559253.3360687) / Math.Sqrt((1057061938211.7935 / (299)) + (375765054305.8864 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.050381355901836185 = (37445824.76035809 - 35559253.3360687) / 37445824.76035809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_ThreadStatic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetThreadStatic - Duration of single invocation	4.97 ns	1.50 ns	0.30	0.71	False
SetThreadStatic - Duration of single invocation	6.63 ns	3.68 ns	0.56	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_ThreadStatic*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_ThreadStatic.GetThreadStatic

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.499779728488368 < 4.582705476335426.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 14.764530463640897 (T) = (0 -0.9940037752183336) / Math.Sqrt((0.7059124607645259 / (299)) + (0.4403224238842345 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.7535847221742126 = (4.033856114721435 - 0.9940037752183336) / 4.033856114721435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_ThreadStatic.SetThreadStatic

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.681492916876427 < 6.322056101090454.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 80.03953460007988 (T) = (0 -3.734910348616386) / Math.Sqrt((0.26868822243252327 / (299)) + (0.0035678878454165957 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4315943937980624 = (6.5708541715007005 - 3.734910348616386) / 6.5708541715007005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableSortedDictionary - Duration of single invocation	263.43 μs	245.08 μs	0.93	0.02	False
SortedDictionary - Duration of single invocation	273.31 μs	258.55 μs	0.95	0.12	False
SortedList - Duration of single invocation	266.97 μs	249.71 μs	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<String, String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<String, String>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 245.0791015625 < 250.04582712339734.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 68.55860607156532 (T) = (0 -245646.2480553301) / Math.Sqrt((11793771.791112322 / (299)) + (364536.6673324189 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06993354223119758 = (264116.876813972 - 245646.2480553301) / 264116.876813972 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueTrue<String, String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.5524077868853 < 259.2563150350906.
IsChangePoint: Marked as a change because one of 3/23/2023 6:11:55 PM, 3/25/2023 12:18:49 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 39.53704131363028 (T) = (0 -257591.674930261) / Math.Sqrt((69174854.61837074 / (299)) + (534338.6705401536 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07510833616233735 = (278510.10556353506 - 257591.674930261) / 278510.10556353506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueTrue<String, String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 249.7066881613756 < 253.67413909537552.
IsChangePoint: Marked as a change because one of 3/9/2023 2:25:45 AM, 3/14/2023 8:42:09 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 43.24896321986152 (T) = (0 -249420.56328674624) / Math.Sqrt((10866143.740514664 / (299)) + (1166980.3481541083 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06352458154720413 = (266339.6800087162 - 249420.56328674624) / 266339.6800087162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Lock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReaderWriterLockSlimPerf - Duration of single invocation	27.69 ns	21.68 ns	0.78	0.29	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Lock*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Lock.ReaderWriterLockSlimPerf

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.67762680247809 < 26.363521964789072.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 26.1761892166537 (T) = (0 -21.987463960590265) / Math.Sqrt((7.858234588194551 / (299)) + (1.12513132737557 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.299165000094897 = (31.373239012845378 - 21.987463960590265) / 31.373239012845378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	339.80 μs	319.54 μs	0.94	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 319.53902150145774 < 322.7091444136705.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 34.89125483448493 (T) = (0 -319357.69319541) / Math.Sqrt((55870386.14937839 / (299)) + (1954427.5885757182 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.061882078631550645 = (340423.8272407772 - 319357.69319541) / 340423.8272407772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Linq.Perf_XElement

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetValue - Duration of single invocation	80.45 ns	74.41 ns	0.92	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.GetValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 74.41289383930197 < 76.65652852159529.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 23.791988080709153 (T) = (0 -73.74256251757403) / Math.Sqrt((2.128110155893803 / (299)) + (0.7384368122631407 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.08080917953259746 = (80.22552105130512 - 73.74256251757403) / 80.22552105130512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableSortedSet - Duration of single invocation	342.02 μs	323.11 μs	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 323.11346574344026 < 323.9535956909937.
IsChangePoint: Marked as a change because one of 2/6/2023 10:25:44 PM, 2/9/2023 1:41:04 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 63.361890955312965 (T) = (0 -322117.53565312235) / Math.Sqrt((6521322.390707422 / (299)) + (708639.3325696208 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05460849304959041 = (340723.957519135 - 322117.53565312235) / 340723.957519135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableSortedSet - Duration of single invocation	260.38 μs	245.25 μs	0.94	0.01	False
SortedSet - Duration of single invocation	250.04 μs	233.61 μs	0.93	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 245.2466099330357 < 246.90945961407104.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 75.9202238380503 (T) = (0 -245575.20404725484) / Math.Sqrt((4372438.507621697 / (299)) + (269605.3230056825 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.05763225198117871 = (260593.81230261514 - 245575.20404725484) / 260593.81230261514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 233.61228855721393 < 238.29730825439492.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 43.499026396517635 (T) = (0 -234617.547780267) / Math.Sqrt((7306383.644168705 / (299)) + (1312988.1802710632 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06568832553493008 = (251112.72200959572 - 234617.547780267) / 251112.72200959572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionary - Duration of single invocation	632.45 μs	583.10 μs	0.92	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 583.1043981481482 < 590.3611155448718.
IsChangePoint: Marked as a change because one of 2/7/2023 2:48:42 AM, 2/9/2023 4:27:10 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 20.519320055065723 (T) = (0 -578033.6145239791) / Math.Sqrt((244061924.25489935 / (299)) + (21370260.253163807 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.055680797988566705 = (612116.7644295987 - 578033.6145239791) / 612116.7644295987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringD - Duration of single invocation	33.15 ns	28.03 ns	0.85	0.03	False
ToStringX - Duration of single invocation	55.19 ns	51.08 ns	0.93	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.025558768487862 < 30.859990761842532.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 31.907386977649914 (T) = (0 -28.340027557381177) / Math.Sqrt((0.40048661290670784 / (299)) + (0.12106506636019462 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.11547699512530833 = (32.03989879426148 - 28.340027557381177) / 32.03989879426148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 51.084151618329585 < 52.55304417251914.
IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.252140566320477 (T) = (0 -51.335747372928054) / Math.Sqrt((0.650630945313308 / (299)) + (0.28403820553332126 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.05245743375511315 = (54.177774383658274 - 51.335747372928054) / 54.177774383658274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchmark.GetChildKeysTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AddChainedConfigurationNoDelimiter - Duration of single invocation	915.74 μs	797.38 μs	0.87	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 797.3801116071428 < 867.759252744868.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 81.17144978156331 (T) = (0 -794935.7066353785) / Math.Sqrt((351207089.38473207 / (299)) + (1557454.3102165433 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10536585617162575 = (888559.5437187765 - 794935.7066353785) / 888559.5437187765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Perf_Convert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ChangeType - Duration of single invocation	34.14 ns	30.17 ns	0.88	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ChangeType

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.16704162419881 < 32.47676147344208.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 23.61768578393713 (T) = (0 -30.05670832944332) / Math.Sqrt((0.6200942535097452 / (299)) + (0.21287413179479037 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.10315382190573501 = (33.51378314764268 - 30.05670832944332) / 33.51378314764268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSet<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	265.12 μs	249.51 μs	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 249.50789399092974 < 252.3806032933565.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/26/2023 5:34:01 AM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 53.735219297073655 (T) = (0 -250209.22193904192) / Math.Sqrt((8303008.3030517185 / (299)) + (689166.4028787337 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.060661335041676756 = (266367.4255867486 - 250209.22193904192) / 266367.4255867486 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f1819bd78f0e7506b8c3c7f90b47aa41b0e22c6f
Compare	aa38055cca67d9ece504cef2adc8a19f0cb2ec8e
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_CultureInfo

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetCurrentCulture - Duration of single invocation	6.02 ns	3.72 ns	0.62	0.45	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_CultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_CultureInfo.GetCurrentCulture

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.7221142668901464 < 6.001040444771132.
IsChangePoint: Marked as a change because one of 4/11/2023 8:14:56 PM, 5/12/2023 9:12:45 PM, 5/18/2023 3:09:06 AM falls between 5/9/2023 1:22:51 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 21.268431695664823 (T) = (0 -3.86792279075394) / Math.Sqrt((0.3352412520415596 / (299)) + (0.23531252596456942 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4520678424931101 = (7.0591271889445615 - 3.86792279075394) / 7.0591271889445615 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

From TLS : dotnet/runtime#85619
@JulieLeeMSFT