[Perf] Linux/arm64: 119 Regressions on 4/26/2023 12:55:36 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_SemaphoreSlim

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReleaseWaitAsync - Duration of single invocation	58.42 ns	64.19 ns	1.10	0.04	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_SemaphoreSlim*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.19114724610218 > 61.84225122140281.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -36.82424838548651 (T) = (0 -66.08155629876315) / Math.Sqrt((0.14000637926795506 / (11)) + (0.6048799617945516 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.11821870155906561 = (59.09537750229859 - 66.08155629876315) / 59.09537750229859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_FileStream

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetLength - Duration of single invocation	334.13 ns	361.65 ns	1.08	0.08	False
SeekForward - Duration of single invocation	24.22 μs	27.63 μs	1.14	0.08	False
GetLength - Duration of single invocation	333.30 ns	359.72 ns	1.08	0.04	False
ReadByte - Duration of single invocation	7.11 μs	12.13 μs	1.71	0.42	True
ReadByte - Duration of single invocation	7.04 μs	12.18 μs	1.73	0.42	True
SeekForward - Duration of single invocation	23.90 μs	27.49 μs	1.15	0.07	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.GetLength(options: Read)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 361.6512722754663 > 351.89909437428355.
IsChangePoint: Marked as a change because one of 1/28/2023 8:08:27 PM, 1/29/2023 5:12:48 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -27.8362056575774 (T) = (0 -362.00516666654863) / Math.Sqrt((4.340900060608027 / (11)) + (12.52641140150423 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07757023249138374 = (335.9457748100361 - 362.00516666654863) / 335.9457748100361 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: Asynchronous)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.62703268045775 > 25.46831462410829.
IsChangePoint: Marked as a change because one of 4/5/2023 11:58:06 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -31.02688674702847 (T) = (0 -27729.815230607994) / Math.Sqrt((122788.13592072537 / (11)) + (43117.20385967708 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.14507195325330538 = (24216.65743521515 - 27729.815230607994) / 24216.65743521515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.GetLength(options: Write)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 359.72049759265985 > 351.4655885447626.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -21.56354010348023 (T) = (0 -362.0372023231808) / Math.Sqrt((14.063089885347058 / (11)) + (3.394203750606319 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07607958571457446 = (336.4409167587439 - 362.0372023231808) / 336.4409167587439 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.ReadByte(fileSize: 1024, options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.12966086359293 > 7.973734423011042.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.241135333707614 (T) = (0 -11224.874845944814) / Math.Sqrt((336131.27847019036 / (11)) + (331132.1036511864 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.4697040172427272 = (7637.507085952929 - 11224.874845944814) / 7637.507085952929 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.ReadByte(fileSize: 1024, options: Asynchronous)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.177366296332549 > 7.841238746452231.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -18.959126930284423 (T) = (0 -11131.094630800133) / Math.Sqrt((220273.55624999976 / (11)) + (353317.7858000321 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.4540747961323622 = (7655.104579494332 - 11131.094630800133) / 7655.104579494332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.49117491930751 > 25.399375676464793.
IsChangePoint: Marked as a change because one of 4/5/2023 12:23:38 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -27.300094212972528 (T) = (0 -27633.853064271676) / Math.Sqrt((144753.97482632898 / (11)) + (96817.96272357549 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.14727065858771082 = (24086.603154558947 - 27633.853064271676) / 24086.603154558947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.GetMember

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetField - Duration of single invocation	52.35 μs	67.11 μs	1.28	0.00	True
GetMethod5 - Duration of single invocation	415.01 μs	596.73 μs	1.44	0.04	True
GetMethod15 - Duration of single invocation	1.25 ms	1.77 ms	1.42	0.03	True
GetMethod3 - Duration of single invocation	249.55 μs	457.92 μs	1.83	0.03	True
GetMethod2 - Duration of single invocation	166.14 μs	240.52 μs	1.45	0.05	True
GetMethod1 - Duration of single invocation	83.44 μs	151.20 μs	1.81	0.04	True
GetMethod4 - Duration of single invocation	331.77 μs	493.48 μs	1.49	0.04	True
GetMethod12 - Duration of single invocation	1.00 ms	1.43 ms	1.43	0.08	True
GetMethod10 - Duration of single invocation	833.28 μs	1.47 ms	1.76	0.05	True
GetMethod20 - Duration of single invocation	1.67 ms	2.39 ms	1.43	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.GetMember.GetField

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.11351855244253 > 54.81237645632366.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -127.05021915049316 (T) = (0 -67603.24289362694) / Math.Sqrt((72833.97916043708 / (11)) + (213613.24823699595 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.2968819537053646 = (52127.522247090776 - 67603.24289362694) / 52127.522247090776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod5

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 596.7277532407408 > 435.6290904851975.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.971007944799208 (T) = (0 -635367.422779123) / Math.Sqrt((12211414.805523174 / (11)) + (2145176363.6335711 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.5266977811756309 = (416171.05272129196 - 635367.422779123) / 416171.05272129196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod15

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7749548870370366 > 1.316702124206731.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.426534594115704 (T) = (0 -1893687.6678848518) / Math.Sqrt((39761936.56444727 / (11)) + (19179566392.527542 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.5078791934085563 = (1255861.6606441638 - 1893687.6678848518) / 1255861.6606441638 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 457.9151302631578 > 262.0025388194445.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -21.06978576335206 (T) = (0 -380745.27054343646) / Math.Sqrt((2363188.060174794 / (11)) + (997045665.5394063 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.5235649194657359 = (249904.1988161235 - 380745.27054343646) / 249904.1988161235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod2

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 240.51737705128207 > 174.6510689428192.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -26.99502230055446 (T) = (0 -256055.52370371638) / Math.Sqrt((63612.89744323521 / (11)) + (286544098.1258799 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.5386608623737205 = (166414.5296506046 - 256055.52370371638) / 166414.5296506046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod1

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.19505973907766 > 87.607700702645.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -27.98634190764289 (T) = (0 -132365.4468772548) / Math.Sqrt((604589.0675973337 / (11)) + (77377006.03105634 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.582526231695755 = (83641.8659142343 - 132365.4468772548) / 83641.8659142343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod4

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 493.4841303030304 > 348.82827681682187.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.199801991623836 (T) = (0 -516324.6345099368) / Math.Sqrt((56663.055895210404 / (11)) + (1637575129.0929403 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.554266479659898 = (332198.26925876836 - 516324.6345099368) / 332198.26925876836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod12

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4334023858901517 > 1.0523409789648437.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.2236293956973 (T) = (0 -1541405.460345359) / Math.Sqrt((4575342.596969547 / (11)) + (13989119845.80851 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.5375498199000537 = (1002507.6523670342 - 1541405.460345359) / 1002507.6523670342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod10

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4654854187499997 > 875.4022838733553.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -25.82631302834375 (T) = (0 -1292446.0595018594) / Math.Sqrt((507193.8442822412 / (11)) + (8183712389.681681 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.549292588977871 = (834216.8991814107 - 1292446.0595018594) / 834216.8991814107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.GetMember.GetMethod20

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3877831154761906 > 1.7581455720937502.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.983630133335105 (T) = (0 -2552171.6311489735) / Math.Sqrt((3001973.728785325 / (11)) + (34795360657.59229 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.5239577011642215 = (1674699.7828084414 - 2552171.6311489735) / 1674699.7828084414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetCustomAttributesClassMiss - Duration of single invocation	182.25 ns	197.96 ns	1.09	0.08	False
GetCustomAttributesClassHit - Duration of single invocation	1.55 μs	1.77 μs	1.14	0.20	False
GetCustomAttributesClassMissInherit - Duration of single invocation	357.12 ns	384.21 ns	1.08	0.19	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesClassMiss

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 197.95557241205117 > 190.95426152328517.
IsChangePoint: Marked as a change because one of 3/9/2023 3:58:44 PM, 3/22/2023 1:13:01 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -21.281654047163148 (T) = (0 -196.74306097765177) / Math.Sqrt((2.0814310620192806 / (11)) + (8.026394771968397 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08263654041445243 = (181.725864252037 - 196.74306097765177) / 181.725864252037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesClassHit

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7671189288151288 > 1.6986905288222207.
IsChangePoint: Marked as a change because one of 2/10/2023 11:58:08 PM, 4/25/2023 1:02:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -6.934536529508078 (T) = (0 -1752.2355098901676) / Math.Sqrt((3063.1914874203194 / (10)) + (1756.5470409120467 / (27))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (27) - 2, .025) and -0.08256297032101952 = (1618.5991558260723 - 1752.2355098901676) / 1618.5991558260723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesClassMissInherit

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 384.2098511607811 > 379.5269736884031.
IsChangePoint: Marked as a change because one of 4/27/2023 1:05:59 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -6.458186586176081 (T) = (0 -391.1607348534885) / Math.Sqrt((80.49000768703807 / (16)) + (186.41946601278508 / (21))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (21) - 2, .025) and -0.06561191760040314 = (367.07616383863586 - 391.1607348534885) / 367.07616383863586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	208.71 ns	228.37 ns	1.09	0.01	False
Remainder - Duration of single invocation	8.62 ns	11.72 ns	1.36	0.22	False
Parse - Duration of single invocation	136.35 ns	153.45 ns	1.13	0.01	False
Divide - Duration of single invocation	9.08 ns	13.92 ns	1.53	0.43	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 228.36928298037702 > 221.61985280829956.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 3/21/2023 3:33:39 AM, 4/9/2023 11:24:15 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -13.113528551221325 (T) = (0 -224.88311169594212) / Math.Sqrt((6.736572881904214 / (11)) + (9.13527073488572 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06072280819464049 = (212.00931097040814 - 224.88311169594212) / 212.00931097040814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 16,8 bits)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.72095175805165 > 8.996272410046968.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -34.07603255128446 (T) = (0 -11.756566613512874) / Math.Sqrt((0.17855806182524517 / (33)) + (0.016156525441122827 / (4))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (4) - 2, .025) and -0.39229813195893576 = (8.444000852728013 - 11.756566613512874) / 8.444000852728013 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 153.4488050001564 > 144.67333615134123.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -18.35549105093233 (T) = (0 -149.08869596791686) / Math.Sqrt((2.671888905390098 / (11)) + (3.1038167494257722 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.0800348583668139 = (138.04063342303868 - 149.08869596791686) / 138.04063342303868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.916232903912622 > 9.582363772257741.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 4:25:00 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -12.398974526385524 (T) = (0 -13.671451182219087) / Math.Sqrt((1.478956237125132 / (33)) + (0.08166118394757409 / (4))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (4) - 2, .025) and -0.30146283082831954 = (10.504680470603878 - 13.671451182219087) / 10.504680470603878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	14.02 ns	15.46 ns	1.10	0.15	False
Parse - Duration of single invocation	10.92 ns	11.93 ns	1.09	0.15	False
Parse - Duration of single invocation	15.50 ns	16.81 ns	1.08	0.04	False
ParseSpan - Duration of single invocation	10.84 ns	12.01 ns	1.11	0.12	False
ParseSpan - Duration of single invocation	15.52 ns	16.57 ns	1.07	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.457318799502419 > 14.7602990525108.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -11.55128072220686 (T) = (0 -15.337904724098292) / Math.Sqrt((0.0306127019223434 / (11)) + (0.2703301972073303 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09464551950449585 = (14.01175490220905 - 15.337904724098292) / 14.01175490220905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.932530441798772 > 11.180316402089133.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -22.690239040673355 (T) = (0 -11.960986522566603) / Math.Sqrt((0.013704635832765617 / (11)) + (0.05058426368865278 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.12003130409938176 = (10.679153768996166 - 11.960986522566603) / 10.679153768996166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.81366166144998 > 16.285095734711252.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.990650725743379 (T) = (0 -16.858386712779204) / Math.Sqrt((0.13730492079678994 / (11)) + (0.04367322982925861 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08410905223236677 = (15.550452860867539 - 16.858386712779204) / 15.550452860867539 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.005925645530509 > 11.355827574966966.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -3.729337037772544 (T) = (0 -12.014125940065822) / Math.Sqrt((0.7613148451160316 / (11)) + (0.2683797776135276 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09593987642049907 = (10.962395108120097 - 12.014125940065822) / 10.962395108120097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.570912070073213 > 16.31624085164388.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.672206696952333 (T) = (0 -16.908630886306153) / Math.Sqrt((0.0077969088620606294 / (11)) + (0.07371522595030053 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09092258114274174 = (15.499386646295612 - 16.908630886306153) / 15.499386646295612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse3 - Duration of single invocation	39.35 ns	53.68 ns	1.36	0.03	True
TryParse2 - Duration of single invocation	34.74 ns	37.83 ns	1.09	0.09	False
TryParse4 - Duration of single invocation	64.05 ns	70.56 ns	1.10	0.04	True
Parse3 - Duration of single invocation	43.72 ns	46.97 ns	1.07	0.31	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryParse3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.67839678308642 > 41.42718990686177.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -74.31457005540804 (T) = (0 -55.25347369791593) / Math.Sqrt((0.29401572250165414 / (11)) + (0.5223117474263433 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.41047194603875425 = (39.17374879599185 - 55.25347369791593) / 39.17374879599185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.TryParse2

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.83046250392331 > 36.07155754697808.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.946874540735738 (T) = (0 -38.225491126625585) / Math.Sqrt((0.38713544273734246 / (11)) + (0.4695370726252777 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.12150998066758066 = (34.08395091042507 - 38.225491126625585) / 34.08395091042507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.TryParse4

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.56103056865314 > 67.08812211430084.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.141855963570496 (T) = (0 -70.94492603990952) / Math.Sqrt((1.4558826887820953 / (11)) + (1.112482810119981 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.11249355310593152 = (63.771089586848284 - 70.94492603990952) / 63.771089586848284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.Parse3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.97442835755501 > 45.38810648753014.
IsChangePoint: Marked as a change because one of 4/9/2023 9:22:58 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -8.8930499257358 (T) = (0 -50.25567459152296) / Math.Sqrt((4.634272181765452 / (11)) + (1.2197657895191023 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.13776512758365242 = (44.170517599053404 - 50.25567459152296) / 44.170517599053404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	71.10 ns	76.22 ns	1.07	0.02	False
Parse - Duration of single invocation	71.24 ns	78.54 ns	1.10	0.03	False
TryParse - Duration of single invocation	108.78 ns	127.20 ns	1.17	0.03	False
Parse - Duration of single invocation	110.10 ns	126.89 ns	1.15	0.03	False
TryParse - Duration of single invocation	114.92 ns	134.32 ns	1.17	0.01	False
Parse - Duration of single invocation	114.30 ns	133.43 ns	1.17	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.21683924158273 > 73.44188954087461.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.00221251948057 (T) = (0 -76.30210336553432) / Math.Sqrt((0.4590182502106819 / (11)) + (2.5214948291398587 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09049980175469192 = (69.96984615931044 - 76.30210336553432) / 69.96984615931044 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.54329939568986 > 74.93215851424678.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -13.097648301925554 (T) = (0 -77.8902044991044) / Math.Sqrt((1.1909106629689807 / (11)) + (2.3509295831366317 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08102695448158491 = (72.05204659902053 - 77.8902044991044) / 72.05204659902053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.19757096225213 > 114.22005953840973.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -69.42861721598787 (T) = (0 -129.4494887963154) / Math.Sqrt((0.03638369310258279 / (11)) + (2.2160849138245458 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.18989868132456644 = (108.79034562187711 - 129.4494887963154) / 108.79034562187711 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.88674479188981 > 115.25731517590417.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -100.05873769713885 (T) = (0 -128.4951994477349) / Math.Sqrt((0.10602374969920728 / (11)) + (0.6694066043575607 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.17161822662963605 = (109.67326773105411 - 128.4951994477349) / 109.67326773105411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.32378758093313 > 122.90244500403072.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -33.841343182837015 (T) = (0 -136.4583463646072) / Math.Sqrt((3.217946829000051 / (11)) + (1.5610848040550906 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.17268710926610537 = (116.36381545117007 - 136.4583463646072) / 116.36381545117007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 133.43020874433563 > 121.1820223998552.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -31.668286486869533 (T) = (0 -136.02351757121053) / Math.Sqrt((3.2326901997727084 / (11)) + (2.758008182161966 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.17265917110570772 = (115.99578199943049 - 136.02351757121053) / 115.99578199943049 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor - Duration of single invocation	41.47 μs	46.10 μs	1.11	0.21	False
Ctor - Duration of single invocation	26.48 μs	30.14 μs	1.14	0.19	False
Ctor - Duration of single invocation	36.13 μs	41.01 μs	1.14	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.10247930588888 > 43.209094924838496.
IsChangePoint: Marked as a change because one of 3/21/2023 12:49:42 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -8.126066281308603 (T) = (0 -44794.300855167894) / Math.Sqrt((1226220.2263827904 / (11)) + (872819.811899648 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07421643903507029 = (41699.51159507947 - 44794.300855167894) / 41699.51159507947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.139593195147423 > 27.981529952826023.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/26/2023 8:38:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -6.81217883994678 (T) = (0 -29226.268087603094) / Math.Sqrt((624687.3361913629 / (15)) + (654048.4532367766 / (22))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (22) - 2, .025) and -0.06640631014387528 = (27406.315781889927 - 29226.268087603094) / 27406.315781889927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w]+://[^/\\s?#]+[^\\s?#]+(?:\?[^\\s#])?(?:#[^\\s])?", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.010776216885425 > 38.4406286326664.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -7.318021457293225 (T) = (0 -40046.69618101464) / Math.Sqrt((1050019.4055245635 / (11)) + (1461111.5177041658 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07661480532646466 = (37196.86556685534 - 40046.69618101464) / 37196.86556685534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BinarySearch - Duration of single invocation	3.71 ns	5.50 ns	1.48	0.43	False
SequenceCompareTo - Duration of single invocation	46.48 ns	51.92 ns	1.12	0.00	True
BinarySearch - Duration of single invocation	10.99 ns	13.77 ns	1.25	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.BinarySearch(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.501569521610614 > 4.13944403595241.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -14.967473845971499 (T) = (0 -5.848376434183) / Math.Sqrt((0.09270186337584269 / (11)) + (0.15509884834923787 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.4430709170834297 = (4.052729748031421 - 5.848376434183) / 4.052729748031421 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.91982507683124 > 48.26984190939125.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -28.925235052098053 (T) = (0 -52.01880976799109) / Math.Sqrt((0.1864181613777155 / (11)) + (0.626338568680096 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.1269422581877593 = (46.159250298806576 - 52.01880976799109) / 46.159250298806576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.770681254030318 > 11.49407840078311.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -22.895824392756914 (T) = (0 -12.89547669802718) / Math.Sqrt((0.005975200922774352 / (11)) + (0.1767825979952219 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.1794392028260287 = (10.933566280592173 - 12.89547669802718) / 10.933566280592173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString_Flags - Duration of single invocation	31.55 ns	38.08 ns	1.21	0.03	False
ToString_Format_NonFlags - Duration of single invocation	36.64 ns	42.52 ns	1.16	0.03	False
ToString_Flags - Duration of single invocation	33.13 ns	36.92 ns	1.11	0.10	False
StringFormat - Duration of single invocation	286.15 ns	309.06 ns	1.08	0.24	False
ToString_NonFlags_Large - Duration of single invocation	20.61 ns	23.57 ns	1.14	0.48	False
TryParseGeneric_Flags - Duration of single invocation	111.93 ns	118.65 ns	1.06	0.01	False
Parse_Flags - Duration of single invocation	229.83 ns	245.96 ns	1.07	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_Flags(value: 36)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.08038425486293 > 35.613435409614794.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -14.049304051985875 (T) = (0 -38.34103155608217) / Math.Sqrt((0.9377195002901377 / (11)) + (0.17292991136441777 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.12496356213949981 = (34.08202082844683 - 38.34103155608217) / 34.08202082844683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: 8, format: "F")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.51561541140073 > 37.56202580831903.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -23.73125355550672 (T) = (0 -41.85619359528868) / Math.Sqrt((0.3948228259163354 / (11)) + (0.6655540839737821 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.16359470072418983 = (35.971454295244314 - 41.85619359528868) / 35.971454295244314 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.923883872050155 > 34.50779419321232.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.605044127615585 (T) = (0 -37.79585427094902) / Math.Sqrt((0.7935726421947127 / (11)) + (0.31821085945969435 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.15647864746173965 = (32.68184358950686 - 37.79585427094902) / 32.68184358950686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.StringFormat(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 309.05987798243274 > 300.4402182669086.
IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -43.873711153100565 (T) = (0 -309.47115947293224) / Math.Sqrt((0.6099057793720501 / (11)) + (5.7581046558943685 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.0806169764670739 = (286.3837661376605 - 309.47115947293224) / 286.3837661376605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.56857959429864 > 21.81222585533668.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -5.112979838498916 (T) = (0 -24.076552921189624) / Math.Sqrt((2.2158999594604225 / (9)) + (0.17474865291032757 / (28))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (28) - 2, .025) and -0.11944625541112716 = (21.50755590526075 - 24.076552921189624) / 21.50755590526075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.64844347727883 > 117.44994377726243.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -43.89806657534979 (T) = (0 -119.6504567743813) / Math.Sqrt((0.05850026599256715 / (11)) + (0.6731203635923194 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06930452366457424 = (111.89558645495262 - 119.6504567743813) / 111.89558645495262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 245.95789249261477 > 242.8912061783117.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -13.26367525374101 (T) = (0 -245.82427445244198) / Math.Sqrt((10.4459784535523 / (11)) + (8.034172818195636 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06443300601368497 = (230.94386688839816 - 245.82427445244198) / 230.94386688839816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsPrefix_FirstHalf - Duration of single invocation	193.25 ns	210.51 ns	1.09	0.01	False
IsPrefix_FirstHalf - Duration of single invocation	191.92 ns	210.44 ns	1.10	0.01	False
LastIndexOf_Word_NotFound - Duration of single invocation	747.62 ns	906.19 ns	1.21	0.01	False
IsPrefix_FirstHalf - Duration of single invocation	191.85 ns	210.49 ns	1.10	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.51157230298395 > 202.84920947210034.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.058646080176914 (T) = (0 -213.21030061221973) / Math.Sqrt((0.047889835592676 / (11)) + (35.56830337354596 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10342198571480608 = (193.2264386358953 - 213.21030061221973) / 193.2264386358953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.4440282706 > 201.42653421633185.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -19.39599454190165 (T) = (0 -213.01449616832133) / Math.Sqrt((0.1524581532527637 / (11)) + (30.17725665741243 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10948515965070782 = (191.99400218690923 - 213.01449616832133) / 191.99400218690923 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 906.1878769473274 > 785.0068455000356.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/28/2023 2:42:21 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -1945.4766491637592 (T) = (0 -905.4893480923976) / Math.Sqrt((0.023039109572212224 / (11)) + (0.11677721322604775 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.21118353904765547 = (747.6070462486446 - 905.4893480923976) / 747.6070462486446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.48516217901468 > 201.52981186174765.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -18.753574920007765 (T) = (0 -212.696201110128) / Math.Sqrt((0.027247725318501163 / (11)) + (31.573150103572445 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10773999341905748 = (192.00913786062532 - 212.696201110128) / 192.00913786062532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	12.02 ns	15.08 ns	1.25	0.36	False
TryParse - Duration of single invocation	9.32 ns	11.10 ns	1.19	0.18	False
Parse - Duration of single invocation	9.63 ns	11.09 ns	1.15	0.26	False
ToString - Duration of single invocation	2.04 ns	3.79 ns	1.86	0.52	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.079321043908388 > 12.651082503115752.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -7.6165105375153646 (T) = (0 -14.675367574235919) / Math.Sqrt((0.7016005985597491 / (11)) + (0.47340939557431005 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.1745495106653485 = (12.494464848844682 - 14.675367574235919) / 12.494464848844682 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.096229529596094 > 10.18935038850587.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.490758975642615 (T) = (0 -10.975162584992656) / Math.Sqrt((0.11515981357409481 / (11)) + (0.13259640171986362 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.13541969473280435 = (9.666172461078734 - 10.975162584992656) / 9.666172461078734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.092047105279644 > 9.898672487590769.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -11.997730487787797 (T) = (0 -10.943458989406926) / Math.Sqrt((0.11548573152516325 / (11)) + (0.04222829896096202 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.13728262806395664 = (9.62246210340558 - 10.943458989406926) / 9.62246210340558 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.ToString(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7928964649225816 > 1.8171853724337987.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -15.163318034009018 (T) = (0 -3.3601769521413583) / Math.Sqrt((0.029231027622637747 / (11)) + (0.17277445507895728 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.770676239277596 = (1.8976800374936131 - 3.3601769521413583) / 1.8976800374936131 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	68.18 ns	75.64 ns	1.11	0.01	False
Parse - Duration of single invocation	156.91 ns	183.75 ns	1.17	0.01	True
TryParse - Duration of single invocation	158.55 ns	183.92 ns	1.16	0.01	True
Parse - Duration of single invocation	154.23 ns	175.73 ns	1.14	0.00	True
Parse - Duration of single invocation	69.09 ns	77.75 ns	1.13	0.02	False
TryParse - Duration of single invocation	154.07 ns	176.89 ns	1.15	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.63816792229542 > 71.66821747306744.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -29.32536880954152 (T) = (0 -75.26666132657962) / Math.Sqrt((0.12379824798693462 / (11)) + (1.2051633233081496 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10316129992033495 = (68.22815605661205 - 75.26666132657962) / 68.22815605661205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 183.7536724642065 > 165.8982567240247.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -114.75256897275666 (T) = (0 -184.5427246255353) / Math.Sqrt((0.2965135592465824 / (11)) + (0.7169267745868182 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.16987169128676452 = (157.74612378435552 - 184.5427246255353) / 157.74612378435552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 183.91652851600034 > 165.42826162770476.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -150.39166687101275 (T) = (0 -184.71467842035997) / Math.Sqrt((0.17256030404794026 / (11)) + (0.42957335991386386 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.17112645537484145 = (157.72393969295015 - 184.71467842035997) / 157.72393969295015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.73405189794892 > 162.06728384695634.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -61.16294436471338 (T) = (0 -177.03060772560318) / Math.Sqrt((1.2851740414182367 / (11)) + (0.4552724639468471 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.144995193469877 = (154.61253351563573 - 177.03060772560318) / 154.61253351563573 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 77.75299723339637 > 73.29170021557137.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.57541979788023 (T) = (0 -76.84206133295692) / Math.Sqrt((1.0482650206883766 / (11)) + (1.2510311693753267 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09483060018864342 = (70.1862565037155 - 76.84206133295692) / 70.1862565037155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 176.8863709759397 > 161.19869568091767.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -94.08598533647047 (T) = (0 -178.02116615689582) / Math.Sqrt((0.6093883196253708 / (11)) + (0.3145316090536 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.15916117233695343 = (153.57757868820966 - 178.02116615689582) / 153.57757868820966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	15.82 ns	18.13 ns	1.15	0.10	False
TryParse - Duration of single invocation	16.07 ns	18.31 ns	1.14	0.07	False
Parse - Duration of single invocation	9.56 ns	11.96 ns	1.25	0.21	False
TryParse - Duration of single invocation	9.48 ns	11.42 ns	1.20	0.22	False
Parse - Duration of single invocation	16.45 ns	18.13 ns	1.10	0.06	False
Parse - Duration of single invocation	15.99 ns	18.52 ns	1.16	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.132615835124156 > 17.13495176546312.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -17.915373446364047 (T) = (0 -17.75081703623944) / Math.Sqrt((0.04596287535216767 / (11)) + (0.07158935580628255 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09173850603918624 = (16.259220443400118 - 17.75081703623944) / 16.259220443400118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.31485888767501 > 17.22760443408676.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.174118289578523 (T) = (0 -17.753304186751112) / Math.Sqrt((0.1465760081920061 / (11)) + (0.07870819691432872 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07907889816145644 = (16.45227630435489 - 17.753304186751112) / 16.45227630435489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.963128716950406 > 9.87751944520196.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.737250815954859 (T) = (0 -10.895228691197117) / Math.Sqrt((0.07173234928583501 / (12)) + (0.25110319674642073 / (25))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (25) - 2, .025) and -0.142520269430466 = (9.536136016762548 - 10.895228691197117) / 9.536136016762548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.418259406446014 > 10.08915740910329.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -4.110367643994692 (T) = (0 -10.942051352034532) / Math.Sqrt((0.5996272288578146 / (11)) + (0.4928865833186844 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.11336261551915684 = (9.827931349151951 - 10.942051352034532) / 9.827931349151951 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.12739041335921 > 17.339916021230145.
IsChangePoint: Marked as a change because one of 4/11/2023 12:48:48 AM, 4/25/2023 1:02:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -21.82187905164966 (T) = (0 -17.913673471193917) / Math.Sqrt((0.006758747967873437 / (10)) + (0.0881013899935086 / (27))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (27) - 2, .025) and -0.08278189432201886 = (16.54411988705308 - 17.913673471193917) / 16.54411988705308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.522363893468263 > 17.558799339517154.
IsChangePoint: Marked as a change because one of 4/12/2023 2:59:13 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -13.005063931861203 (T) = (0 -18.047319453563752) / Math.Sqrt((0.08491031369389136 / (11)) + (0.10783138225065433 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08518529821147053 = (16.630633941786837 - 18.047319453563752) / 16.630633941786837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	14.32 ns	15.67 ns	1.09	0.05	False
ToString - Duration of single invocation	1.70 ns	3.40 ns	2.00	0.51	False
Parse - Duration of single invocation	14.34 ns	15.37 ns	1.07	0.08	False
TryParse - Duration of single invocation	13.42 ns	14.77 ns	1.10	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.TryParse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.6659949165335 > 14.992299112223362.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -25.04953815113715 (T) = (0 -15.745261631469232) / Math.Sqrt((0.02717368624441573 / (11)) + (0.032477555104418014 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10745224327909252 = (14.217553602897185 - 15.745261631469232) / 14.217553602897185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3987650434547967 > 1.8025384177409873.
IsChangePoint: Marked as a change because one of 3/17/2023 7:47:41 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -15.935809349257847 (T) = (0 -3.300187156136014) / Math.Sqrt((0.047815824147966354 / (11)) + (0.15559996458048972 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.9639208583961834 = (1.6804074064527628 - 3.300187156136014) / 1.6804074064527628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.373308605062114 > 15.077346437912828.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -16.86269010555479 (T) = (0 -15.681943417607005) / Math.Sqrt((0.05680484615140813 / (11)) + (0.03836171280738035 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09603268366502504 = (14.30791585992503 - 15.681943417607005) / 14.30791585992503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "127")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.774793536162456 > 14.155965474704576.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -7.201045017936518 (T) = (0 -14.659993736239006) / Math.Sqrt((0.20588151963979265 / (11)) + (0.03623128346811519 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07487296899164136 = (13.638815152261037 - 14.659993736239006) / 13.638815152261037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IntegerFormatting - Duration of single invocation	1.88 ms	2.24 ms	1.19	0.31	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.IntegerFormatting

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2365835171703297 > 1.9908306603766026.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -25.012055558591953 (T) = (0 -2182889.314045605) / Math.Sqrt((560876515.3005743 / (11)) + (2017247887.493616 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.1493276307699155 = (1899275.0679658884 - 2182889.314045605) / 1899275.0679658884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Perf_Convert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FromBase64Chars - Duration of single invocation	87.38 ns	93.57 ns	1.07	0.00	False
ChangeType - Duration of single invocation	34.94 ns	41.17 ns	1.18	0.33	False
ToBase64CharArray - Duration of single invocation	1.37 μs	1.54 μs	1.13	0.01	False
ToBase64String - Duration of single invocation	1.54 μs	1.72 μs	1.11	0.01	False
FromBase64String - Duration of single invocation	86.60 ns	94.89 ns	1.10	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.FromBase64Chars

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.57304766468727 > 90.46983148632494.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -25.68230993087624 (T) = (0 -94.8846887359987) / Math.Sqrt((0.8585335086512511 / (11)) + (0.8398222768343282 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09879613956299829 = (86.35331461369644 - 94.8846887359987) / 86.35331461369644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ChangeType

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.17027295129579 > 36.6738276647219.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -15.49749498287632 (T) = (0 -40.3801573499363) / Math.Sqrt((0.5093485471494371 / (11)) + (1.711206412485314 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.14745939768390012 = (35.19092477820304 - 40.3801573499363) / 35.19092477820304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: InsertLineBreaks)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5390276931698312 > 1.4354709789294493.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -520.6017753514893 (T) = (0 -1539.9530515922443) / Math.Sqrt((0.8127854943308448 / (11)) + (0.9333978252026525 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.12614593317439476 = (1367.4542581274566 - 1539.9530515922443) / 1367.4542581274566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToBase64String(formattingOptions: InsertLineBreaks)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7183764074785615 > 1.6364470840897627.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -64.27277586429646 (T) = (0 -1718.3776320571212) / Math.Sqrt((66.00198402029854 / (11)) + (11.26622264353038 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10481449551730079 = (1555.3539884110005 - 1718.3776320571212) / 1555.3539884110005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.FromBase64String

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.89418565816415 > 90.0245454144204.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -28.1620021556235 (T) = (0 -94.11022802407251) / Math.Sqrt((0.5371156784010348 / (11)) + (1.036714706880029 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09784416703727389 = (85.72275633439449 - 94.11022802407251) / 85.72275633439449 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfString - Duration of single invocation	32.02 ns	35.59 ns	1.11	0.00	True
IndexOfString - Duration of single invocation	34.28 ns	38.97 ns	1.14	0.00	True
IndexOfString - Duration of single invocation	45.35 ns	48.10 ns	1.06	0.00	False
IndexOfString - Duration of single invocation	12.54 ns	18.03 ns	1.44	0.00	True
IndexOfString - Duration of single invocation	20.89 ns	33.00 ns	1.58	0.00	True
IndexOfString - Duration of single invocation	26.50 ns	36.12 ns	1.36	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.593558273391224 > 33.66910130054294.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -68.70816537408105 (T) = (0 -35.52808856286865) / Math.Sqrt((0.017832891385283142 / (11)) + (0.021902244049277403 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10618045615922762 = (32.11780534093491 - 35.52808856286865) / 32.11780534093491 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.97391102641692 > 35.9659523233278.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -194.9758875257388 (T) = (0 -39.58370424196601) / Math.Sqrt((0.0001797767491418778 / (11)) + (0.019009663905810163 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.15562626842902805 = (34.25303259658208 - 39.58370424196601) / 34.25303259658208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.1024485264631 > 47.63320927117399.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -62.811633480763234 (T) = (0 -48.354535105897455) / Math.Sqrt((0.01336160221549329 / (11)) + (0.02626591511535158 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06527096450164775 = (45.39177046707412 - 48.354535105897455) / 45.39177046707412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.027201315520546 > 13.17674026769946.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -304.0746545198908 (T) = (0 -18.020323857762573) / Math.Sqrt((0.007808347222667941 / (30)) + (0.00036964664902454364 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.4256625413349647 = (12.6399644623395 - 18.020323857762573) / 12.6399644623395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.00140866045238 > 21.93014854319319.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -295.42981506737465 (T) = (0 -33.11256578434755) / Math.Sqrt((0.010721405711789676 / (30)) + (0.009299016474391951 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.5780947388777797 = (20.98262225238434 - 33.11256578434755) / 20.98262225238434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.12106231168136 > 27.820181623506315.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -73.54292985431708 (T) = (0 -36.5971678445914) / Math.Sqrt((0.008684414801556994 / (11)) + (0.4668741672269908 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.3795722068990963 = (26.527910363497313 - 36.5971678445914) / 26.527910363497313 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	12.32 ns	14.66 ns	1.19	0.41	False
TryParseSpan - Duration of single invocation	14.43 ns	15.46 ns	1.07	0.03	False
ToStringHex - Duration of single invocation	27.30 ns	30.94 ns	1.13	0.14	False
ParseSpan - Duration of single invocation	10.88 ns	12.16 ns	1.12	0.08	False
ParseHex - Duration of single invocation	20.98 ns	22.84 ns	1.09	0.02	True
Parse - Duration of single invocation	24.33 ns	25.75 ns	1.06	0.05	False
Parse - Duration of single invocation	15.75 ns	16.89 ns	1.07	0.02	True
ParseHex - Duration of single invocation	14.86 ns	16.26 ns	1.09	0.04	True
ParseHex - Duration of single invocation	10.24 ns	11.26 ns	1.10	0.10	False
ToString - Duration of single invocation	1.70 ns	3.40 ns	2.00	0.52	True
TryParse - Duration of single invocation	22.79 ns	24.39 ns	1.07	0.03	False
ParseHex - Duration of single invocation	20.93 ns	23.00 ns	1.10	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.66106675579819 > 12.707456115513121.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/9/2023 1:58:52 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -9.176924691373918 (T) = (0 -14.47947711838278) / Math.Sqrt((0.3717079129201449 / (11)) + (0.41667280606915524 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.16476881243451558 = (12.431202624767076 - 14.47947711838278) / 12.431202624767076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.460483279579135 > 15.146420678662615.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -21.49670677164967 (T) = (0 -15.8069950630117) / Math.Sqrt((0.022554904779617142 / (11)) + (0.05327526438464586 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09537885944225477 = (14.430619074628032 - 15.8069950630117) / 14.430619074628032 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.939761180248436 > 29.163953187860784.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/17/2023 1:30:52 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -12.595549673857475 (T) = (0 -30.983746863297497) / Math.Sqrt((0.7275474451782498 / (11)) + (0.34501402126045716 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.12937830261744562 = (27.434338690135633 - 30.983746863297497) / 27.434338690135633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.158189498453336 > 11.639911806238329.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -30.777201240928058 (T) = (0 -12.262011582040873) / Math.Sqrt((0.013225034573819036 / (11)) + (0.009974675337651817 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.11105644971629623 = (11.036353360058284 - 12.262011582040873) / 11.036353360058284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.8399460836835 > 22.198367941107474.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -34.725879895536714 (T) = (0 -22.80835700634756) / Math.Sqrt((0.01341801789280157 / (11)) + (0.03155666517501628 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08120572188592375 = (21.095298096058386 - 22.80835700634756) / 21.095298096058386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.750813024745376 > 25.547351018005134.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -30.599368931591837 (T) = (0 -25.71484969785456) / Math.Sqrt((0.01559997428969119 / (11)) + (0.015932404045974465 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.05666528244424348 = (24.33585178304696 - 25.71484969785456) / 24.33585178304696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.888560856332255 > 16.588097403272574.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -9.837759766721176 (T) = (0 -17.291483211533524) / Math.Sqrt((0.14365840441346672 / (11)) + (0.10074000849438743 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.0799569505982682 = (16.01127082144755 - 17.291483211533524) / 16.01127082144755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.260461292852465 > 15.606813283957795.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -36.5555820453316 (T) = (0 -16.155936417690608) / Math.Sqrt((0.008762477081448929 / (11)) + (0.012209804045903565 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08756430327794619 = (14.85515511036562 - 16.155936417690608) / 14.85515511036562 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.26293078001538 > 10.750818902894185.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -31.560470977210514 (T) = (0 -11.169801463935444) / Math.Sqrt((0.0074668457011596275 / (11)) + (0.005692057437774389 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.0924884752792435 = (10.224182420853829 - 11.169801463935444) / 10.224182420853829 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToString(value: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.398694141371339 > 2.1363035280927796.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -24.24940789877344 (T) = (0 -3.921020064130936) / Math.Sqrt((0.06318562979494194 / (11)) + (0.03354203572121518 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -1.0776752736553217 = (1.8872150589887697 - 3.921020064130936) / 1.8872150589887697 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.388459201725947 > 23.931542615584384.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -20.005354406311046 (T) = (0 -24.258186930212403) / Math.Sqrt((0.02101678236785618 / (11)) + (0.09487919621036635 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06552094116608409 = (22.76650415116642 - 24.258186930212403) / 22.76650415116642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "80000000")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.998300897592813 > 22.167133931921075.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/26/2023 8:38:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -6.412579176838925 (T) = (0 -22.783472843105685) / Math.Sqrt((0.7604152432179172 / (15)) + (0.017307905038697207 / (22))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (22) - 2, .025) and -0.06821758370316737 = (21.328494485292687 - 22.783472843105685) / 21.328494485292687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	9.57 ns	10.93 ns	1.14	0.21	False
TryParse - Duration of single invocation	12.95 ns	14.29 ns	1.10	0.20	False
TryParse - Duration of single invocation	9.16 ns	10.77 ns	1.18	0.27	False
Parse - Duration of single invocation	12.95 ns	14.17 ns	1.09	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.927060698107983 > 9.899870953335931.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -14.259351532782206 (T) = (0 -10.568780682454433) / Math.Sqrt((0.033291052272997404 / (11)) + (0.10860901344964505 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.1293216471485333 = (9.358521293858082 - 10.568780682454433) / 9.358521293858082 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "255")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.285483147460045 > 13.572042426367982.
IsChangePoint: Marked as a change because one of 4/9/2023 1:58:52 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -8.66238550575044 (T) = (0 -14.102908580678633) / Math.Sqrt((0.17092322450040182 / (11)) + (0.0319615649202471 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08640917151931987 = (12.981212742300416 - 14.102908580678633) / 12.981212742300416 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.770352864170526 > 10.098501309770931.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -12.84481539695862 (T) = (0 -10.59291359108812) / Math.Sqrt((0.04935625644976842 / (11)) + (0.04917516409161984 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10722615406422503 = (9.567073133348035 - 10.59291359108812) / 9.567073133348035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.Parse(value: "255")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.165289994832015 > 13.595868748164653.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -13.06797227943999 (T) = (0 -14.103558734655639) / Math.Sqrt((0.07596565497448773 / (11)) + (0.03602331094354565 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.09214594191276078 = (12.91362096713464 - 14.103558734655639) / 12.91362096713464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor - Duration of single invocation	41.27 μs	45.20 μs	1.10	0.11	False
Ctor - Duration of single invocation	31.53 μs	34.93 μs	1.11	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.200226072698975 > 44.2878101373488.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -7.853169557497445 (T) = (0 -44756.89082552337) / Math.Sqrt((795556.0119915517 / (11)) + (906534.3957837573 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06094752389646779 = (42185.772450976525 - 44756.89082552337) / 42185.772450976525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.93139728147971 > 33.45070200346547.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -8.430188872570398 (T) = (0 -34496.86063270963) / Math.Sqrt((826957.1573019032 / (11)) + (417989.5540447456 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07970585017718577 = (31950.23962040078 - 34496.86063270963) / 31950.23962040078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Property_Get_int - Duration of single invocation	22.23 ns	30.74 ns	1.38	0.53	False
Field_Set_int - Duration of single invocation	72.15 ns	92.67 ns	1.28	0.17	False
Field_Set_class - Duration of single invocation	73.04 ns	77.89 ns	1.07	0.18	False
Ctor4_ActivatorCreateInstance - Duration of single invocation	578.35 ns	611.21 ns	1.06	0.32	False
StaticMethod4_ByRefParams_int_string_struct_class - Duration of single invocation	387.00 ns	424.18 ns	1.10	0.12	False
Method0_NoParms - Duration of single invocation	15.87 ns	20.45 ns	1.29	0.33	False
Property_Get_class - Duration of single invocation	17.69 ns	28.37 ns	1.60	0.59	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Property_Get_int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.74190133840865 > 23.342666370188514.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -15.584326937145248 (T) = (0 -31.097072048385275) / Math.Sqrt((2.1399480014974217 / (11)) + (1.4899799750504383 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.3360048398641608 = (23.27616721174984 - 31.097072048385275) / 23.27616721174984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Field_Set_int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.67366642011834 > 71.67293954050163.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 3/17/2023 1:30:52 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -30.716538799277515 (T) = (0 -89.87974289470398) / Math.Sqrt((2.0249988297673083 / (11)) + (7.274674187862791 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.30338016119528294 = (68.9589619135207 - 89.87974289470398) / 68.9589619135207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Field_Set_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 77.88525584134615 > 75.9606425066507.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -7.574990999104316 (T) = (0 -80.02207269739677) / Math.Sqrt((1.5012890457547554 / (10)) + (24.39567541137636 / (27))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (27) - 2, .025) and -0.10762629562984483 = (72.24645443424825 - 80.02207269739677) / 72.24645443424825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Ctor4_ActivatorCreateInstance

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 611.2089117973372 > 596.1382135044644.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -6.831590499672808 (T) = (0 -628.3001881980772) / Math.Sqrt((269.92613852147394 / (11)) + (705.9378015646679 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08480265633466662 = (579.18385849181 - 628.3001881980772) / 579.18385849181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 424.1807397349273 > 405.5011755525915.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.39234690686227 (T) = (0 -419.1456268429819) / Math.Sqrt((60.861645718145994 / (11)) + (100.97705288358544 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.08235002030170968 = (387.255156817148 - 419.1456268429819) / 387.255156817148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Method0_NoParms

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.453463168294398 > 16.58811444629414.
IsChangePoint: Marked as a change because one of 1/7/2023 2:04:07 AM, 4/25/2023 1:02:54 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.92857482780062 (T) = (0 -20.359213259883386) / Math.Sqrt((0.40651055459627883 / (10)) + (3.170590850049165 / (27))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (27) - 2, .025) and -0.27133155545175813 = (16.01408631177167 - 20.359213259883386) / 16.01408631177167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Property_Get_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.3723019771529 > 18.57422713164593.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -5.835060806724249 (T) = (0 -28.47511495858143) / Math.Sqrt((11.771613205855937 / (9)) + (19.285646480886474 / (28))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (9) + (28) - 2, .025) and -0.40757974396172414 = (20.229841386063413 - 28.47511495858143) / 20.229841386063413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	6.30 μs	7.34 μs	1.17	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.336976116283409 > 6.637405789921207.
IsChangePoint: Marked as a change because one of 1/26/2023 2:33:00 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -20.03108949680325 (T) = (0 -7566.140323911813) / Math.Sqrt((32090.60709274123 / (11)) + (17362.13575176391 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.18838077419569144 = (6366.7643302565675 - 7566.140323911813) / 6366.7643302565675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EnumerateArray - Duration of single invocation	855.44 ns	929.46 ns	1.09	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 929.4610625161432 > 902.3022787256879.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.093982792922112 (T) = (0 -928.3173847130141) / Math.Sqrt((415.3520787438136 / (11)) + (0.40136530087723854 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.07161526046873555 = (866.2786159903682 - 928.3173847130141) / 866.2786159903682 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
JsonNet_ - Duration of single invocation	1.79 μs	1.95 μs	1.09	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<Location>*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9515463950619194 > 1.9046873391488341.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 3/15/2023 10:57:55 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -8.51390811241 (T) = (0 -1920.7460125055238) / Math.Sqrt((561.6078511324774 / (11)) + (2352.8837994222213 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.05567299416195434 = (1819.4516892329025 - 1920.7460125055238) / 1819.4516892329025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Char

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Char_ToLowerInvariant - Duration of single invocation	18.25 ns	20.26 ns	1.11	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.25600498335806 > 19.157696789534842.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -10.38784632214916 (T) = (0 -20.360787143487848) / Math.Sqrt((0.30458844923705136 / (11)) + (0.16624386089287785 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.10398441182503627 = (18.44300238789486 - 20.360787143487848) / 18.44300238789486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	98.71 ns	105.78 ns	1.07	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Parse(value: "123456.789")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.78012808105385 > 104.01552892567264.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -20.668508122926767 (T) = (0 -104.78622636213753) / Math.Sqrt((0.3717045008536031 / (11)) + (1.2428349387407673 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.05970569368110381 = (98.88238497439907 - 104.78622636213753) / 98.88238497439907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Split - Duration of single invocation	469.21 ns	517.65 ns	1.10	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 517.6541133771069 > 495.65455353802827.
IsChangePoint: Marked as a change because one of 1/20/2023 1:51:38 AM, 4/25/2023 6:46:56 PM, 5/3/2023 11:00:11 PM falls between 4/25/2023 6:51:20 AM and 5/3/2023 11:00:11 PM.
IsRegressionStdDev: Marked as regression because -3.120176436303922 (T) = (0 -519.5441337529509) / Math.Sqrt((1041.7907050876915 / (11)) + (507.9933425047272 / (26))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (26) - 2, .025) and -0.06859478805076409 = (486.1937748177279 - 519.5441337529509) / 486.1937748177279 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

Most likely native PGO being enabled for Linux in dotnet/runtime#85355