Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	83ce1253cf71348776fec4464e3fad1277fac7f0
Compare	5a10aa6a3fa43d9d0e30e48ae22f2767a358af41
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	4.25 ms	4.94 ms	1.16	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.944040256410257 > 4.470104023437501.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -46.744375261168585 (T) = (0 -4768527.395233576) / Math.Sqrt((253083086.14412147 / (27)) + (2248016736.454753 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.12129495544576763 = (4252696.734319884 - 4768527.395233576) / 4252696.734319884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[EgorBo]

Regressed via #78927 cc @MihaZupan

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	83ce1253cf71348776fec4464e3fad1277fac7f0
Compare	5a10aa6a3fa43d9d0e30e48ae22f2767a358af41
Diff	Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IterateGetPositionTenSegments - Duration of single invocation	88.64 ns	94.53 ns	1.07	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionTenSegments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.52713939332814 > 93.02948390094744.
IsChangePoint: Marked as a change because one of 12/6/2022 6:34:01 PM, 12/17/2022 10:44:50 PM, 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -17.118853775804876 (T) = (0 -95.47996438655115) / Math.Sqrt((0.6120732677830997 / (27)) + (2.4221451343567497 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.07294293228394086 = (88.9888562696488 - 95.47996438655115) / 88.9888562696488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	83ce1253cf71348776fec4464e3fad1277fac7f0
Compare	5a10aa6a3fa43d9d0e30e48ae22f2767a358af41
Diff	Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IDictionary - Duration of single invocation	4.91 μs	6.06 μs	1.24	0.20	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.IDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.060088645548168 > 5.343659686397301.
IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 12/20/2022 11:23:14 PM, 12/28/2022 5:13:50 PM, 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -8.619112141642141 (T) = (0 -5912.352564087677) / Math.Sqrt((49704.028608457615 / (27)) + (105325.01627739171 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.1401200372941473 = (5185.728143257173 - 5912.352564087677) / 5185.728143257173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	83ce1253cf71348776fec4464e3fad1277fac7f0
Compare	5a10aa6a3fa43d9d0e30e48ae22f2767a358af41
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	4.25 ms	4.94 ms	1.16	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.944040256410257 > 4.470104023437501.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -46.744375261168585 (T) = (0 -4768527.395233576) / Math.Sqrt((253083086.14412147 / (27)) + (2248016736.454753 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.12129495544576763 = (4252696.734319884 - 4768527.395233576) / 4252696.734319884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	dakersnar
Labels:	area-System.Text.RegularExpressions, os-windows, tenet-performance, tenet-performance-benchmarks, arch-x64, untriaged
Milestone:	-