Skip to content

[Perf] Alpine/x64: 4 Regressions on 1/3/2023 9:28:36 AM #11415

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyTwoValues - Duration of single invocation 7.00 ns 8.87 ns 1.27 0.19 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.865383362022623 > 7.347480226005509.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM.
IsRegressionStdDev: Marked as regression because -12.803954024964908 (T) = (0 -7.944410787921018) / Math.Sqrt((0.03223053059958381 / (13)) + (0.07585064659128789 / (30))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (30) - 2, .025) and -0.12873290265358006 = (7.038344296727957 - 7.944410787921018) / 7.038344296727957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 4.04 μs 4.35 μs 1.08 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.IDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.354598658328728 > 4.259384390236758.
IsChangePoint: Marked as a change because one of 11/16/2022 6:51:59 PM, 12/14/2022 12:26:48 AM, 12/21/2022 3:18:57 AM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM.
IsRegressionStdDev: Marked as regression because -9.959330984028687 (T) = (0 -4288.813933144703) / Math.Sqrt((1990.7501838033504 / (14)) + (12401.548625168 / (30))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (30) - 2, .025) and -0.05790463492200513 = (4054.0647914458746 - 4288.813933144703) / 4054.0647914458746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAny - Duration of single invocation 4.89 ns 6.00 ns 1.23 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.004671479508234 > 5.134335117184184.
IsChangePoint: Marked as a change because one of 11/4/2022 4:27:34 PM, 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM.
IsRegressionStdDev: Marked as regression because -36.4142450102855 (T) = (0 -6.010982075556171) / Math.Sqrt((0.006150546727840375 / (13)) + (0.012346158398923989 / (30))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (30) - 2, .025) and -0.21978421041181445 = (4.927906119990509 - 6.010982075556171) / 4.927906119990509 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline ee6de921299d1b86a02d9049b9c8dd164cab1401
Compare 7fa0d5b5942f138c362f2753398b4a8d4f71eb73
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HtmlDecode_Entities - Duration of single invocation 116.54 ns 123.34 ns 1.06 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 7:48:18 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 7:48:18 AM.
IsRegressionStdDev: Marked as regression because -13.068181845624293 (T) = (0 -123.7616681517338) / Math.Sqrt((2.9388924732445436 / (11)) + (1.3779871647413013 / (32))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (32) - 2, .025) and -0.06248818637471656 = (116.48286516391039 - 123.7616681517338) / 116.48286516391039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions