[Perf] Linux/x64: 141 Improvements on 12/18/2022 12:43:31 PM #11120
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 01b57ccc4cc532034f717226f4c7cc07a1ee9518 |
| Compare | fc65281072f9259f0b03a291b9c549bc9d792752 |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.4914994698660715 < 34.134923858574794.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 323.14790820794354 (T) = (0 -7588.838816984584) / Math.Sqrt((862575.817051592 / (299)) + (165266.72990855563 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.790929776437225 = (36298.037509420734 - 7588.838816984584) / 36298.037509420734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 389.8231096207667 < 2.1324269399415114.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 454.9577952420192 (T) = (0 -389.8934789354446) / Math.Sqrt((4514.776355916569 / (299)) + (19.76819442986105 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8221697701209937 = (2192.5039359209277 - 389.8934789354446) / 2192.5039359209277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.57575727228036 < 115.50642455252859.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 664.79974503212 (T) = (0 -17564.407622785668) / Math.Sqrt((7279859.5814475985 / (299)) + (11189.211263508401 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8560497053993933 = (122017.1703817523 - 17564.407622785668) / 122017.1703817523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 369.04906063523765 < 593.6533121329092.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 321.4331190753153 (T) = (0 -368.3482762244578) / Math.Sqrt((143.8627300641115 / (299)) + (6.084317123923363 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4158500272098044 = (630.5714172424595 - 368.3482762244578) / 630.5714172424595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.94668925120774 < 242.73622346631382.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 411.6728663678483 (T) = (0 -114814.39286913526) / Math.Sqrt((25187555.68858278 / (282)) + (977683.4539012865 / (33))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (282) + (33) - 2, .975) and 0.5528919677313768 = (256793.40245033795 - 114814.39286913526) / 256793.40245033795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.421587283871883 < 67.11468231135066.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 380.7917444846839 (T) = (0 -12556.876759341161) / Math.Sqrt((6330989.540350357 / (299)) + (30025.954449869878 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8183974610350875 = (69144.83041323161 - 12556.876759341161) / 69144.83041323161 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6981974896761578 < 1.9203875762223916.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 137.66014122767538 (T) = (0 -1697.7743596331943) / Math.Sqrt((1419.5763375245722 / (299)) + (28.90929564509956 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1612734985533086 = (2024.228823942918 - 1697.7743596331943) / 2024.228823942918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.111336962594697 < 33.636852982656805.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 574.3156522435836 (T) = (0 -7128.803676121713) / Math.Sqrt((738516.7309338709 / (299)) + (993.8033443441914 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.801121355763791 = (35844.99332997664 - 7128.803676121713) / 35844.99332997664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 533.3743871474677 < 3.5175438330559463.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 715.060669951745 (T) = (0 -530.9406789932791) / Math.Sqrt((5820.641355477921 / (299)) + (17.019369353739656 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.85755779089807 = (3727.4111539041382 - 530.9406789932791) / 3727.4111539041382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 271.3044746859844 < 1.2880728646253456.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 528.521284284486 (T) = (0 -272.2277152935718) / Math.Sqrt((1098.5998797388243 / (299)) + (11.968352985706344 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7959523434406693 = (1334.137916033437 - 272.2277152935718) / 1334.137916033437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.045294838978702 < 67.03409183365844.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 371.526777794764 (T) = (0 -12238.70548188482) / Math.Sqrt((6470345.695028366 / (299)) + (40335.80607968716 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.8212321778482392 = (68461.45651142437 - 12238.70548188482) / 68461.45651142437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 383.10830651055886 < 901.412874890448.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 484.2003396946704 (T) = (0 -382.5945004969561) / Math.Sqrt((434.9378160199789 / (299)) + (1.187359815435157 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6070955755078546 = (973.7597152067823 - 382.5945004969561) / 973.7597152067823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.25198492968921 < 64.9610060293491.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 384.35023856000845 (T) = (0 -10332.925354449704) / Math.Sqrt((6057599.0330866845 / (299)) + (45852.50163199758 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8455113205804965 = (66884.67655543452 - 10332.925354449704) / 66884.67655543452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.45165515873015 < 244.204683777228.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 423.57788118159385 (T) = (0 -115532.42654192702) / Math.Sqrt((9050101.72483086 / (299)) + (2661719.8672148893 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5497697703655005 = (256607.43978856583 - 115532.42654192702) / 256607.43978856583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.329304496997917 < 113.23762394755853.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 700.9937382302921 (T) = (0 -15377.673766366537) / Math.Sqrt((6459916.4473476745 / (299)) + (16653.317077082556 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8714357350834891 = (119610.79368635404 - 15377.673766366537) / 119610.79368635404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.931600180654158 < 59.521399215624996.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 358.5638716008966 (T) = (0 -7790.256983573591) / Math.Sqrt((6090125.705321033 / (289)) + (20145.418820306684 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.8714267080965036 = (60590.0095442897 - 7790.256983573591) / 60590.0095442897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.996911073702705 < 115.43483147425249.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 683.9204320073237 (T) = (0 -17115.790232166197) / Math.Sqrt((6656341.857929843 / (288)) + (15303.336519679402 / (33))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (33) - 2, .975) and 0.859854000903676 = (122128.28295156911 - 17115.790232166197) / 122128.28295156911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.383229634418902 < 2.530595050074358.
IsChangePoint: Marked as a change because one of 12/5/2022 8:20:45 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 62.17090753281283 (T) = (0 -2386.451152242458) / Math.Sqrt((1796.2257283282856 / (298)) + (205.11321068637858 / (33))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (298) + (33) - 2, .975) and 0.08354006596864177 = (2603.9885254392384 - 2386.451152242458) / 2603.9885254392384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.03484650161724 < 657.3133430418428.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 314.45985869812404 (T) = (0 -254.31319705740498) / Math.Sqrt((174.26745485342252 / (299)) + (48.00425705402154 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6383370644250789 = (703.177384359648 - 254.31319705740498) / 703.177384359648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0938318366993602 < 6.485755413087226.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 819.3238369891567 (T) = (0 -3100.8388069101075) / Math.Sqrt((4226.659367691249 / (299)) + (214.6464843807695 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5455427205194489 = (6823.168968608875 - 3100.8388069101075) / 6823.168968608875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.05983085175089 < 242.13728502868184.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 308.225604053091 (T) = (0 -113486.89772535418) / Math.Sqrt((18068397.281722024 / (289)) + (4874718.54924198 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.5546285135715219 = (254814.0174743292 - 113486.89772535418) / 254814.0174743292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 01b57ccc4cc532034f717226f4c7cc07a1ee9518 |
| Compare | fc65281072f9259f0b03a291b9c549bc9d792752 |
| Diff | Diff |
Improvements in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOf_Word_NotFound - Duration of single invocation | 3.36 μs | 388.63 ns | 0.12 | 0.09 | False | |||||
| LastIndexOf_Word_NotFound - Duration of single invocation | 1.21 μs | 1.05 μs | 0.87 | 0.06 | False | |||||
| IndexOf_Word_NotFound - Duration of single invocation | 1.61 μs | 234.51 ns | 0.15 | 0.08 | False | |||||
| LastIndexOf_Word_NotFound - Duration of single invocation | 1.42 μs | 230.05 ns | 0.16 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'Payloads
Histogram
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 388.6335807238261 < 3.1872003443668255.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 331.120344835703 (T) = (0 -384.0189838209742) / Math.Sqrt((22589.25866242762 / (299)) + (55.07234078201694 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8834081152460818 = (3293.702513099382 - 384.0189838209742) / 3293.702513099382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0470266361883749 < 1.1477196560290905.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM, 11/8/2022 11:25:18 AM, 11/10/2022 11:11:41 PM, 12/10/2022 11:25:25 PM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 32.39743603857915 (T) = (0 -1046.8635413246207) / Math.Sqrt((3683.631422022154 / (299)) + (1.3770139310685867 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.09825425718960787 = (1160.9298404469896 - 1046.8635413246207) / 1160.9298404469896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 234.50572586752529 < 1.5397788602384967.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 413.95974076532127 (T) = (0 -233.26174638507945) / Math.Sqrt((3459.366868430624 / (299)) + (0.6563700805561798 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8579863488116164 = (1642.530449947053 - 233.26174638507945) / 1642.530449947053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.05433045893736 < 1.3469426210582482.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 336.8821107980936 (T) = (0 -229.6346621789563) / Math.Sqrt((4194.827235901139 / (299)) + (1.2015120324640476 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8462024205351516 = (1493.0967247858466 - 229.6346621789563) / 1493.0967247858466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 01b57ccc4cc532034f717226f4c7cc07a1ee9518 |
| Compare | fc65281072f9259f0b03a291b9c549bc9d792752 |
| Diff | Diff |
Improvements in Microsoft.Extensions.Logging.FormattingOverhead
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TwoArguments - Duration of single invocation | 675.98 ns | 390.05 ns | 0.58 | 0.08 | False | |||||
| TwoArguments_DefineMessage - Duration of single invocation | 536.29 ns | 256.55 ns | 0.48 | 0.04 | False | |||||
| FourArguments_DefineMessage - Duration of single invocation | 1.05 μs | 450.85 ns | 0.43 | 0.02 | False | |||||
| FourArguments_EnumerableArgument - Duration of single invocation | 3.58 μs | 2.91 μs | 0.81 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'Payloads
Histogram
Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 390.0496035572363 < 644.4920669932276.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 172.1873783409561 (T) = (0 -392.99658134529176) / Math.Sqrt((189.70731935738422 / (299)) + (72.3488200964479 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4241807511285213 = (682.4999027307744 - 392.99658134529176) / 682.4999027307744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 256.54939985490995 < 510.4968817250954.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 257.60152374972324 (T) = (0 -258.6119089050357) / Math.Sqrt((106.45001145642051 / (299)) + (26.70242869747459 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5181233473224988 = (536.6765695496629 - 258.6119089050357) / 536.6765695496629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 450.84791166950833 < 0.9991890235997299.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 457.2221769295112 (T) = (0 -452.9850555547874) / Math.Sqrt((298.535662735949 / (299)) + (21.510232957292914 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5645823198132545 = (1040.346031333655 - 452.9850555547874) / 1040.346031333655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_EnumerableArgument
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9056516887218735 < 3.403747550548424.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.48876522658255 (T) = (0 -2930.736947802502) / Math.Sqrt((14211.923360637933 / (299)) + (4006.33250517245 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1585560949106894 = (3482.985532459748 - 2930.736947802502) / 3482.985532459748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository