[Perf] Changes at 11/17/2021 8:43:26 PM #2515
Description
Run Information
| Architecture | x86 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 44d28bf5b6c8b448ac4f19638b90cdbdba6be5bc |
| Compare | 9962c10813de695782baa20f1bf0bb2e9810971d |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IP_IsMatch - Duration of single invocation | 383.42 ns | 231.38 ns | 0.60 | 0.05 | True | |||||
| SplitWords - Duration of single invocation | 84.53 μs | 72.51 μs | 0.86 | 0.01 | True | |||||
| SplitWords - Duration of single invocation | 8.24 μs | 2.97 μs | 0.36 | 0.00 | True | |||||
| Ctor - Duration of single invocation | 33.29 μs | 30.22 μs | 0.91 | 0.01 | False | |||||
| MatchesSet - Duration of single invocation | 225.70 μs | 197.01 μs | 0.87 | 0.00 | False | |||||
| Email_IsMatch - Duration of single invocation | 543.06 ns | 478.35 ns | 0.88 | 0.05 | False | |||||
| Date_IsNotMatch - Duration of single invocation | 213.81 ns | 190.84 ns | 0.89 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 231.37633768580358 < 364.2455648792514.
IsChangePoint: Marked as a change because one of 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 94.599881460361 (T) = (0 -236.83118837458903) / Math.Sqrt((97.173055914894 / (299)) + (56.89169911425687 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.38771537749132473 = (386.79917748748204 - 236.83118837458903) / 386.79917748748204 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.51101792943898 < 80.27449868867699.
IsChangePoint: Marked as a change because one of 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 20.001039399860773 (T) = (0 -70567.71799766294) / Math.Sqrt((1512314.7568634236 / (299)) + (13258859.115022648 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16902274743699328 = (84921.35949571294 - 70567.71799766294) / 84921.35949571294 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9697811390330506 < 7.866193287170886.
IsChangePoint: Marked as a change because one of 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 915.8123380008738 (T) = (0 -2968.267995573514) / Math.Sqrt((2797.5199471182636 / (299)) + (644.302655368523 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.6431977088093472 = (8319.083337913482 - 2968.267995573514) / 8319.083337913482 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.21801598837209 < 31.771805178872082.
IsChangePoint: Marked as a change because one of 10/25/2021 8:49:36 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 16.676380444611734 (T) = (0 -30434.292430716043) / Math.Sqrt((1142298.610044415 / (299)) + (139761.22902831036 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.059094378440794196 = (32345.744071846835 - 30434.292430716043) / 32345.744071846835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.01073260073255 < 214.41498053980334.
IsChangePoint: Marked as a change because one of 10/26/2021 3:15:01 AM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 88.18189267466457 (T) = (0 -197089.1091302313) / Math.Sqrt((26828604.394114528 / (299)) + (1029542.7017646139 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13860017250598403 = (228800.96192218064 - 197089.1091302313) / 228800.96192218064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 478.352448940034 < 515.8294981206701.
IsChangePoint: Marked as a change because one of 9/11/2021 10:26:04 AM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 90.07393467149183 (T) = (0 -480.17148309279526) / Math.Sqrt((71.85378902333699 / (299)) + (6.470462342913648 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11598355131357137 = (543.1703039081323 - 480.17148309279526) / 543.1703039081323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 190.84136730732854 < 203.48090221963918.
IsChangePoint: Marked as a change because one of 11/3/2021 2:05:01 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 59.174740096211146 (T) = (0 -191.49293024287505) / Math.Sqrt((32.55683765768047 / (299)) + (2.290590226862622 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.120609708035095 = (217.75647513120057 - 191.49293024287505) / 217.75647513120057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x86 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 44d28bf5b6c8b448ac4f19638b90cdbdba6be5bc |
| Compare | 9962c10813de695782baa20f1bf0bb2e9810971d |
| Diff | Diff |
Improvements in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ObjInt - Duration of single invocation | 218.28 μs | 187.20 μs | 0.86 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'Payloads
Histogram
PerfLabTests.CastingPerf.ObjInt
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 187.20218063186812 < 207.37012432113602.
IsChangePoint: Marked as a change because one of 9/11/2021 4:06:00 PM, 10/4/2021 7:47:07 AM, 10/12/2021 4:02:36 PM, 11/1/2021 1:52:02 PM, 11/10/2021 7:00:39 PM, 11/17/2021 2:42:53 PM, 11/23/2021 3:51:55 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 3:51:55 AM.
IsImprovementStdDev: Marked as improvement because 26.904340978515062 (T) = (0 -187106.607071052) / Math.Sqrt((255556340.71794918 / (299)) + (394.75470817688523 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11733813761379557 = (211979.9382350389 - 187106.607071052) / 211979.9382350389 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository