[Perf] Linux/x64: 4 Improvements on 3/17/2023 5:15:23 PM #14214
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 27fe4461113c4c0be354ce188d27cf9797d7d632 |
| Compare | a2474a35672a616d78991dc1a587c21a2e73190a |
| Diff | Diff |
| Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Stack - Duration of single invocation | 770.39 μs | 657.97 μs | 0.85 | 0.03 | False | |||||
| HashSet - Duration of single invocation | 6.90 μs | 6.49 μs | 0.94 | 0.01 | False | |||||
| Span - Duration of single invocation | 705.46 μs | 606.91 μs | 0.86 | 0.01 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'Payloads
Histogram
System.Collections.ContainsFalse<String>.Stack(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 657.9738583333332 < 730.5807158184523.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/17/2023 8:36:05 PM, 3/14/2023 8:42:09 PM, 3/17/2023 1:30:52 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 12.3871914912407 (T) = (0 -673977.066472532) / Math.Sqrt((168690014.74331915 / (299)) + (645863118.9669305 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.11936273920571519 = (765328.8095766502 - 673977.066472532) / 765328.8095766502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<String>.HashSet(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.4936953338026075 < 6.50717860031651.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 31.618007173417887 (T) = (0 -6470.034317727055) / Math.Sqrt((9127.430091060976 / (299)) + (4914.105061919263 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.06296069454512959 = (6904.7629913307455 - 6470.034317727055) / 6904.7629913307455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<String>.Span(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 606.9119822115383 < 669.9509052083334.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 129.98849508600145 (T) = (0 -609837.5654571606) / Math.Sqrt((5656467.883080807 / (299)) + (16789258.75823521 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.13761955301611545 = (707156.0673598583 - 609837.5654571606) / 707156.0673598583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 27fe4461113c4c0be354ce188d27cf9797d7d632 |
| Compare | a2474a35672a616d78991dc1a587c21a2e73190a |
| Diff | Diff |
| Configs | CompliationMode:tiered, RunKind:micro |
Improvements in BilinearTest
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Interpol_Scalar - Duration of single invocation | 7.94 μs | 6.76 μs | 0.85 | 0.14 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BilinearTest*'Payloads
Histogram
BilinearTest.Interpol_Scalar
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.761113731385545 < 7.542357323562189.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 2/16/2023 9:36:08 AM, 3/17/2023 1:30:52 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 43.73313259264953 (T) = (0 -6782.611893421113) / Math.Sqrt((66898.66321121245 / (299)) + (872.1165948705088 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09906589932182112 = (7528.421766159696 - 6782.611893421113) / 7528.421766159696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository