[Perf] Linux/x64: 4 Improvements on 11/7/2022 8:57:30 PM #9763
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | bfa4812aec806f09f1e2927939ba98118e5e5bf7 |
| Compare | 2aebbf75167b3aa21cae23e6fd702688dd724663 |
| Diff | Diff |
Improvements in System.IO.Tests.StreamReaderReadLineTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadLine - Duration of single invocation | 388.73 μs | 274.38 μs | 0.71 | 0.05 | False | |||||
| ReadLine - Duration of single invocation | 385.04 μs | 270.26 μs | 0.70 | 0.04 | False | |||||
| ReadLine - Duration of single invocation | 381.19 μs | 274.60 μs | 0.72 | 0.03 | False | |||||
| ReadLine - Duration of single invocation | 421.74 μs | 354.48 μs | 0.84 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'Related Issues
Regressions
- [Perf] Windows/x86: 22 Regressions on 11/7/2022 8:57:30 PM
- [Perf] Windows/x64: 29 Regressions on 11/7/2022 8:57:30 PM
- [Perf] Alpine/x64: 22 Regressions on 11/7/2022 8:57:30 PM
- [Perf] Linux/x64: 22 Regressions on 11/7/2022 8:57:30 PM
Improvements
- [Perf] Windows/x86: 21 Improvements on 11/7/2022 8:57:30 PM
- [Perf] Alpine/x64: 16 Improvements on 11/7/2022 8:57:30 PM
- [Perf] Windows/x64: 15 Improvements on 11/7/2022 8:57:30 PM
- [Perf] Linux/x64: 10 Improvements on 11/7/2022 8:57:30 PM
Payloads
Histogram
Edge Detector Info
System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 274.37523629385964 < 369.97641214793845.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsImprovementStdDev: Marked as improvement because 74.74869655935957 (T) = (0 -277206.9190962127) / Math.Sqrt((952332630.1837969 / (299)) + (17232107.513312463 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.34003001971443025 = (420029.5882795532 - 277206.9190962127) / 420029.5882795532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 270.2596979166667 < 365.4191964802918.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsImprovementStdDev: Marked as improvement because 86.3659701886609 (T) = (0 -269591.46341431554) / Math.Sqrt((797188068.139038 / (299)) + (3898200.6182537046 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.3478251360937056 = (413372.97454176476 - 269591.46341431554) / 413372.97454176476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 274.59722214052283 < 362.60162870172763.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsImprovementStdDev: Marked as improvement because 75.45598359798855 (T) = (0 -268018.4892684927) / Math.Sqrt((1005056850.1432915 / (299)) + (8355435.918999832 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.34797673949398394 = (411056.63785750745 - 268018.4892684927) / 411056.63785750745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 354.47976190476203 < 401.63046793247577.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 10:11:20 AM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsImprovementStdDev: Marked as improvement because 60.1840061178279 (T) = (0 -353891.48749783525) / Math.Sqrt((751954679.0045993 / (299)) + (3112620.3200075706 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.21524976363287388 = (450960.6637853573 - 353891.48749783525) / 450960.6637853573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository