[Perf] Linux/arm64: 79 Improvements on 12/10/2022 1:39:11 AM #10695
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5b8a981452b02e034857caa566f83ca1801000b7 |
| Compare | 6407eae2c9cda49c725e744f9628bca65b6e71d2 |
| Diff | Diff |
Improvements in Interop.StructureToPtr
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MarshalPtrToStructure - Duration of single invocation | 7.71 μs | 3.41 μs | 0.44 | 0.11 | False | |||||
| MarshalDestroyStructure - Duration of single invocation | 2.13 μs | 1.67 μs | 0.78 | 0.14 | False | |||||
| MarshalStructureToPtr - Duration of single invocation | 1.78 μs | 1.30 μs | 0.73 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Interop.StructureToPtr*'Payloads
Histogram
Interop.StructureToPtr.MarshalPtrToStructure
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 85.97173636755564 (T) = (0 -3451.9263683172476) / Math.Sqrt((37164.389813066795 / (33)) + (2969.99471899917 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.5599875996032989 = (7845.06610542135 - 3451.9263683172476) / 7845.06610542135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Interop.StructureToPtr.MarshalDestroyStructure
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.71207976672952 (T) = (0 -1676.5345694021146) / Math.Sqrt((2546.7501765275438 / (33)) + (198.6824098640696 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.22536439185757218 = (2164.2880236585506 - 1676.5345694021146) / 2164.2880236585506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Interop.StructureToPtr.MarshalStructureToPtr
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 54.505001981608984 (T) = (0 -1292.4393515116967) / Math.Sqrt((3000.024556588361 / (33)) + (13.884601699394686 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2943675276217562 = (1831.604131192681 - 1292.4393515116967) / 1831.604131192681 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5b8a981452b02e034857caa566f83ca1801000b7 |
| Compare | 6407eae2c9cda49c725e744f9628bca65b6e71d2 |
| Diff | Diff |
Improvements in MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| BinaryFormatter_ - Duration of single invocation | 1.13 ms | 781.71 μs | 0.69 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>*'Payloads
Histogram
MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>.BinaryFormatter_
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.73240666215288 (T) = (0 -788965.3900972771) / Math.Sqrt((223902805.90715772 / (33)) + (105145442.08474953 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3087169027199321 = (1141305.7735702656 - 788965.3900972771) / 1141305.7735702656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5b8a981452b02e034857caa566f83ca1801000b7 |
| Compare | 6407eae2c9cda49c725e744f9628bca65b6e71d2 |
| Diff | Diff |
Improvements in System.Reflection.Attributes
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'Payloads
Histogram
System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 39.69701050768853 (T) = (0 -17755.869787852727) / Math.Sqrt((848781.8807489843 / (33)) + (289.4417088695564 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.26446968239435 = (24140.228299022237 - 17755.869787852727) / 24140.228299022237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesClassMissInherit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 64.85620873878447 (T) = (0 -15091.766495896893) / Math.Sqrt((692885.529128834 / (32)) + (19534.397049341642 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.43238326290746526 = (26587.951886691077 - 15091.766495896893) / 26587.951886691077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesClassHit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.503234221054537 (T) = (0 -13037.644948093772) / Math.Sqrt((861285.8887206583 / (33)) + (8639.458935720735 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.24687021973673356 = (17311.28590285766 - 13037.644948093772) / 17311.28590285766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.98598222519713 (T) = (0 -10593.114833127129) / Math.Sqrt((488767.5244169311 / (29)) + (36757.85205936265 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.3959063276904651 = (17535.550062340768 - 10593.114833127129) / 17535.550062340768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 63.594688054111025 (T) = (0 -10331.468502533287) / Math.Sqrt((345382.53394158074 / (33)) + (1995.2986107788465 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.39724787563268427 = (17140.49289063528 - 10331.468502533287) / 17140.49289063528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesClassMiss
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 41.80388648681255 (T) = (0 -9217.562315412739) / Math.Sqrt((427143.68358193524 / (33)) + (35456.98703517261 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4426732183477976 = (16538.882786302063 - 9217.562315412739) / 16538.882786302063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.01472381066601 (T) = (0 -12583.354248609603) / Math.Sqrt((767831.7790209649 / (30)) + (1066.2373425324467 / (2))) is greater than 2.0422724562973102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (2) - 2, .975) and 0.3395028222979267 = (19051.336891988212 - 12583.354248609603) / 19051.336891988212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 26.21731687968956 (T) = (0 -17686.350843239445) / Math.Sqrt((602109.1685001137 / (33)) + (83302.4376174045 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2662099052825399 = (24102.738604081907 - 17686.350843239445) / 24102.738604081907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedClassMiss
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 32.2884914410365 (T) = (0 -1397.892707341739) / Math.Sqrt((1989.9629137281722 / (33)) + (394.5938567574875 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2704563326024664 = (1916.1193082908594 - 1397.892707341739) / 1916.1193082908594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.IsDefinedClassMissInherit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 68.04821463196667 (T) = (0 -7106.05182729861) / Math.Sqrt((55590.64460399214 / (27)) + (5080.77495705225 / (2))) is greater than 2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (2) - 2, .975) and 0.3937309986495147 = (11720.955238466147 - 7106.05182729861) / 11720.955238466147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Attributes.GetCustomAttributesClassHitInherit
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.90500945384042 (T) = (0 -28118.150539718947) / Math.Sqrt((3934469.1873802515 / (33)) + (3982.0855815741843 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.33620602988633735 = (42359.75589670425 - 28118.150539718947) / 42359.75589670425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5b8a981452b02e034857caa566f83ca1801000b7 |
| Compare | 6407eae2c9cda49c725e744f9628bca65b6e71d2 |
| Diff | Diff |
Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Transient - Duration of single invocation | 78.32 μs | 65.35 μs | 0.83 | 0.16 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'Payloads
Histogram
Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 19.428687874112658 (T) = (0 -65544.03542717994) / Math.Sqrt((15503006.475434586 / (33)) + (78111.06180159663 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.17454116675947892 = (79403.15469140036 - 65544.03542717994) / 79403.15469140036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5b8a981452b02e034857caa566f83ca1801000b7 |
| Compare | 6407eae2c9cda49c725e744f9628bca65b6e71d2 |
| Diff | Diff |
Improvements in System.Reflection.Invoke
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'Payloads
Histogram
System.Reflection.Invoke.Field_Set_int
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 94.28171210988425 (T) = (0 -2365.1292285714285) / Math.Sqrt((9574.102103001274 / (33)) + (0.022911249535226632 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4044092945137225 = (3971.0647039738287 - 2365.1292285714285) / 3971.0647039738287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Field_Get_class
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 30.858252467049788 (T) = (0 -2198.109186598558) / Math.Sqrt((8480.062762604577 / (29)) + (4745.959753187335 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.42021407335293137 = (3791.242742490034 - 2198.109186598558) / 3791.242742490034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Property_Set_class
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 67.20298761488192 (T) = (0 -2341.841537202381) / Math.Sqrt((5457.773203271072 / (33)) + (562.8390143537583 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3775612823183801 = (3762.364825126838 - 2341.841537202381) / 3762.364825126838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Property_Set_int
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 33.85698777942084 (T) = (0 -2354.7309137436223) / Math.Sqrt((9131.103228658018 / (33)) + (985.6035615584985 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.28512813742694004 = (3293.920263231746 - 2354.7309137436223) / 3293.920263231746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Field_Set_class
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 28.346151495333345 (T) = (0 -3335.879161089743) / Math.Sqrt((98161.51605461809 / (29)) + (1202.161159120456 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.34916001953091136 = (5125.498219524606 - 3335.879161089743) / 5125.498219524606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Ctor4_int_string_struct_class
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 120.76967992381098 (T) = (0 -1431.5507488162878) / Math.Sqrt((26057.073132415477 / (33)) + (235.42826704920765 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.7176066483733995 = (5069.350041601477 - 1431.5507488162878) / 5069.350041601477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 96.6695143945589 (T) = (0 -1766.9401969907408) / Math.Sqrt((38930.59785230643 / (33)) + (509.26855641451266 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.6744819005483538 = (5428.0858728508565 - 1766.9401969907408) / 5428.0858728508565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 115.64433967188748 (T) = (0 -2150.030017578125) / Math.Sqrt((45309.65064434444 / (29)) + (0.537326354859428 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.6801275605604229 = (6721.523183882363 - 2150.030017578125) / 6721.523183882363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Ctor4_ActivatorCreateInstance
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 57.4415398964921 (T) = (0 -9989.64144561012) / Math.Sqrt((75138.77160699888 / (33)) + (29933.13385687595 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4302230939743098 = (17532.548862483563 - 9989.64144561012) / 17532.548862483563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.StaticMethod4_int_string_struct_class
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 100.70754553063318 (T) = (0 -1504.280227083333) / Math.Sqrt((27452.9322340278 / (33)) + (911.4300408168743 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.7060812888514478 = (5118.014505456377 - 1504.280227083333) / 5118.014505456377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 107.55946343248478 (T) = (0 -2042.67862330443) / Math.Sqrt((8206.305639041591 / (32)) + (552.1076960636332 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.5485512509220376 = (4524.718758167767 - 2042.67862330443) / 4524.718758167767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Reflection.Invoke.Field_Get_int
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 104.2770803159117 (T) = (0 -2225.897859906463) / Math.Sqrt((6662.075341681476 / (33)) + (92.90292664392777 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.424707062859651 = (3869.1555487729393 - 2225.897859906463) / 3869.1555487729393 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository