Skip to content

[Perf] Linux/x64: 90 Improvements on 12/9/2022 5:51:37 PM #10671

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 91af060b086ab83ba3200fbc7049854900b08523
Diff Diff

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor4_ActivatorCreateInstance - Duration of single invocation 18.40 μs 10.70 μs 0.58 0.00 False
Ctor4_int_string_struct_class - Duration of single invocation 4.62 μs 968.69 ns 0.21 0.00 False
Field_Get_int - Duration of single invocation 3.51 μs 1.96 μs 0.56 0.01 False
Property_Set_class - Duration of single invocation 3.32 μs 1.87 μs 0.56 0.01 False
StaticMethod4_arrayNotCached_int_string_struct_class - Duration of single invocation 4.89 μs 1.20 μs 0.25 0.01 False
Field_Get_class - Duration of single invocation 3.45 μs 1.92 μs 0.56 0.00 False
Field_Set_int - Duration of single invocation 3.57 μs 1.98 μs 0.55 0.00 False
StaticMethod5_ByRefParams_int_string_struct_class_bool - Duration of single invocation 13.26 μs 6.35 μs 0.48 0.02 False
Ctor0_ActivatorCreateInstance_NoParams - Duration of single invocation 3.96 μs 1.65 μs 0.42 0.01 False
Property_Set_int - Duration of single invocation 2.74 μs 1.89 μs 0.69 0.01 False
StaticMethod4_ByRefParams_int_string_struct_class - Duration of single invocation 10.63 μs 4.87 μs 0.46 0.01 False
StaticMethod4_int_string_struct_class - Duration of single invocation 4.61 μs 989.43 ns 0.21 0.01 False
StaticMethod5_arrayNotCached_int_string_struct_class_bool - Duration of single invocation 6.16 μs 1.50 μs 0.24 0.03 False
Field_Set_class - Duration of single invocation 4.78 μs 2.78 μs 0.58 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Ctor4_ActivatorCreateInstance


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.703351617424241 < 17.49046635405325.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 205.13559163127513 (T) = (0 -10699.824105967273) / Math.Sqrt((92665.26641143313 / (299)) + (17819.3083229085 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.432696044422107 = (18860.831130760813 - 10699.824105967273) / 18860.831130760813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Ctor4_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 968.6882360011311 < 4.390964770809152.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 441.23378115293406 (T) = (0 -943.5672352727928) / Math.Sqrt((17789.72820884301 / (299)) + (281.0753353938645 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8066288680347269 = (4879.566177655955 - 943.5672352727928) / 4879.566177655955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Get_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9591169101562502 < 3.3324999883258926.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 157.07577110144553 (T) = (0 -1931.0694081961524) / Math.Sqrt((2487.204045091306 / (299)) + (1419.541518411146 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.4600450287543844 = (3576.3526794491618 - 1931.0694081961524) / 3576.3526794491618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Set_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8742201160714282 < 3.1685416420833326.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 187.7251202427059 (T) = (0 -1869.5240838928678) / Math.Sqrt((3337.94296082433 / (299)) + (743.6688090505407 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.4460079240033301 = (3374.6404775365518 - 1869.5240838928678) / 3374.6404775365518 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2046435272435898 < 4.6925086471354165.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 402.31301866793063 (T) = (0 -1177.5547172525603) / Math.Sqrt((13207.319332559666 / (299)) + (735.3344605577734 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.7706205916145888 = (5133.6548713823195 - 1177.5547172525603) / 5133.6548713823195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Get_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.91667094140625 < 3.3393225403318443.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 219.97971106077313 (T) = (0 -1874.4459017935499) / Math.Sqrt((2995.069895881112 / (219)) + (594.5596011871131 / (14))) is greater than 1.970286659282671 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (219) + (14) - 2, .975) and 0.46790238647393795 = (3522.748184063674 - 1874.4459017935499) / 3522.748184063674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Set_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9819639757812502 < 3.3952646205536863.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 192.45665839090594 (T) = (0 -1964.0888759044408) / Math.Sqrt((2760.813089799057 / (299)) + (915.5535526589048 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.4584360548262232 = (3626.6979982838493 - 1964.0888759044408) / 3626.6979982838493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod5_ByRefParams_int_string_struct_class_bool

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.347580998611111 < 12.676268643229168.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 12.49253545327311 (T) = (0 -6238.431885952217) / Math.Sqrt((17626462.036938105 / (181)) + (6863.902137258171 / (14))) is greater than 1.9723316757956526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (14) - 2, .975) and 0.38517628314107744 = (10146.700127027936 - 6238.431885952217) / 10146.700127027936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6488484026785715 < 3.9162350657612177.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 270.07363200020393 (T) = (0 -1613.1822855208327) / Math.Sqrt((3599.0942538810455 / (299)) + (1010.6312179811364 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.6057487105472027 = (4091.7615964169854 - 1613.1822855208327) / 4091.7615964169854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Set_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8943427608796295 < 2.6594752031063984.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 126.73655226249609 (T) = (0 -1880.4764750394736) / Math.Sqrt((2284.2846733965243 / (299)) + (777.5085387910923 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.34882521465307054 = (2887.8213919748187 - 1880.4764750394736) / 2887.8213919748187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.873785359375 < 10.00077640393229.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 8.884818272891351 (T) = (0 -4787.387977416656) / Math.Sqrt((10167461.240922332 / (299)) + (2472.8696030055476 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.255464704189399 = (6430.034955165508 - 4787.387977416656) / 6430.034955165508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod4_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 989.4340177083334 < 4.456590998872767.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 456.9684604584471 (T) = (0 -935.6583718008688) / Math.Sqrt((10161.784385959712 / (299)) + (552.3506047695998 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8071497177725068 = (4851.734521690416 - 935.6583718008688) / 4851.734521690416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5030660449810604 < 5.889251538541667.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 432.7553192375978 (T) = (0 -1469.5094079183764) / Math.Sqrt((11373.62571284282 / (181)) + (878.065230858674 / (14))) is greater than 1.9723316757956526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (14) - 2, .975) and 0.7674319869920361 = (6318.6221910407585 - 1469.5094079183764) / 6318.6221910407585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Set_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.776106046527778 < 4.578477947291666.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 74.87050028472953 (T) = (0 -2714.389538871173) / Math.Sqrt((95574.50145395604 / (219)) + (2684.8777248715237 / (14))) is greater than 1.970286659282671 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (219) + (14) - 2, .975) and 0.4087486303992351 = (4590.923046324663 - 2714.389538871173) / 4590.923046324663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 91af060b086ab83ba3200fbc7049854900b08523
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCustomAttributesMethodBaseMiss - Duration of single invocation 16.41 μs 9.67 μs 0.59 0.01 False
IsDefinedClassMissInherit - Duration of single invocation 10.79 μs 6.38 μs 0.59 0.01 False
GetCustomAttributesClassHitInherit - Duration of single invocation 42.16 μs 28.70 μs 0.68 0.01 False
GetCustomAttributesMethodBaseHitInherit - Duration of single invocation 18.55 μs 12.32 μs 0.66 0.01 False
GetCustomAttributesClassHit - Duration of single invocation 16.43 μs 12.35 μs 0.75 0.01 False
GetCustomAttributesMethodOverrideHit - Duration of single invocation 4.36 μs 3.74 μs 0.86 0.02 False
GetCustomAttributesClassMissInherit - Duration of single invocation 25.11 μs 14.23 μs 0.57 0.01 False
GetCustomAttributesMethodOverrideMissInherit - Duration of single invocation 25.38 μs 18.72 μs 0.74 0.01 False
GetCustomAttributesMethodOverrideMiss - Duration of single invocation 4.44 μs 3.76 μs 0.85 0.01 False
GetCustomAttributesClassMiss - Duration of single invocation 15.99 μs 8.58 μs 0.54 0.01 False
GetCustomAttributesMethodBaseMissInherit - Duration of single invocation 16.93 μs 10.10 μs 0.60 0.01 False
IsDefinedClassMiss - Duration of single invocation 1.95 μs 1.45 μs 0.75 0.01 False
GetCustomAttributesMethodBaseHit - Duration of single invocation 4.41 μs 3.77 μs 0.86 0.01 False
GetCustomAttributesMethodOverrideHitInherit - Duration of single invocation 25.33 μs 19.26 μs 0.76 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.671020895099856 < 15.688526575223031.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 149.55505427041624 (T) = (0 -9672.773712164108) / Math.Sqrt((307913.86769932613 / (299)) + (14281.493481124202 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.41177606741006423 = (16444.032920549693 - 9672.773712164108) / 16444.032920549693 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedClassMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.377105275704809 < 10.199650384596383.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 203.15187875204026 (T) = (0 -6387.1841466239985) / Math.Sqrt((19383.9970911356 / (299)) + (6054.128722695857 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.4149519687729662 = (10917.367131768686 - 6387.1841466239985) / 10917.367131768686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassHitInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.69916149691358 < 39.958388877834686.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 115.00373853548847 (T) = (0 -28410.263925391944) / Math.Sqrt((2161278.565491652 / (299)) + (105769.08379980706 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.32983869364597346 = (42393.172592963216 - 28410.263925391944) / 42393.172592963216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.32380527039721 < 17.801772306316188.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 100.47256002368061 (T) = (0 -12404.81275795209) / Math.Sqrt((553001.5335829258 / (299)) + (24630.494528975876 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.32730788758422935 = (18440.550333500938 - 12404.81275795209) / 18440.550333500938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.353736436029605 < 15.807150865648458.
IsChangePoint: Marked as a change because one of 11/17/2022 4:58:45 AM, 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 74.03977997700474 (T) = (0 -12344.355736908949) / Math.Sqrt((738440.5381675196 / (299)) + (10742.683871443316 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.25442585894364206 = (16556.84533186598 - 12344.355736908949) / 16556.84533186598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.742087465609427 < 4.1445567474189.
IsChangePoint: Marked as a change because one of 11/17/2022 12:32:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 9.695530022233378 (T) = (0 -3777.0127812196965) / Math.Sqrt((98415.40007064436 / (299)) + (2085.7001224580845 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05314683482954296 = (3989.0163756697593 - 3777.0127812196965) / 3989.0163756697593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.232886400888479 < 23.965689113389296.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 212.38768217257856 (T) = (0 -14145.821550915616) / Math.Sqrt((413402.0714952939 / (299)) + (18452.43978837336 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.43828186965772464 = (25183.131515260244 - 14145.821550915616) / 25183.131515260244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.719250663389598 < 24.173035813806987.
IsChangePoint: Marked as a change because one of 11/16/2022 9:45:14 AM, 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 87.35451775578336 (T) = (0 -19100.64905960853) / Math.Sqrt((749086.1569061405 / (299)) + (34697.35302418938 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.24406142485551194 = (25267.4617854469 - 19100.64905960853) / 25267.4617854469 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7636957175925927 < 4.209713299597597.
IsChangePoint: Marked as a change because one of 11/17/2022 12:32:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 11.62224184459547 (T) = (0 -3785.011750788781) / Math.Sqrt((94169.25162906188 / (299)) + (2986.944950423186 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.0659242646590093 = (4052.146531145075 - 3785.011750788781) / 4052.146531145075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.58219990163013 < 15.140122258491159.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 171.72736349637142 (T) = (0 -8691.491407239542) / Math.Sqrt((279120.49521108414 / (299)) + (10998.87875423644 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.45031457080726023 = (15811.755134211477 - 8691.491407239542) / 15811.755134211477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.098165367148171 < 16.111820843801222.
IsChangePoint: Marked as a change because one of 11/17/2022 12:32:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 145.48251342255304 (T) = (0 -10046.346412643) / Math.Sqrt((303939.9379432 / (299)) + (16713.080221805438 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.40505075760315035 = (16886.056316618982 - 10046.346412643) / 16886.056316618982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedClassMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.453335879586515 < 1.827700661867095.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 95.81840716104809 (T) = (0 -1423.3184757562074) / Math.Sqrt((701.2088372682545 / (299)) + (394.66182239517343 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.2711393417919437 = (1952.7991526604187 - 1423.3184757562074) / 1952.7991526604187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.772231225961538 < 4.169910381017483.
IsChangePoint: Marked as a change because one of 11/17/2022 12:32:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 13.75639854832396 (T) = (0 -3785.357062816051) / Math.Sqrt((92073.22073136612 / (299)) + (964.5861606267281 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.06589747224154152 = (4052.399977869318 - 3785.357062816051) / 4052.399977869318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.264608785892637 < 23.834203745400593.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 85.54208189159617 (T) = (0 -19186.076944560748) / Math.Sqrt((738918.7411807446 / (299)) + (33377.112261512324 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.23703390315440778 = (25146.696588332932 - 19186.076944560748) / 25146.696588332932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 91af060b086ab83ba3200fbc7049854900b08523
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 65.77 μs 53.90 μs 0.82 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.89505079635951 < 61.8920987249255.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 60.52658653212231 (T) = (0 -53442.452533404976) / Math.Sqrt((6736583.274684435 / (299)) + (451569.5173774294 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.2095418910145446 = (67609.4683904221 - 53442.452533404976) / 67609.4683904221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions