Test failure System.Reflection.Tests.ModuleTests.GetMethods

[VincentBu]

Run: runtime-coreclr libraries-jitstress 20210406.1

Failed test:

net6.0-Linux-Release-arm-CoreCLR_checked-jitstress2-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440
 -System.Reflection.Tests.ModuleTests.GetMethods

Error message:

Showing first 10 differences\n
  Position 0: Expected: TestMethodFoo, Actual: TestMethodBar
  Position 2: Expected: TestMethodBar, Actual: TestMethodFoo
Total number of differences: 2 out of 3


Stack trace
   at System.AssertExtensions.SequenceEqual[T](ReadOnlySpan`1 expected, ReadOnlySpan`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 473
   at System.AssertExtensions.SequenceEqual[T](Span`1 expected, Span`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 478
   at System.AssertExtensions.SequenceEqual[T](T[] expected, T[] actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 480
   at System.Reflection.Tests.ModuleTests.GetMethods() in /_/src/libraries/System.Runtime/tests/System/Reflection/ModuleTests.cs:line 230

[BruceForstall]

Looks like this test was added with #50076

[BruceForstall]

@MichalStrehovsky @erafalovsky PTAL

[MichalStrehovsky]

@erafalovsky would you be interested in fixing this? Maybe we can just sort the sequence before comparing. I forgot that the reflection stack doesn't guarantee ordering of members.

From the docs:

Remarks

The GetMethods method does not return methods in a particular order, such as alphabetical or declaration order. Your code must not depend on the order in which methods are returned, because that order varies.

[VincentBu]

Failed again in runtime-coreclr libraries-jitstress 20210428.1

Failed test:

net6.0-Linux-Release-arm-CoreCLR_checked-no_tiered_compilation-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440
 -System.Reflection.Tests.ModuleTests.GetMethods

Error message:

Showing first 10 differences
 Position 0: Expected: TestMethodFoo, Actual: TestMethodBar
 Position 2: Expected: TestMethodBar, Actual: TestMethodFoo
Total number of differences: 2 out of 3

Stack trace
   at System.AssertExtensions.SequenceEqual[T](ReadOnlySpan`1 expected, ReadOnlySpan`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 473
   at System.AssertExtensions.SequenceEqual[T](Span`1 expected, Span`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 478
   at System.AssertExtensions.SequenceEqual[T](T[] expected, T[] actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 480
   at System.Reflection.Tests.ModuleTests.GetMethods() in /_/src/libraries/System.Runtime/tests/System/Reflection/ModuleTests.cs:line 238

[VincentBu]

Failed again in runtime-coreclr libraries-jitstress 20210512.1

Failed test:

net6.0-Linux-Release-arm64-CoreCLR_checked-jitstress2-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-a45aeeb-20190620155855
 -System.Reflection.Tests.ModuleTests.GetMethods

Error message:

Showing first 10 differences
Position 0: Expected: TestMethodFoo, Actual: TestMethodBar
Position 2: Expected: TestMethodBar, Actual: TestMethodFoo
Total number of differences: 2 out of 3


Stack trace
   at System.AssertExtensions.SequenceEqual[T](ReadOnlySpan`1 expected, ReadOnlySpan`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 473
   at System.AssertExtensions.SequenceEqual[T](Span`1 expected, Span`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 478
   at System.AssertExtensions.SequenceEqual[T](T[] expected, T[] actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 480

[VincentBu]

Failed again in runtime-coreclr libraries-jitstress2-jitstressregs 20210605.1

Failed test:

net6.0-windows-Release-x86-CoreCLR_checked-jitstress2_jitstressregs2-Windows.10.Amd64.Open

- System.Reflection.Tests.ModuleTests.GetMethods

Error message:

Showing first 10 differences
Position 0: Expected: TestMethodFoo, Actual: TestMethodBar
Position 2: Expected: TestMethodBar, Actual: TestMethodFoo
Total number of differences: 2 out of 3


Stack trace
   at System.AssertExtensions.SequenceEqual[T](ReadOnlySpan`1 expected, ReadOnlySpan`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 473
   at System.AssertExtensions.SequenceEqual[T](Span`1 expected, Span`1 actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 491
   at System.AssertExtensions.SequenceEqual[T](T[] expected, T[] actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 493
   at System.Reflection.Tests.ModuleTests.GetMethods() in /_/src/libraries/System.Runtime/tests/System/Reflection/ModuleTests.cs:line 241