Flaky tests: Microsoft.AspNetCore.Components.E2ETests

[halter73]

Example failure:

CanInvokeDotNetMethods

OpenQA.Selenium.BrowserAssertFailedException : Xunit.Sdk.NotEmptyException: Assert.NotEmpty() Failure
   at Xunit.Assert.NotEmpty(IEnumerable collection) in C:\Dev\xunit\xunit\src\xunit.assert\Asserts\CollectionAsserts.cs:line 331
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.<>c__DisplayClass15_0.<Exists>b__0() in /_/src/Shared/E2ETesting/WaitAssert.cs:line 72
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.<>c__DisplayClass18_0`1.<WaitAssertCore>b__0(IWebDriver ) in //src/Shared/E2ETesting/WaitAssert.cs:line 109
Screen shot captured at 'F:\workspace\_work\1\s\artifacts\TestResults\Release\Microsoft.AspNetCore.Components.E2ETests\1d1f4408eb7d409183a8e135bc9d9119.png'
Encountered browser errors
[2021-01-22T13:59:50Z] [Debug] http://127.0.0.1:53114/subdir/_framework/dotnet.6.0.0-alpha.1.21061.20.js 0:140953 "mono_wasm_runtime_ready" "fe00e07a-5519-4dfe-b35a-f867dbaf2e28"
[2021-01-22T13:59:51Z] [Info] http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js 0:44430 "Starting interop invocations."
[2021-01-22T13:59:51Z] [Info] http://127.0.0.1:53114/subdir/js/jsinteroptests.js 6:12 "Invoking void sync methods."
[2021-01-22T13:59:51Z] [Severe] http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js 0:44469 "crit: Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]
Unhandled exception rendering component: Expected string argument
Error: Expected string argument
    at Object.js_string_to_mono_string (http://127.0.0.1:53114/subdir/_framework/dotnet.6.0.0-alpha.1.21061.20.js:1:171532)
    at Object.converter_ssss [as compiled_function] (https://mono-wasm.invalid/converter_ssss:19:18)
    at managed__Microsoft_AspNetCore_Components_WebAssembly__Microsoft_AspNetCore_Components_WebAssembly_Services_DefaultWebAssemblyJSRuntime_InvokeDotNet (https://mono-wasm.invalid/managed__Microsoft_AspNetCore_Components_WebAssembly__Microsoft_AspNetCore_Components_WebAssembly_Services_DefaultWebAssemblyJSRuntime_InvokeDotNet:10:33)
    at Object.invokeDotNetFromJS (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:51643)
    at p (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:2157)
    at Object.e.invokeMethod (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:3273)
    at Object.invokeDotNetInteropMethodsAsync (http://127.0.0.1:53114/subdir/js/jsinteroptests.js:8:12)
    at http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:3942
    at new Promise (\u003Canonymous>)
    at Object.beginInvokeJSFromDotNet (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:3908)
Microsoft.JSInterop.JSException: Expected string argument
Error: Expected string argument
    at Object.js_string_to_mono_string (http://127.0.0.1:53114/subdir/_framework/dotnet.6.0.0-alpha.1.21061.20.js:1:171532)
    at Object.converter_ssss [as compiled_function] (https://mono-wasm.invalid/converter_ssss:19:18)
    at managed__Microsoft_AspNetCore_Components_WebAssembly__Microsoft_AspNetCore_Components_WebAssembly_Services_DefaultWebAssemblyJSRuntime_InvokeDotNet (https://mono-wasm.invalid/managed__Microsoft_AspNetCore_Components_WebAssembly__Microsoft_AspNetCore_Components_WebAssembly_Services_DefaultWebAssemblyJSRuntime_InvokeDotNet:10:33)
    at Object.invokeDotNetFromJS (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:51643)
    at p (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:2157)
    at Object.e.invokeMethod (http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:3273)
    at Object.invokeDotNetInteropMethodsAsync (http://127.0.0.1:53114/subdir/js/jsinteroptests.js:8:12)
    at http://127.0.0.1:53114/subdir/_framework/blazor.webassembly.js:1:3942
    at new Promise (\u003Canonymous>)
    at Object.beginInvokeJSFromDotNet (http://127.0.0.1:53114/s


   at Microsoft.AspNetCore.E2ETesting.WaitAssert.WaitAssertCore[TResult](IWebDriver driver, Func`1 assertion, TimeSpan timeout) in /_/src/Shared/E2ETesting/WaitAssert.cs:line 129
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.Exists(IWebDriver driver, By finder, TimeSpan timeout) in /_/src/Shared/E2ETesting/WaitAssert.cs:line 67
   at Microsoft.AspNetCore.Components.E2ETest.Tests.InteropTest.CanInvokeDotNetMethods() in /_/src/Components/test/E2ETest/Tests/InteropTest.cs:line 137
----- Inner Stack Trace -----
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.<>c__DisplayClass15_0.<Exists>b__0() in /_/src/Shared/E2ETesting/WaitAssert.cs:line 72
   at Microsoft.AspNetCore.E2ETesting.WaitAssert.<>c__DisplayClass18_0`1.<WaitAssertCore>b__0(IWebDriver _) in /_/src/Shared/E2ETesting/WaitAssert.cs:line 109

https://dev.azure.com/dnceng/public/_build/results?buildId=961819&view=ms.vss-test-web.build-test-results-tab&runId=30306712&resultId=100473&paneView=debug

https://dev.azure.com/dnceng/public/_build/results?buildId=961240&view=ms.vss-test-web.build-test-results-tab&runId=30294640&resultId=100388

https://dev.azure.com/dnceng/public/_build/results?buildId=961243&view=ms.vss-test-web.build-test-results-tab&runId=30294740&resultId=100524

[halter73]

The latest aspnetcore-ci run had 7 different Selenium tests fail with similar errors. I'm now wondering if it was the right move to quarantine CanInvokeDotNetMethods (#29554).

https://dev.azure.com/dnceng/public/_build/results?buildId=962800&view=results

[halter73]

I've already started rerunning the tests since it's blocking a dependency update PR, but there are more failures at https://dev.azure.com/dnceng/public/_build/results?buildId=961819&view=logs&j=6c33d704-163a-5170-38fa-caa7bc99f17c&t=4a2ac175-72f0-54eb-c8b6-eb1e5d86c7d5&s=b49e7e65-b64b-557c-018e-3fd79c15ded6

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[HaoK]

Maybe be good candidates for p2w