Update ChromeDriver version #36107

sebastienros · 2021-09-02T19:07:04Z

Version compatible with Chrome 93 is required

sebastienros · 2021-09-02T20:50:16Z

/azp run

azure-pipelines · 2021-09-02T20:50:37Z

Azure Pipelines successfully started running 2 pipeline(s).

sebastienros · 2021-09-02T22:52:17Z

Same "91 instead of 93" message in the quarantined step. Does this step use the Versions.props from this PR or does it need to be merged first?

https://dev.azure.com/dnceng/_apis/resources/Containers/8072696/Windows_Quarantined_Test_Results?itemPath=Windows_Quarantined_Test_Results%2FRelease%2FQuarantined%2FMicrosoft.AspNetCore.Components.E2ETests_net6.0_x64.html

dougbu · 2021-09-02T23:13:14Z

Is there a JavaScript driver that also needs to be updated to align w/ the new Chrome❔

sebastienros · 2021-09-02T23:17:09Z

Didn't find anything else related to selenium. Who would know?

sebastienros · 2021-09-03T16:10:07Z

@dougbu do you think I can merge this or should all the tests pass with that change?

sebastienros · 2021-09-03T17:13:56Z

Is there a JavaScript driver that also needs to be updated to align w/ the new Chrome

Brennan mentioned this one to me:

aspnetcore/src/Components/test/E2ETest/package.json

Line 16 in 8b30d86

"selenium-standalone": "^6.17.0"

Though the current version is 2 years old, there has been many updates in the meantime.
I can try to change it but there are probably breaking changes and I might not know how to fix it.

https://www.npmjs.com/package/selenium-standalone

sebastienros · 2021-09-03T17:34:32Z

I updated the minor version only of the npm package. It is 3 months old.

TanayParikh · 2021-09-03T17:39:24Z

I updated the minor version only of the npm package. It is 3 months old.

@javiercn is also doing a bunch of NPM upgrades: #36088

Perhaps that may help resolve some of the dependency issues too?

sebastienros · 2021-09-03T17:43:03Z

I didn't find this one though.

dougbu · 2021-09-03T18:40:10Z

@sebastienros the problem here appears to be the new package.lock file. Should use yarn instead of npm to update the information in yarn.lock. Don't check in the new package,lock file.

sebastienros · 2021-09-03T18:43:28Z

Sorry, didn't realize the file was new, thought it was updating the whole content.

sebastienros · 2021-09-03T18:49:40Z

Updated all yarn.lock files that depend on selenium package.

dougbu · 2021-09-03T20:09:21Z

eng/Versions.props

-    <SeleniumWebDriverVersion>4.0.0-beta4</SeleniumWebDriverVersion>
+    <SeleniumSupportVersion>4.0.0-rc1</SeleniumSupportVersion>
+    <SeleniumWebDriverChromeDriverVersion>93.0.4577.1500</SeleniumWebDriverChromeDriverVersion>
+    <SeleniumWebDriverVersion>4.0.0-rc1</SeleniumWebDriverVersion>


This duplicates a bit of what @javiercn is doing in #35564 but the rest supplements #36088 nicely. No conflicts to worry about 😀

dougbu · 2021-09-03T20:11:15Z

I defer to @dotnet/aspnet-blazor-eng for an actual approval

TanayParikh

LGTM. As far as I know there shouldn't be any issues in updating src/ProjectTemplates, if anything we can address later. Let's get this merged to get the quarantine pipeline orange again. 😄

sebastienros · 2021-09-03T23:05:12Z

@SteveSandersonMS @javiercn Would you have an idea about why the driver is failing, even after this change?

The result is that because there is a retry logic (3 times) in the browser connection management, with increasing timeout, the pipeline fails with a global timeout and blocks PRs. A mitigation for now would be to intercept the connection issue and not retry in this case, or reduce the max delay after a few retries.

c.f. logs:

Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.CircuitGracefulTerminationTests.ClosingTheBrowserWindow_GracefullyDisconnects_WhenNavigatingAwayFromThePage
System.InvalidOperationException : Couldn't create a Selenium remote driver client. The server is irresponsive\r\n---- System.InvalidOperationException : session not created: This version of ChromeDriver only supports Chrome version 91\nCurrent browser version is 93.0.4577.63 with binary path C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe\nBuild info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'\nSystem info: host: 'a000H89', ip: '10.0.0.11', os.name: 'Windows Server 2016', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.3'\nDriver info: driver.version: unknown\nremote stacktrace: Backtrace:\n\tOrdinal0 [0x00592DB3+2502067]\n\tOrdinal0 [0x0052C5B1+2082225]\n\tOrdinal0 [0x00432498+1057944]\n\tOrdinal0 [0x004505DA+1181146]\n\tOrdinal0 [0x0044C43C+1164348]\n\tOrdinal0 [0x00449C73+1154163]\n\tOrdinal0 [0x00479A22+1350178]\n\tOrdinal0 [0x0047967A+1349242]\n\tOrdinal0 [0x00475BEB+1334251]\n\tOrdinal0 [0x00452174+1188212]\n\tOrdinal0 [0x00453009+1191945]\n\tGetHandleVerifier [0x0070EC5C+1511084]\n\tGetHandleVerifier [0x007B8522+2205554]\n\tGetHandleVerifier [0x00613393+480739]\n\tGetHandleVerifier [0x00612579+477129]\n\tOrdinal0 [0x00531E5D+2104925]\n\tOrdinal0 [0x005363F8+2122744]\n\tOrdinal0 [0x00536537+2123063]\n\tOrdinal0 [0x0053EE53+2158163]\n\tBaseThreadInitThunk [0x76B80419+25]\n\tRtlGetAppContainerNamedObjectPath [0x76F172FD+237]\n\tRtlGetAppContainerNamedObjectPath [0x76F172CD+189]\n (SessionNotCreated)
   at Microsoft.AspNetCore.E2ETesting.BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output) in /_/src/Shared/E2ETesting/BrowserFixture.cs:line 218
   at Microsoft.AspNetCore.E2ETesting.BrowserTestBase.InitializeBrowser(String isolationContext) in /_/src/Shared/E2ETesting/BrowserTestBase.cs:line 92
   at Microsoft.AspNetCore.E2ETesting.BrowserTestBase.InitializeAsync(String isolationContext) in /_/src/Shared/E2ETesting/BrowserTestBase.cs:line 71
   at Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.CircuitGracefulTerminationTests.InitializeAsync() in /_/src/Components/test/E2ETest/ServerExecutionTests/CircuitGracefulTerminationTests.cs:line 43
----- Inner Stack Trace -----
   at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse)
   at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.WebDriver.StartSession(ICapabilities desiredCapabilities)
   at OpenQA.Selenium.WebDriver..ctor(ICommandExecutor executor, ICapabilities capabilities)
   at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(ICommandExecutor commandExecutor, ICapabilities desiredCapabilities)
   at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(Uri remoteAddress, ICapabilities desiredCapabilities, TimeSpan commandTimeout)
   at Microsoft.AspNetCore.E2ETesting.BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output)
Output:
Error initializing RemoteWebDriver: session not created: This version of ChromeDriver only supports Chrome version 91
Current browser version is 93.0.4577.63 with binary path C:\Program Files\Google\Chrome\Application\chrome.exe
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'a000H89', ip: '10.0.0.11', os.name: 'Windows Server 2016', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.3'
Driver info: driver.version: unknown

sebastienros · 2021-09-04T01:38:00Z

#36173

HaoK · 2021-09-07T16:04:06Z

/backport to release/6.0

HaoK · 2021-09-07T16:04:13Z

/backport to release/5.0

github-actions · 2021-09-07T16:04:19Z

Started backporting to release/6.0: https://github.com/dotnet/aspnetcore/actions/runs/1210151267

github-actions · 2021-09-07T16:04:28Z

Started backporting to release/5.0: https://github.com/dotnet/aspnetcore/actions/runs/1210151501

github-actions · 2021-09-07T16:05:00Z

@HaoK backporting to release/5.0 failed, the patch most likely resulted in conflicts:

$ git am --3way --ignore-whitespace --keep-non-patch changes.patch

Applying: Update ChromeDriver version
Using index info to reconstruct a base tree...
M	eng/Versions.props
Falling back to patching base and 3-way merge...
Auto-merging eng/Versions.props
CONFLICT (content): Merge conflict in eng/Versions.props
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Update ChromeDriver version
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

SteveSandersonMS · 2021-09-09T14:22:49Z

@SteveSandersonMS @javiercn Would you have an idea about why the driver is failing, even after this change?

FYI as part of #36207 I changed the code to detect if it's running in Azure Pipelines, and if so, to use the chromedriver binary that's preinstalled on the host image, which in turn should always match the Chrome binary. So we should no longer get version mismatches in CI when Chrome is updated.

However, the selenium-config.json file is still used for local development, so people will still need to update that if they want to run the tests locally and Chrome has moved on and become incompatible. I don't actually know what SeleniumWebDriverChromeDriverVersion is used for. Does it somehow cause selenium-config.json to get updated? I suspect we could even just delete selenium-config.json completely, so that when people build locally, it implicitly uses the latest available chromedriver version. Or at least that's what I think it might do. Shouldn't affect CI either way.

sebastienros · 2021-09-09T14:28:43Z

NB: I just found out there is a readme in the Components folder (https://github.com/dotnet/aspnetcore/tree/main/src/Components). Didn't think about looking for one when I wanted to run the tests, and all my attempts to run the tests failed. Something I don't see however is how to debug the tests in VS, which I was not successful in either :/

javiercn · 2021-09-09T14:30:13Z

@SteveSandersonMS @sebastienros does this still happen after the "mergethon"?

@SteveSandersonMS I don't remember the exact details of how this worked, I mainly updated the versions in eng\Versions.props and ran the tests locally to ensure they were working.

SteveSandersonMS · 2021-09-09T14:31:43Z

The E2E tests work directly from VS 2022 for me without version changes, so I'm not sure what difficulties you're seeing. That is, I can right-click on a test in the Test Explorer pane and choose Debug Test, and it does run under the debugger.

SteveSandersonMS · 2021-09-09T14:32:35Z

does this still happen after the "mergethon"?

Could you clarify what is the "this" in your question?

javiercn · 2021-09-09T14:45:33Z

@SteveSandersonMS the specific details about how we picket the selenium driver and the selenium-config.json stuff, I might have missed something on the update?

Everything seemed to work for me, so it's not clear to me if there's any issue we need solving? I might have misread the thread

sebastienros · 2021-09-09T14:48:29Z

Last week running an E2E test in VS was throwing exceptions with selenium instance issues. Today, it can't even build the solution (./startvs.cmd from the Components folder) without crashing or hanging indefinitely (VS2022).

SteveSandersonMS · 2021-09-09T15:27:56Z

@javiercn The version-picking logic I was referring to is this, so yes it is still there in main.

Last week running an E2E test in VS was throwing exceptions with selenium instance issues

It's possible you might be missing some dependency. I was running the E2E tests from VS a lot over the last couple of weeks. If you can still repro the issue, please let me know and I'll work with you to track down what's missing or wrong.

Today, it can't even build the solution (./startvs.cmd from the Components folder) without crashing or hanging indefinitely (VS2022).

That's new to me. My normal workflow is first building on the CLI via src\Components\build.cmd, then startvs, and then only building the specific projects I need from inside VS. Things are pretty sluggish inside the IDE when working on our repo so I try hard not to give VS extra things to do (e.g., building the entire solution). Are you doing something different?

sebastienros · 2021-09-09T15:38:51Z

Yep, have already reinstalled dependencies, called ./build.cmd successfully. But building in VS takes ages, so does starting a test. I'll blame my laptop.

VS still had the trace for the failing test before the "mergethon":

    1)   Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.CircuitGracefulTerminationTests.ClosingTheBrowserWindow_GracefullyDisconnects_TheCurrentCircuit 
      Duration: 1 ms

      Message: 
System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:54370)
---- System.Net.Sockets.SocketException : No connection could be made because the target machine actively refused it.

      Stack Trace: 
HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
SeleniumStandaloneServer.InitializeInstance(ITestOutputHelper output) line 181
SeleniumStandaloneServer.GetInstanceAsync(ITestOutputHelper output) line 74
BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output) line 180
BrowserTestBase.InitializeBrowser(String isolationContext) line 81
BrowserTestBase.InitializeAsync(String isolationContext) line 68
CircuitGracefulTerminationTests.InitializeAsync() line 43
----- Inner Stack Trace -----
AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
IValueTaskSource.GetResult(Int16 token)
Socket.<ConnectAsync>g__WaitForConnectWithCancellation|278_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

    2)   Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.CircuitGracefulTerminationTests.ClosingTheBrowserWindow_GracefullyDisconnects_TheCurrentCircuit 
      Duration: 1 ms

      Message: 
[Test Class Cleanup Failure (Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.CircuitGracefulTerminationTests)]: System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:54381)
---- System.Net.Sockets.SocketException : No connection could be made because the target machine actively refused it.

      Stack Trace: 
HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
SeleniumStandaloneServer.InitializeInstance(ITestOutputHelper output) line 181
SeleniumStandaloneServer.GetInstanceAsync(ITestOutputHelper output) line 74
BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output) line 180
BrowserFixture.DisposeAsync() line 74
----- Inner Stack Trace -----
AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
IValueTaskSource.GetResult(Int16 token)
Socket.<ConnectAsync>g__WaitForConnectWithCancellation|278_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

sebastienros · 2021-09-09T15:42:18Z

Now I get JS/TS errors that prevent from running the tests

javiercn · 2021-09-09T15:46:03Z

Switch to Build (ignore intellisense)

javiercn · 2021-09-09T15:54:39Z

I'm able to build and run tests without an issue from a clean environment

javiercn · 2021-09-09T15:55:22Z

@SteveSandersonMS have you seen the error that @sebastienros is pointing out? I've seen that in the past "sporadically"

SteveSandersonMS · 2021-09-09T16:08:16Z

I've seen the "connection refused" error occur when the Selenium (Java) server was left running by a previous test run attempt, and I had to kill all the Java and Node processes in the OS task manager. I also wonder if this might happen if you actually didn't have a usable java.exe on the system path.

sebastienros · 2021-09-09T16:55:47Z

Worked on the command line at least with ./build.cmd -test -c release (not the documented dotnet command which fails due to not finding the local dotnet sdk). 48 minutes though :/

  Running tests: C:\code\aspnetcore\src\Components\test\E2ETest\bin\release\net6.0\Microsoft.AspNetCore.Components.E2ETests.dll [net6.0|x64]
  Tests succeeded: C:\code\aspnetcore\src\Components\test\E2ETest\bin\release\net6.0\Microsoft.AspNetCore.Components.E2ETests.dll [net6.0|x64]

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:48:21.87
build.cmd completed

SteveSandersonMS · 2021-09-09T17:52:55Z

Yeah, it's not something I'd run locally just for the entertainment value. But it's only 23 mins or so in CI :) https://dev.azure.com/dnceng/public/_build/results?buildId=1352586&view=logs&j=fe94c0c9-bb8c-5d6f-3b51-887173cc2f5c&t=2128ef37-3fe5-5980-876d-edb5d5b391c1

Update ChromeDriver version

cf05098

Version compatible with Chrome 93 is required

sebastienros requested a review from dougbu as a code owner September 2, 2021 19:07

ghost added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Sep 2, 2021

Update Versions.props

c38328b

sebastienros mentioned this pull request Sep 2, 2021

Upgrade Selenium Version #35949

Merged

Update package.json

9a6351e

sebastienros requested a review from a team as a code owner September 3, 2021 17:33

Update package-lock

e402b80

Update yarn.lock files

a7eda07

sebastienros requested a review from Pilchie as a code owner September 3, 2021 18:48

dougbu reviewed Sep 3, 2021

View reviewed changes

TanayParikh approved these changes Sep 3, 2021

View reviewed changes

sebastienros merged commit d631d7f into main Sep 3, 2021

sebastienros deleted the sebros/seleniumpackage branch September 3, 2021 20:28

ghost added this to the 7.0-preview1 milestone Sep 3, 2021

github-actions bot mentioned this pull request Sep 7, 2021

[release/6.0] Update ChromeDriver version #36232

Merged

github-actions bot locked and limited conversation to collaborators Dec 8, 2023

Update ChromeDriver version #36107

Update ChromeDriver version #36107

Conversation

sebastienros commented Sep 2, 2021 • edited Loading

sebastienros commented Sep 2, 2021

azure-pipelines bot commented Sep 2, 2021

sebastienros commented Sep 2, 2021 • edited Loading

dougbu commented Sep 2, 2021

sebastienros commented Sep 2, 2021

sebastienros commented Sep 3, 2021

sebastienros commented Sep 3, 2021

sebastienros commented Sep 3, 2021

TanayParikh commented Sep 3, 2021

sebastienros commented Sep 3, 2021

dougbu commented Sep 3, 2021

sebastienros commented Sep 3, 2021

sebastienros commented Sep 3, 2021

dougbu Sep 3, 2021

Choose a reason for hiding this comment

dougbu commented Sep 3, 2021

TanayParikh left a comment

Choose a reason for hiding this comment

sebastienros commented Sep 3, 2021

sebastienros commented Sep 4, 2021

HaoK commented Sep 7, 2021

HaoK commented Sep 7, 2021

github-actions bot commented Sep 7, 2021

github-actions bot commented Sep 7, 2021

github-actions bot commented Sep 7, 2021

SteveSandersonMS commented Sep 9, 2021

sebastienros commented Sep 9, 2021

javiercn commented Sep 9, 2021

SteveSandersonMS commented Sep 9, 2021

SteveSandersonMS commented Sep 9, 2021

javiercn commented Sep 9, 2021

sebastienros commented Sep 9, 2021

SteveSandersonMS commented Sep 9, 2021

sebastienros commented Sep 9, 2021

sebastienros commented Sep 9, 2021

javiercn commented Sep 9, 2021

javiercn commented Sep 9, 2021 • edited Loading

javiercn commented Sep 9, 2021

SteveSandersonMS commented Sep 9, 2021

sebastienros commented Sep 9, 2021

SteveSandersonMS commented Sep 9, 2021

sebastienros commented Sep 2, 2021 •

edited

Loading

sebastienros commented Sep 2, 2021 •

edited

Loading

javiercn commented Sep 9, 2021 •

edited

Loading