Skip to content

[Blazor][Templates] Replaces selenium with playwright for our E2E Tests #29873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 11, 2021
Merged

[Blazor][Templates] Replaces selenium with playwright for our E2E Tests #29873

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0f79e6c
[Blazor][Templates] Replaces selenium with playwright for our E2E Tests
javiercn Oct 23, 2020
9854bbb
Validate slowing down tests improves reliability
javiercn Feb 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .config/dotnet-tools.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@
"commands": [
"dotnet-format"
]
},
"playwright-sharp-tool": {
"version": "0.170.2",
"commands": [
"playwright-sharp"
]
}
}
}
18 changes: 18 additions & 0 deletions AspNetCore.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1568,6 +1568,10 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "IIS.Common.TestLib", "src\S
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "InProcessNewShimWebSite", "src\Servers\IIS\IIS\test\testassets\InProcessNewShimWebSite\InProcessNewShimWebSite.csproj", "{22EA0993-8DFC-40C2-8481-8E85E21EFB56}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "BrowserTesting", "BrowserTesting", "{8F33439F-5532-45D6-8A44-20EF9104AA9D}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.AspNetCore.BrowserTesting", "src\Shared\BrowserTesting\src\Microsoft.AspNetCore.BrowserTesting.csproj", "{B739074E-6652-4F5B-B37E-775DC2245FEC}"
EndProject
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😃 looks much better

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor but FAE04EC0-301F-11D3-BF4B-00C04F79EFBC is the old C# GUID. Should be 9A19103F-16F7-4668-BE54-9A1E7A4F7556

Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -7435,6 +7439,18 @@ Global
{22EA0993-8DFC-40C2-8481-8E85E21EFB56}.Release|x64.Build.0 = Release|x64
{22EA0993-8DFC-40C2-8481-8E85E21EFB56}.Release|x86.ActiveCfg = Release|x86
{22EA0993-8DFC-40C2-8481-8E85E21EFB56}.Release|x86.Build.0 = Release|x86
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Debug|x64.ActiveCfg = Debug|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Debug|x64.Build.0 = Debug|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Debug|x86.ActiveCfg = Debug|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Debug|x86.Build.0 = Debug|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Release|Any CPU.ActiveCfg = Release|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Release|Any CPU.Build.0 = Release|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Release|x64.ActiveCfg = Release|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Release|x64.Build.0 = Release|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Release|x86.ActiveCfg = Release|Any CPU
{B739074E-6652-4F5B-B37E-775DC2245FEC}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -8209,6 +8225,8 @@ Global
{9B8F871E-ED33-4D2F-AA49-E39D9299EC85} = {41BB7BA4-AC08-4E9A-83EA-6D587A5B951C}
{7F295396-DBBD-40A5-A645-10004D1324DA} = {41BB7BA4-AC08-4E9A-83EA-6D587A5B951C}
{22EA0993-8DFC-40C2-8481-8E85E21EFB56} = {41BB7BA4-AC08-4E9A-83EA-6D587A5B951C}
{8F33439F-5532-45D6-8A44-20EF9104AA9D} = {5F0044F2-4C66-46A8-BD79-075F001AA034}
{B739074E-6652-4F5B-B37E-775DC2245FEC} = {8F33439F-5532-45D6-8A44-20EF9104AA9D}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {3E8720B3-DBDD-498C-B383-2CC32A054E8F}
Expand Down
1 change: 1 addition & 0 deletions eng/Dependencies.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ and are generated based on the last package release.
<LatestPackageReference Include="Newtonsoft.Json.Bson" />
<LatestPackageReference Include="NSwag.ApiDescription.Client" />
<LatestPackageReference Include="NuGet.Versioning" />
<LatestPackageReference Include="PlaywrightSharp" />
<LatestPackageReference Include="Polly" />
<LatestPackageReference Include="Polly.Extensions.Http" />
<LatestPackageReference Include="Selenium.Support" />
Expand Down
1 change: 1 addition & 0 deletions eng/Versions.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@
<NewtonsoftJsonBsonVersion>1.0.2</NewtonsoftJsonBsonVersion>
<NewtonsoftJsonVersion>12.0.2</NewtonsoftJsonVersion>
<NSwagApiDescriptionClientVersion>13.0.4</NSwagApiDescriptionClientVersion>
<PlaywrightSharpVersion>0.180.0</PlaywrightSharpVersion>
<PollyExtensionsHttpVersion>3.0.0</PollyExtensionsHttpVersion>
<PollyVersion>7.1.0</PollyVersion>
<SeleniumSupportVersion>4.0.0-alpha07</SeleniumSupportVersion>
Expand Down
5 changes: 3 additions & 2 deletions src/ProjectTemplates/BlazorTemplates.Tests/AssemblyInfo.AssemblyFixtures.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information.

using Microsoft.AspNetCore.E2ETesting;
using Microsoft.AspNetCore.Testing;
using ProjectTemplates.Tests.Infrastructure;
using Templates.Test;
using Templates.Test.Helpers;

[assembly: AssemblyFixture(typeof(ProjectFactoryFixture))]
[assembly: AssemblyFixture(typeof(SeleniumStandaloneServer))]
[assembly: AssemblyFixture(typeof(PlaywrightFixture<BlazorServerTemplateTest>))]

142 changes: 81 additions & 61 deletions src/ProjectTemplates/BlazorTemplates.Tests/BlazorServerTemplateTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,44 @@
// Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Threading;
using System.Runtime.InteropServices;
using System.Threading.Tasks;
using Microsoft.AspNetCore.E2ETesting;
using Microsoft.AspNetCore.Testing;
using OpenQA.Selenium;
using Microsoft.AspNetCore.BrowserTesting;
using PlaywrightSharp;
using ProjectTemplates.Tests.Infrastructure;
using Templates.Test.Helpers;
using Xunit;
using Xunit.Abstractions;

namespace Templates.Test
{
public class BlazorServerTemplateTest : BrowserTestBase
public class BlazorServerTemplateTest : BlazorTemplateTest
{
public BlazorServerTemplateTest(ProjectFactoryFixture projectFactory, BrowserFixture browserFixture, ITestOutputHelper output) : base(browserFixture, output)
public BlazorServerTemplateTest(ProjectFactoryFixture projectFactory, PlaywrightFixture<BlazorServerTemplateTest> fixture, ITestOutputHelper output)
: base(fixture)
{
ProjectFactory = projectFactory;
ProjectFactory = projectFactory; ;
Output = output;
BrowserContextInfo = new ContextInformation(CreateFactory(output));
}

public ProjectFactoryFixture ProjectFactory { get; set; }

public ITestOutputHelper Output { get; }
public ContextInformation BrowserContextInfo { get; }
public Project Project { get; private set; }

[Fact]
public async Task BlazorServerTemplateWorks_NoAuth()

[Theory]
[InlineData(BrowserKind.Chromium)]
public async Task BlazorServerTemplateWorks_NoAuth(BrowserKind browserKind)
{
// Additional arguments are needed. See: https://github.com/dotnet/aspnetcore/issues/24278
Environment.SetEnvironmentVariable("EnableDefaultScopedCssItems", "true");

Project = await ProjectFactory.GetOrCreateProject("blazorservernoauth", Output);
Project = await ProjectFactory.GetOrCreateProject("blazorservernoauth" + browserKind.ToString(), Output);

var createResult = await Project.RunDotNetNewAsync("blazorserver");
Assert.True(0 == createResult.ExitCode, ErrorMessages.GetFailedProcessMessage("create/restore", Project, createResult));
Expand All @@ -47,21 +54,28 @@ public async Task BlazorServerTemplateWorks_NoAuth()
var buildResult = await Project.RunDotNetBuildAsync();
Assert.True(0 == buildResult.ExitCode, ErrorMessages.GetFailedProcessMessage("build", Project, buildResult));

await using var browser = Fixture.BrowserManager.IsAvailable(browserKind) ?
await Fixture.BrowserManager.GetBrowserInstance(browserKind, BrowserContextInfo) :
null;

using (var aspNetProcess = Project.StartBuiltProjectAsync())
{
Assert.False(
aspNetProcess.Process.HasExited,
ErrorMessages.GetFailedProcessMessageOrEmpty("Run built project", Project, aspNetProcess.Process));

await aspNetProcess.AssertStatusCode("/", HttpStatusCode.OK, "text/html");
if (BrowserFixture.IsHostAutomationSupported())

if (Fixture.BrowserManager.IsAvailable(browserKind))
{
aspNetProcess.VisitInBrowser(Browser);
TestBasicNavigation();
var page = await browser.NewPageAsync();
await aspNetProcess.VisitInBrowserAsync(page);
await TestBasicNavigation(page);
await page.CloseAsync();
}
else
{
BrowserFixture.EnforceSupportedConfigurations();
EnsureBrowserAvailable(browserKind);
}
}

Expand All @@ -72,27 +86,31 @@ public async Task BlazorServerTemplateWorks_NoAuth()
ErrorMessages.GetFailedProcessMessageOrEmpty("Run published project", Project, aspNetProcess.Process));

await aspNetProcess.AssertStatusCode("/", HttpStatusCode.OK, "text/html");
if (BrowserFixture.IsHostAutomationSupported())
if (Fixture.BrowserManager.IsAvailable(browserKind))
{
aspNetProcess.VisitInBrowser(Browser);
TestBasicNavigation();
var page = await browser.NewPageAsync();
await aspNetProcess.VisitInBrowserAsync(page);
await TestBasicNavigation(page);
await page.CloseAsync();
}
else
{
BrowserFixture.EnforceSupportedConfigurations();
EnsureBrowserAvailable(browserKind);
}
}
}

public static IEnumerable<object[]> BlazorServerTemplateWorks_IndividualAuthData =>
BrowserManager.WithBrowsers(new[] { BrowserKind.Chromium }, true, false);

[Theory]
[InlineData(true)]
[InlineData(false)]
public async Task BlazorServerTemplateWorks_IndividualAuth(bool useLocalDB)
[MemberData(nameof(BlazorServerTemplateWorks_IndividualAuthData))]
public async Task BlazorServerTemplateWorks_IndividualAuth(BrowserKind browserKind, bool useLocalDB)
{
// Additional arguments are needed. See: https://github.com/dotnet/aspnetcore/issues/24278
Environment.SetEnvironmentVariable("EnableDefaultScopedCssItems", "true");

Project = await ProjectFactory.GetOrCreateProject("blazorserverindividual" + (useLocalDB ? "uld" : ""), Output);
Project = await ProjectFactory.GetOrCreateProject("blazorserverindividual" + browserKind + (useLocalDB ? "uld" : ""), Output);

var createResult = await Project.RunDotNetNewAsync("blazorserver", auth: "Individual", useLocalDB: useLocalDB);
Assert.True(0 == createResult.ExitCode, ErrorMessages.GetFailedProcessMessage("create/restore", Project, createResult));
Expand All @@ -107,84 +125,86 @@ public async Task BlazorServerTemplateWorks_IndividualAuth(bool useLocalDB)
var buildResult = await Project.RunDotNetBuildAsync();
Assert.True(0 == buildResult.ExitCode, ErrorMessages.GetFailedProcessMessage("build", Project, buildResult));

var browser = !Fixture.BrowserManager.IsAvailable(browserKind) ?
null :
await Fixture.BrowserManager.GetBrowserInstance(browserKind, BrowserContextInfo);

using (var aspNetProcess = Project.StartBuiltProjectAsync())
{
Assert.False(
aspNetProcess.Process.HasExited,
ErrorMessages.GetFailedProcessMessageOrEmpty("Run built project", Project, aspNetProcess.Process));

await aspNetProcess.AssertStatusCode("/", HttpStatusCode.OK, "text/html");
if (BrowserFixture.IsHostAutomationSupported())
if (Fixture.BrowserManager.IsAvailable(browserKind))
{
aspNetProcess.VisitInBrowser(Browser);
TestBasicNavigation();
var page = await browser.NewPageAsync();
await aspNetProcess.VisitInBrowserAsync(page);
await TestBasicNavigation(page);
await page.CloseAsync();
}
else
{
BrowserFixture.EnforceSupportedConfigurations();
EnsureBrowserAvailable(browserKind);
}
}


using (var aspNetProcess = Project.StartPublishedProjectAsync())
{
Assert.False(
aspNetProcess.Process.HasExited,
ErrorMessages.GetFailedProcessMessageOrEmpty("Run published project", Project, aspNetProcess.Process));

await aspNetProcess.AssertStatusCode("/", HttpStatusCode.OK, "text/html");
if (BrowserFixture.IsHostAutomationSupported())
if (Fixture.BrowserManager.IsAvailable(browserKind))
{
aspNetProcess.VisitInBrowser(Browser);
TestBasicNavigation();
var page = await browser.NewPageAsync();
await aspNetProcess.VisitInBrowserAsync(page);
await TestBasicNavigation(page);
await page.CloseAsync();
}
else
{
EnsureBrowserAvailable(browserKind);
}
}
}

private void TestBasicNavigation()
private async Task TestBasicNavigation(IPage page)
{
var retries = 3;
var connected = false;
do
{
try
{
Browser.Contains("Information: WebSocket connected to",
() => string.Join(Environment.NewLine, Browser.GetBrowserLogs(LogLevel.Info).Select(b => b.Message)));
connected = true;
}
catch (TimeoutException) when(retries-- > 0)
{
Browser.Navigate().Refresh();
}
} while (!connected && retries > 0);
var socket = BrowserContextInfo.Pages[page].WebSockets.SingleOrDefault() ??
(await page.WaitForEventAsync(PageEvent.WebSocket)).WebSocket;

// Receive render batch
await socket.WaitForEventAsync(WebSocketEvent.FrameReceived);
await socket.WaitForEventAsync(WebSocketEvent.FrameSent);

// JS interop call to intercept navigation
await socket.WaitForEventAsync(WebSocketEvent.FrameReceived);
await socket.WaitForEventAsync(WebSocketEvent.FrameSent);

Browser.Exists(By.TagName("ul"));
await page.WaitForSelectorAsync("ul");
// <title> element gets project ID injected into it during template execution
Browser.Equal(Project.ProjectName.Trim(), () => Browser.Title.Trim());
Assert.Equal(Project.ProjectName.Trim(), (await page.GetTitleAsync()).Trim());

// Initially displays the home page
Browser.Equal("Hello, world!", () => Browser.FindElement(By.TagName("h1")).Text);
await page.WaitForSelectorAsync("h1 >> text=Hello, world!");

// Can navigate to the counter page
Browser.Click(By.PartialLinkText("Counter"));
Browser.Contains("counter", () => Browser.Url);
Browser.Equal("Counter", () => Browser.FindElement(By.TagName("h1")).Text);
await page.ClickAsync("a[href=counter] >> text=Counter");
await page.WaitForSelectorAsync("h1+p >> text=Current count: 0");

// Clicking the counter button works
Browser.Equal("Current count: 0", () => Browser.FindElement(By.CssSelector("h1 + p")).Text);
Browser.Click(By.CssSelector("p+button"));
Browser.Equal("Current count: 1", () => Browser.FindElement(By.CssSelector("h1 + p")).Text);
await page.ClickAsync("p+button >> text=Click me");
await page.WaitForSelectorAsync("h1+p >> text=Current count: 1");

// Can navigate to the 'fetch data' page
Browser.Click(By.PartialLinkText("Fetch data"));
Browser.Contains("fetchdata", () => Browser.Url);
Browser.Equal("Weather forecast", () => Browser.FindElement(By.TagName("h1")).Text);
await page.ClickAsync("a[href=fetchdata] >> text=Fetch data");
await page.WaitForSelectorAsync("h1 >> text=Weather forecast");

// Asynchronously loads and displays the table of weather forecasts
Browser.Exists(By.CssSelector("table>tbody>tr"));
Browser.Equal(5, () => Browser.FindElements(By.CssSelector("p+table>tbody>tr")).Count);
await page.WaitForSelectorAsync("table>tbody>tr");
Assert.Equal(5, (await page.QuerySelectorAllAsync("p+table>tbody>tr")).Count());
}

[Theory]
Expand Down
Loading