Move Blazor Server template tests to new project

[captainsafia]

Changes in this PR

• Creates new BlazorTemplates.Test project with Blazor Server tests
• Removes skip test on Helix from ProjectTemplate config
• Move E2E helpers to shared asets

Everything in the new test project is copied from the original, the key differences are in:

• src/ProjectTemplates/test/ProjectTemplates.Tests.csproj
• src/ProjectTemplates/test/BlazorServerTemplateTest.cs → src/ProjectTemplates/BlazorTemplates.Tests/BlazorServerTemplateTest.cs
• src/ProjectTemplates/BlazorTemplates.Tests/BlazorTemplates.Tests.csproj

Addresses #21293

[HaoK]

Maybe want to consider a comment/note that mentions how we have a copy of the utility files between BlazorTemplates/ProjectTemplates, or consider just moving them to shared sources, otherwise there's a consistency issue if we update one set of files and not the other.

[captainsafia]

Maybe want to consider a comment/note that mentions how we have a copy of the utility files between BlazorTemplates/ProjectTemplates, or consider just moving them to shared sources, otherwise there's a consistency issue if we update one set of files and not the other.

I moved these into Shared sources.

[HaoK]

Looks good I think, looks like the project template tests are skipped in the output, only thing that looks a bit sketchy is that it only takes 9 seconds to run the blazor tests? Might be worth double checking that things actually ran

https://dev.azure.com/dnceng/public/_build/results?buildId=625519&view=logs&j=80f594f0-dab9-53e4-691e-a6c036ec4279&t=08be69de-580a-5f94-b421-8e6f5d337215

[SteveSandersonMS]

A lot of the things in this file don't appear to be specific to Blazor. Should this be trimmed to just the Blazor tests?

[captainsafia]

It can be. I was trying to avoid making too many changes considering the temporary nature of this PR.

[SteveSandersonMS]

Thanks for taking this on, @captainsafia!

I'm a little concerned about the amount of stuff in the following files that appears to be generic infrastructure boilerplate:

• BlazorTemplates.Tests.csproj
• Infrastructure/GenerateTestProps.targets
• Infrastructure/TemplateTests.props.in

My concern is that I don't know how we'll maintain this. It doesn't appear specific to Blazor and I don't even know what a lot of it is for (example). Some of that seems to be about EF :)

Having generic infrastructure boilerplate stuff is kind of OK as long as it's really owned by a sufficiently large part of the whole ASP.NET org. However having a separate Blazor-specific copy of it is likely to mean, in practice, that the Blazor team is on the hook for figuring out how to keep it updated to match the rest of the CI infrastructure. I don't know that we can realistically do that.

Do we have agreements and systems to ensure that anyone who updates the original test project will take care of updating BlazorTemplates.Tests too? Unless we do I'd be reluctant to even have a separate project, as it will mean duplicated effort and more costs to coordinate.

[HaoK]

I think the short/medium term goal is to have selenium working on helix, so we can delete this project and move it back into the project templates. The reason we want to move the blazor server tests out immediately, is we have no way easy way to run a few tests on azdo and a few on helix, and since template tests are pretty flaky, we really want to run them only in one place (helix or azdo not both). You don't need to worry about the blazor team owning this long term. The idea agreed to in DOI this week is for the larger team as a whole to collectively own the templates tests, and this new test project would also fall under that same umbrella...

[SteveSandersonMS]

Thanks for responding, @HaoK!

If this is broadly agreed to be technical debt, and we have a plan to fix it in the short/medium term, and during that time we're treating the infrastructure in this new project as jointly owned by the whole ASP.NET group, then that does allay my concerns.

I'm still interested in whether some of those large stretches of infrastructure boilerplate (that I don't recognize, like the EF stuff) have ways of being factored out into more shared locations to reduce the opportunities for things diverging by mistake. However I trust you and @captainsafia to make good choices about what is/isn't reasonable for that 😄

[HaoK]

I took a closer look at the shared infrastructure stuff most of that stuff is involved in getting dotnet new/dotnet net ef to work in the template tests, since those require a bunch of configuration so the directories and commands work. That said, this setup is mostly for the azdo side, as helix does its own things via environment and in the testrunner bootstrapping. Unfortunately I'm not that familiar with all of that complexity either, so I think for now, we should just duplicate those rather than investing in teasing that logic apart for this PR. There probably are a bunch of settings that aren't needed by the blazor tests, but Safia would have basically have to do several iterations of nuking all the settings and adding them back piece by piece, which I don't think is worth it

[captainsafia]

Ditto @HaoK's point. This is a temporary stopgap until we get Selenium working on Helix. Once that is done, we can refactor our test setup on the assumption that we are only targeting one environment. Hopefully, we'll be able to get Selenium working on Helix before the blazor-wasm branch is merged into master so that our Selenium-based tests can run alongside everything there.

Looks good I think, looks like the project template tests are skipped in the output, only thing that looks a bit sketchy is that it only takes 9 seconds to run the blazor tests?

Yes, it looks like the test project is detected by the actual the cases are not running, I will take a look into this.

This is actually running. I had the wrong filter in the AzDO UI.

[captainsafia]

I'll merge this for now and focus on getting #21280 squared away so that we have a consistent environment to start making improvements on.