Quarantine Microsoft.AspNetCore.Components.E2ETests

[wtgodbe]

#29553

[HaoK]

@dotnet/aspnet-blazor-eng this seems pretty risky to quarantine all component tests, would we have any pr validation of components at all if we quarantine the whole project?

In practice I believe this would basically mean as long as the PR for a change in components compiles, it would be green :)

[pranavkm]

@HaoK are you able to figure out if this is a infrastructure / product / dependency issue? It's very worrying to quarantine all of Blazor's end-to-end test coverage.

[captainsafia]

There was a thread on this over on the Teams channel. Apparently, these tests have been failing frequently since the 22nd.

We triaged the issue for this for preview2, but not sure when it will be assigned/worked on.

It's very worrying to quarantine all of Blazor's end-to-end test coverage.

Ditto. If we can figure out the root issue soon, we can address instead of quarantining. If not, I'm favor of quarantining and following up with the fix. Especially because aggressive failures here will end up increasing the amount of time people need to wait before being able to merge their PRs.

Is it possible that these failures are related to the recent changes in component tests?

[HaoK]

Do we think the jump in failures might be related to the unquarantine of a bunch of tests in 3f7b205#diff-161948171825dc8c1b30fdc7c517d8e960d462359d1959cd30aa428ad16172f2 that was 6 days ago?

[HaoK]

Maybe we just try rolling back that PR as a first step to see if that brings us back to a better place?

[HaoK]

I opened #29652 to see if that helps, basically it requarantines only the component tests in that commit

[captainsafia]

Are they the same tests that are currently failing?

I'd find it rather surprising if the tests currently failing are the ones that were unquarantined since they would've had to pass the 30+ days passing requirement.

[HaoK]

I think maybe we should tweak our quarantined runs to run all tests not just quarantined tests, it wouldn't be surprising if there are tests that behave differently when we only run a small set of tests, we for sure would miss any poorly written tests that affect other tests intentionally or not.

[HaoK]

But at this point it seems like we have the pretty bad options of just not testing components at all, or trying to rollback to some kind of known good state.

[HaoK]

At least 3 of the 10 component tests that failed in the last week were in the batch we recently unquarantined:

EventDuringBatchRendering_CanTriggerDOMEvents()
BubblingStandardEvent_FiredOnElementWithoutHandler
CanUploadAndConvertImageFile

[mkArtakMSFT]

@wtgodbe thanks for the PR. Let's take the PR @HaoK has instead. We want to continue improving the stability here and this won't help much.