Flaky test Microsoft.AspNetCore.Components.E2ETest.Tests.DownloadAnchorTest.DownloadFileFromAnchor

[TanayParikh]

   at Microsoft.AspNetCore.Components.E2ETest.Tests.DownloadAnchorTest.DownloadFileFromAnchor() in /_/src/Components/test/E2ETest/Tests/DownloadAnchorTest.cs:line 48

https://dev.azure.com/dnceng/public/_build/results?buildId=970420&view=ms.vss-test-web.build-test-results-tab&runId=30490800&resultId=100005&paneView=debug

Extension of tests quarantined in #29652 / #29698.

[captainsafia]

Hm -- this test has been passing since it was quarantined.

I noticed that this particular failure occurred when the change to run all quarantined tests as CI runs was reverted so I wonder if that had something to do with it.

Recommend we see if this can be unquarantined next week and then verifying if failures continue once it is running in the regular CI.

Side-note: if we have a hypothesis that the environment the tests run on is affecting their flakiness then we should probably sort that out separately.

[TanayParikh]

Hm -- this test has been passing since it was quarantined.

Where are you seeing this? I'd expect to see it in the quarantine tests dashboard but I'm not finding it.

[captainsafia]

Does this work for you?

Also, looks like it got flaky again towards the end of last month. The bigger red bar is due to some issues with Selenium but the other incidents are repros of the flakiness. Bummer. :/

[TanayParikh]

Ah got it, thanks! Yeah, looks like the flakiness has returned :/

[captainsafia]

Put in a fix for this. Not sure if we are supposed to be closing these now or once 30 days have passed but I'll close and we can re-open if need be.

[HaoK]

Well we made this test consistent, but probably not in the way we want, its been full red since 3/11 which I think maps to when you made this change 😢

[captainsafia]

Boo. Maybe we can we move this test to the new Playwright E2E tests project and use the Download APIs from Playwright? As seen in https://github.com/microsoft/playwright-sharp/blob/3709d37ab2badc7745d2fadc1ce7806e141702e8/src/PlaywrightSharp.Tests/DownloadsPathTests.cs#L25.

[HaoK]

Sure sounds like a good candidate for early conversion once its ready

[captainsafia]

Moving this to Next sprint planning pending the Playwright move so we can use the APIs mentioned above.

[ghost]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. Because it's not immediately obvious that this is a bug in our framework, we would like to keep this around to collect more feedback, which can later help us determine the impact of it. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[captainsafia]

This is "Done" but was merged into the Playwright PR since the fix requires using some APIs that are only available in Playwright. Will close once that PR is merged into main.

[BrennanConroy]

merged into the Playwright PR since the fix requires using some APIs that are only available in Playwright

What PR is that? If it was merged, this test still has a 0% pass rate.

[captainsafia]

@BrennanConroy It's #32428

[captainsafia]

The PR with the Playwright tests has now been merged.