Skip to content

Template tests are run twice azdo + helix #30756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HaoK opened this issue Mar 8, 2021 · 7 comments · Fixed by #32985
Closed

Template tests are run twice azdo + helix #30756

HaoK opened this issue Mar 8, 2021 · 7 comments · Fixed by #32985
Assignees
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework Done This issue has been fixed ✔️ Resolution: Fixed The bug or enhancement requested in this issue has been checked-in!
Milestone

Comments

@HaoK
Copy link
Member

HaoK commented Mar 8, 2021

See #30703 (comment) for more context, but basically today we run the template tests on both the helix job and the windows test job, this issue is to track following up and deciding what the long term plan is here. cc @dougbu

@HaoK HaoK added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Mar 8, 2021
@HaoK HaoK self-assigned this May 24, 2021
@HaoK
Copy link
Member Author

HaoK commented May 24, 2021

Going to address this to see if the blazor template flakiness is from azdo or helix runs

@HaoK HaoK added this to the 6.0-preview6 milestone May 24, 2021
@ghost ghost added the Working label May 24, 2021
@HaoK
Copy link
Member Author

HaoK commented May 24, 2021

Note, looks like the flakiness is almost entirely on the azdo side. There hasn't been any failures on the helix blazor template work items since 4/27. I'm going to remove the templates test job from ci as all project template tests are helix ready so the job is no longer needed

@dougbu
Copy link
Contributor

dougbu commented May 24, 2021

We see lots of failures in the Helix jobs with dotnet new timeouts in both the Blazor WASM and template tests. That's what I'm trying to debug in #32889. Just haven't had time to examine the dump I captured.

Were you looking only at the Helix jobs for 'main'❔

@HaoK
Copy link
Member Author

HaoK commented May 24, 2021

I was just looking at the quarantined-pr analytics which showed no blazortemplate.tests--net6.0 failures in the past 14 days, as compared to the pretty awful 56-97% pass rate of the azdo runs in the same time frame. I was just hoping to get rid of the extra noise since the helix blazor templates looks like something like 9/1k runs passed in the last 30 days, for like 98%+ which actually looks reasonable for them to be unquarantined since all the failures were between 4-26 and 4-27 and likely due a sdk issue

@HaoK
Copy link
Member Author

HaoK commented May 24, 2021

image

@HaoK
Copy link
Member Author

HaoK commented May 24, 2021

But the basic idea is we aren't going to be running these tests on azdo in the long term anyways, we might as well stop wasting the CPU cycles, and its adding a lot of noise to the quarantined results since we only care about the helix runs

@dougbu
Copy link
Contributor

dougbu commented May 25, 2021

the basic idea is we aren't going to be running these tests on azdo in the long term anyways

Alrighty-o I can buy that. Is #32985 ready for review❔

@ghost ghost added Done This issue has been fixed and removed Working labels May 25, 2021
@HaoK HaoK added the ✔️ Resolution: Fixed The bug or enhancement requested in this issue has been checked-in! label May 25, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jun 24, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework Done This issue has been fixed ✔️ Resolution: Fixed The bug or enhancement requested in this issue has been checked-in!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants