test(playwright): only use retries on CI #28196
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We currently use retries on CI to help catch flaky tests. However, retries are enabled even when testing locally. This is not ideal because a failing test will be re-run multiple times before erroring out which increases the time it takes to run tests locally.
liamdebeasi
requested review from
a team and
thetaPC
and removed request for
a team
averyjohnston
approved these changes
Sep 19, 2023
liamdebeasi
added a commit
that referenced
this pull request
Sep 22, 2023
Issue number: N/A --------- <!-- Please do not submit updates to dependencies unless it fixes an issue. --> <!-- Please try to limit your pull request to one type (bugfix, feature, etc). Submit multiple pull requests if needed. --> ## What is the current behavior? <!-- Please describe the current behavior that you are modifying. --> We currently use retries on CI to help catch flaky tests. However, retries are enabled even when testing locally. This is not ideal because a failing test will be re-run multiple times before erroring out which increases the time it takes to run tests locally. I typically write tests before fixes so I can verify my test is checking the correct behavior. In this case I don't need it to be re-run twice -- I already know it's supposed to fail. ## What is the new behavior? <!-- Please describe the behavior or changes that are being added by this PR. --> - Test retries are only enabled on CI ## Does this introduce a breaking change? - [ ] Yes - [x] No <!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. --> ## Other information <!-- Any other information that is important to this PR such as screenshots of how the component looks before and after the change. -->
liamdebeasi
added a commit
that referenced
this pull request
Sep 26, 2023
Issue number: N/A --------- <!-- Please do not submit updates to dependencies unless it fixes an issue. --> <!-- Please try to limit your pull request to one type (bugfix, feature, etc). Submit multiple pull requests if needed. --> ## What is the current behavior? <!-- Please describe the current behavior that you are modifying. --> We currently use retries on CI to help catch flaky tests. However, retries are enabled even when testing locally. This is not ideal because a failing test will be re-run multiple times before erroring out which increases the time it takes to run tests locally. I typically write tests before fixes so I can verify my test is checking the correct behavior. In this case I don't need it to be re-run twice -- I already know it's supposed to fail. ## What is the new behavior? <!-- Please describe the behavior or changes that are being added by this PR. --> - Test retries are only enabled on CI ## Does this introduce a breaking change? - [ ] Yes - [x] No <!-- If this introduces a breaking change, please describe the impact and migration path for existing applications below. --> ## Other information <!-- Any other information that is important to this PR such as screenshots of how the component looks before and after the change. -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue number: N/A
What is the current behavior?
We currently use retries on CI to help catch flaky tests. However, retries are enabled even when testing locally. This is not ideal because a failing test will be re-run multiple times before erroring out which increases the time it takes to run tests locally. I typically write tests before fixes so I can verify my test is checking the correct behavior. In this case I don't need it to be re-run twice -- I already know it's supposed to fail.
What is the new behavior?
Does this introduce a breaking change?
Other information