Skip to content

test: update screenshot system-tests flake so that it no longer compares images for uniqueness that represent the same state #20463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 4, 2022
Merged

test: update screenshot system-tests flake so that it no longer compares images for uniqueness that represent the same state #20463

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9cfdca0
test: update screenshot system-tests flake so that it no longer compa…
ryanthemanuel Mar 3, 2022
543fdff
Merge branch 'develop' into ryanm/chore/fix-system-screenshot-flake
ryanthemanuel Mar 3, 2022
378394b
Add a comment for a previous flake
ryanthemanuel Mar 3, 2022
4d7a110
Tweak the screenshot logic a bit
ryanthemanuel Mar 3, 2022
70fe7b4
Revert experiment
ryanthemanuel Mar 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/driver/cypress/integration/commands/net_stubbing_spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ const testFail = (cb, expectedDocsUrl = 'https://on.cypress.io/intercept') => {
})
}

// Due to an issue with requests leaking between tests, we need to ensure that routes are unique from test to test
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot to add this comment on a previous PR so I thought I'd go ahead and get it on this one.

// See https://github.com/cypress-io/cypress/issues/20397
let routeCount = 0
const uniqueRoute = (route) => {
routeCount += 1
Expand Down
4 changes: 3 additions & 1 deletion system-tests/test/screenshots_spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,9 @@ describe('e2e screenshots', () => {
fs.statAsync(screenshot4).get('size'),
fs.statAsync(screenshot5).get('size'),
fs.statAsync(screenshot6).get('size'),
fs.statAsync(screenshot7).get('size'),
// Ignore comparing 6 and 7 since they can sometimes be the same since we take the screenshot as close to the failure as possible and
// the test run error may not have displayed yet. Leaving this commented in case we want to change this behavior in the future
// fs.statAsync(screenshot7).get('size'),
fs.statAsync(screenshot8).get('size'),
fs.statAsync(screenshot9).get('size'),
])
Expand Down