Skip to content

Find way to make flaky tests less flaky on CI #3587

Closed
@etpinard

Description

@etpinard

Since about a week ago, the test-jasmine CI test container has been failing roughly (my estimate) 1 out of two runs. Of those failure most are due to

retry npm run test-jasmine -- --tags=flaky --skip-tags=noCI

We should try to find a solution.

Potential solutions

  1. split off that
retry npm run test-jasmine -- --tags=flaky --skip-tags=noCI

into a new test container (named test-jasmine3)

  1. use shard_jasmine_tests.js util as in our @gl tests:

SHARDS=($(node $ROOT/tasks/shard_jasmine_tests.js --tag=gl))
for s in ${SHARDS[@]}; do
retry npm run test-jasmine -- "$s" --tags=gl --skip-tags=noCI
done

to split off the @flaky tests into multiple runs.

  1. combine 1) and 2)

@antoinerg @archmoj wanna give this a shot?

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions