Skip to content

CI Conda: Add to known-test-failures #36937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 2, 2024
Merged

CI Conda: Add to known-test-failures #36937

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3bfed13
.github/workflows/ci-conda-known-test-failures.json: Add random failu…
mkoeppe Dec 20, 2023
d803c6c
.github/workflows/ci-conda-known-test-failures.json: Add https://gith…
mkoeppe Dec 21, 2023
255fc6a
.github/workflows/ci-conda-known-test-failures.json: Add https://gith…
mkoeppe Dec 21, 2023
410bc13
.github/workflows/ci-conda-known-test-failures.json: Add some failure…
mkoeppe Dec 21, 2023
4d3d50e
.github/workflows/ci-conda-known-test-failures.json: Add references f…
mkoeppe Dec 21, 2023
9db526c
.github/workflows/ci-conda-known-test-failures.json: Add details from…
mkoeppe Dec 21, 2023
5e69712
src/sage/doctest/reporting.py: Include message in '[failed in baselin…
mkoeppe Dec 21, 2023
fa6ec3d
.github/workflows/ci-conda-known-test-failures.json: Add https://gith…
mkoeppe Dec 23, 2023
8f4bc66
.github/workflows/ci-conda-known-test-failures.json: Add https://gith…
mkoeppe Dec 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 39 additions & 9 deletions .github/workflows/ci-conda-known-test-failures.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
{
"doc.en.constructions.calculus": {
"failed": "unreported random failures in plotting"
},
"sage_setup.clean": {
"failed": true
"failed": "_find_stale_files finds some stale files under sage/tests when executed under conda"
},
"sage.algebras.fusion_rings.fusion_ring": {
"failed": "unreported random timeouts"
},
"sage.combinat.cluster_algebra_quiver.quiver": {
"failed": true
"failed": "failure seen in https://github.com/sagemath/sage/actions/runs/6836592771/job/18591690058#step:11:10059"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These comments are pretty useless as the logs are deleted quite quickly (90 days?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, 90 days. Enough time for people to investigate and open Issues / PRs.

Not helpful to demand that it's done here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You missed step 2 of your own procedure (#36678 (comment)):

Test failure is noticed
You record it in a GitHub Issue. There you include the necessary information so that it does not take 2 hours to reconstruct it.
One adds stuff to the exclusion list to eliminate noise in the tests. It's not for humans to read, and definitely not for humans to use 2 hours of guessing time to reconstruct the failure.
The exclusion list is maintained like everything else in Sage -- by PRs, not by specific people in charge.

},
"sage.geometry.cone": {
"failed": true
"failed": "unreported random timeouts seen in https://github.com/sagemath/sage/actions/runs/6827937663/job/18571052628#step:11:12362"
},
"sage.groups.matrix_gps.finitely_generated_gap": {
"failed": true
Expand All @@ -15,38 +21,62 @@
"failed": true
},
"sage.libs.gap.element": {
"failed": true
"failed": "failure seen in https://github.com/sagemath/sage/actions/runs/6840579851/job/18600012965#step:11:13016"
},
"sage.libs.singular.singular": {
"failed": true
},
"sage.matrix.matrix2": {
"failed": true
"failed": "failure seen in https://github.com/sagemath/sage/actions/runs/6835143781/job/18588599649#step:11:16939"
},
"sage.matrix.matrix_integer_sparse": {
"failed": true
"failed": "failure seen in https://github.com/sagemath/sage/actions/runs/6827937663/job/18571052628#step:11:12362"
},
"sage.misc.lazy_import": {
"failed": true
},
"sage.misc.weak_dict": {
"failed": true
"failed": "failure seen in https://github.com/sagemath/sage/actions/runs/6870325919/job/18684964234#step:11:10059"
},
"sage.modular.modform.l_series_gross_zagier": {
"failed": true
"failed": "unreported failure seen in https://github.com/sagemath/sage/actions/runs/6859244281/job/18651257775#step:11:10102"
},
"sage.numerical.linear_tensor_element": {
"failed": "random segfaults https://github.com/sagemath/sage/issues/28559"
},
"sage.parallel.map_reduce": {
"failed": "random failure https://github.com/sagemath/sage/issues/36939"
},
"sage.plot.plot": {
"failed": "unreported random failure (macOS)"
},
"sage.rings.function_field.drinfeld_modules.morphism": {
"failed": true
"failed": "unreported random failure seen in https://github.com/sagemath/sage/actions/runs/6840502530/job/18599835766#step:11:10107"
},
"sage.rings.polynomial.multi_polynomial_ideal": {
"failed": true
},
"sage.rings.polynomial.multi_polynomial_libsingular": {
"failed": true
},
"sage.rings.polynomial.polynomial_element": {
"failed": "unreported random failure in symbolic 'roots' (macOS)"
},
"sage.rings.polynomial.skew_polynomial_finite_field": {
"failed": true
},
"sage.schemes.elliptic_curves.descent_two_isogeny": {
"failed": "random segfault (macOS) https://github.com/sagemath/sage/issues/36949"
},
"sage.schemes.elliptic_curves.ell_field": {
"failed": "random failure https://github.com/sagemath/sage/issues/36832"
},
"sage.sets.recursively_enumerated_set": {
"failed": "random failures related to AlarmInterrupt (macOS)"
},
"sage.structure.coerce_actions": {
"failed": "random failure https://github.com/sagemath/sage/issues/35973"
},
"sage.tests.gap_packages": {
"failed": true
}
Expand Down
7 changes: 5 additions & 2 deletions src/sage/doctest/reporting.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,10 +213,13 @@ def report_head(self, source, fail_msg=None):
baseline = self.controller.source_baseline(source)
if fail_msg:
cmd += " # " + fail_msg
if baseline.get('failed', False):
if failed := baseline.get('failed', False):
if not fail_msg:
cmd += " #"
cmd += " [failed in baseline]"
if failed is True:
cmd += " [failed in baseline]"
else:
cmd += f" [failed in baseline: {failed}]"
return cmd

def report(self, source, timeout, return_code, results, output, pid=None):
Expand Down