Skip to content

Enable Conditional Numerical Reproducibility for tests #4569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 4, 2020
Merged

Enable Conditional Numerical Reproducibility for tests #4569

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cfc9abd
Enable Conditional Numerical Reproducibility for tests
sharwell Dec 12, 2019
d0fe25f
Update baselines for failing tests
sharwell Dec 12, 2019
21bd1ff
Skip BinaryClassifierSymSgdTest on Linux due to numerical differences
sharwell Jan 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions test/BaselineOutput/Common/SymSGD/SymSGD-CV-breast-cancer-out.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ maml.exe CV tr=SymSGD{nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1
Not adding a normalizer.
Data fully loaded into memory.
Initial learning rate is tuned to 100.000000
Bias: -468.3528, Weights: [4.515409,75.74901,22.2914,-10.50209,-28.58107,44.81024,23.8734,13.20304,2.448269]
Bias: -467.9297, Weights: [5.415065,76.39395,22.35155,-11.98839,-28.26446,44.58415,22.72012,11.13254,2.851256]
Not training a calibrator because it is not needed.
Not adding a normalizer.
Data fully loaded into memory.
Expand All @@ -15,15 +15,15 @@ Confusion table
||======================
PREDICTED || positive | negative | Recall
TRUTH ||======================
positive || 132 | 2 | 0.9851
positive || 130 | 4 | 0.9701
negative || 8 | 212 | 0.9636
||======================
Precision || 0.9429 | 0.9907 |
OVERALL 0/1 ACCURACY: 0.971751
Precision || 0.9420 | 0.9815 |
OVERALL 0/1 ACCURACY: 0.966102
LOG LOSS/instance: Infinity
Test-set entropy (prior Log-Loss/instance): 0.956998
LOG-LOSS REDUCTION (RIG): -Infinity
AUC: 0.991045
AUC: 0.990706
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
TEST POSITIVE RATIO: 0.3191 (105.0/(105.0+224.0))
Confusion table
Expand All @@ -42,16 +42,16 @@ AUC: 0.963435

OVERALL RESULTS
---------------------------------------
AUC: 0.977240 (0.0138)
Accuracy: 0.955481 (0.0163)
Positive precision: 0.920027 (0.0228)
Positive recall: 0.949680 (0.0354)
Negative precision: 0.975057 (0.0156)
AUC: 0.977070 (0.0136)
Accuracy: 0.952656 (0.0134)
Positive precision: 0.919613 (0.0224)
Positive recall: 0.942217 (0.0279)
Negative precision: 0.970470 (0.0110)
Negative recall: 0.957265 (0.0064)
Log-loss: Infinity (NaN)
Log-loss reduction: -Infinity (NaN)
F1 Score: 0.934582 (0.0289)
AUPRC: 0.964431 (0.0168)
F1 Score: 0.930771 (0.0251)
AUPRC: 0.964018 (0.0164)

---------------------------------------
Physical memory usage(MB): %Number%
Expand Down
2 changes: 1 addition & 1 deletion test/BaselineOutput/Common/SymSGD/SymSGD-CV-breast-cancer-rp.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
SymSGD
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC /nt Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.97724 0.955481 0.920027 0.94968 0.975057 0.957265 Infinity -Infinity 0.934582 0.964431 1 SymSGD %Data% %Output% 99 0 0 maml.exe CV tr=SymSGD{nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1 /nt:1
0.97707 0.952656 0.919613 0.942217 0.97047 0.957265 Infinity -Infinity 0.930771 0.964018 1 SymSGD %Data% %Output% 99 0 0 maml.exe CV tr=SymSGD{nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1 /nt:1

Loading