Skip to content

Commit cdd309e

Browse files
authored
Enable Conditional Numerical Reproducibility for tests (#4569)
* Enable Conditional Numerical Reproducibility for tests * Update baselines for failing tests * Skip BinaryClassifierSymSgdTest on Linux due to numerical differences
1 parent ce5b23a commit cdd309e

File tree

9 files changed

+1411
-1402
lines changed

9 files changed

+1411
-1402
lines changed

test/BaselineOutput/Common/SymSGD/SymSGD-CV-breast-cancer-out.txt

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ maml.exe CV tr=SymSGD{nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1
22
Not adding a normalizer.
33
Data fully loaded into memory.
44
Initial learning rate is tuned to 100.000000
5-
Bias: -468.3528, Weights: [4.515409,75.74901,22.2914,-10.50209,-28.58107,44.81024,23.8734,13.20304,2.448269]
5+
Bias: -467.9297, Weights: [5.415065,76.39395,22.35155,-11.98839,-28.26446,44.58415,22.72012,11.13254,2.851256]
66
Not training a calibrator because it is not needed.
77
Not adding a normalizer.
88
Data fully loaded into memory.
@@ -15,15 +15,15 @@ Confusion table
1515
||======================
1616
PREDICTED || positive | negative | Recall
1717
TRUTH ||======================
18-
positive || 132 | 2 | 0.9851
18+
positive || 130 | 4 | 0.9701
1919
negative || 8 | 212 | 0.9636
2020
||======================
21-
Precision || 0.9429 | 0.9907 |
22-
OVERALL 0/1 ACCURACY: 0.971751
21+
Precision || 0.9420 | 0.9815 |
22+
OVERALL 0/1 ACCURACY: 0.966102
2323
LOG LOSS/instance: Infinity
2424
Test-set entropy (prior Log-Loss/instance): 0.956998
2525
LOG-LOSS REDUCTION (RIG): -Infinity
26-
AUC: 0.991045
26+
AUC: 0.990706
2727
Warning: The predictor produced non-finite prediction values on 8 instances during testing. Possible causes: abnormal data or the predictor is numerically unstable.
2828
TEST POSITIVE RATIO: 0.3191 (105.0/(105.0+224.0))
2929
Confusion table
@@ -42,16 +42,16 @@ AUC: 0.963435
4242

4343
OVERALL RESULTS
4444
---------------------------------------
45-
AUC: 0.977240 (0.0138)
46-
Accuracy: 0.955481 (0.0163)
47-
Positive precision: 0.920027 (0.0228)
48-
Positive recall: 0.949680 (0.0354)
49-
Negative precision: 0.975057 (0.0156)
45+
AUC: 0.977070 (0.0136)
46+
Accuracy: 0.952656 (0.0134)
47+
Positive precision: 0.919613 (0.0224)
48+
Positive recall: 0.942217 (0.0279)
49+
Negative precision: 0.970470 (0.0110)
5050
Negative recall: 0.957265 (0.0064)
5151
Log-loss: Infinity (NaN)
5252
Log-loss reduction: -Infinity (NaN)
53-
F1 Score: 0.934582 (0.0289)
54-
AUPRC: 0.964431 (0.0168)
53+
F1 Score: 0.930771 (0.0251)
54+
AUPRC: 0.964018 (0.0164)
5555

5656
---------------------------------------
5757
Physical memory usage(MB): %Number%
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
SymSGD
22
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC /nt Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
3-
0.97724 0.955481 0.920027 0.94968 0.975057 0.957265 Infinity -Infinity 0.934582 0.964431 1 SymSGD %Data% %Output% 99 0 0 maml.exe CV tr=SymSGD{nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1 /nt:1
3+
0.97707 0.952656 0.919613 0.942217 0.97047 0.957265 Infinity -Infinity 0.930771 0.964018 1 SymSGD %Data% %Output% 99 0 0 maml.exe CV tr=SymSGD{nt=1} threads=- norm=No dout=%Output% data=%Data% seed=1 /nt:1
44

0 commit comments

Comments
 (0)