Skip to content

F1-score to return 0.0 instead of NaN #4674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion src/Microsoft.ML.Data/Evaluators/BinaryClassifierEvaluator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,16 @@ public Double LogLossReduction
}
}

public Double F1 { get { return 2 * PrecisionPos * RecallPos / (PrecisionPos + RecallPos); } }
public Double F1
{
get
{
var precisionPlusRecall = PrecisionPos + RecallPos;
if (precisionPlusRecall == 0)
return 0;
return 2 * PrecisionPos * RecallPos / precisionPlusRecall;
}
}

public Counters(bool useRaw, Single threshold)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Negative precision: 0.655222 (0.0000)
Negative recall: 1.000000 (0.0000)
Log-loss: 1.000000 (0.0000)
Log-loss reduction: -0.076058 (0.0000)
F1 Score: NaN (0.0000)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.415719 (0.0000)

---------------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FastTreeBinaryClassification
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC /mil /iter Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.5 0.655222 0 0 0.655222 1 1 -0.076058 NaN 0.415719 10000 5 FastTreeBinaryClassification %Data% %Data% %Output% 99 0 0 maml.exe TrainTest test=%Data% tr=FastTreeBinaryClassification{mil=10000 iter=5} cache=- dout=%Output% loader=Text{sparse- col=Attr:TX:6 col=Label:0 col=Features:1-5,6,7-9} data=%Data% out=%Output% seed=1 /mil:10000;/iter:5
0.5 0.655222 0 0 0.655222 1 1 -0.076058 0 0.415719 10000 5 FastTreeBinaryClassification %Data% %Data% %Output% 99 0 0 maml.exe TrainTest test=%Data% tr=FastTreeBinaryClassification{mil=10000 iter=5} cache=- dout=%Output% loader=Text{sparse- col=Attr:TX:6 col=Label:0 col=Features:1-5,6,7-9} data=%Data% out=%Output% seed=1 /mil:10000;/iter:5

Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Negative precision: 0.656163 (0.0263)
Negative recall: 1.000000 (0.0000)
Log-loss: 0.935104 (0.0247)
Log-loss reduction: -0.009590 (0.0001)
F1 Score: NaN (NaN)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.418968 (0.0212)

---------------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
PriorPredictor
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.5 0.656163 0 0 0.656163 1 0.935104 -0.00959 NaN 0.418968 PriorPredictor %Data% %Output% 99 0 0 maml.exe CV tr=PriorPredictor threads=- dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% seed=1
0.5 0.656163 0 0 0.656163 1 0.935104 -0.00959 0 0.418968 PriorPredictor %Data% %Output% 99 0 0 maml.exe CV tr=PriorPredictor threads=- dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% seed=1

Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Negative precision: 0.655222 (0.0000)
Negative recall: 1.000000 (0.0000)
Log-loss: 0.929318 (0.0000)
Log-loss reduction: 0.000000 (0.0000)
F1 Score: NaN (0.0000)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.415719 (0.0000)

---------------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
PriorPredictor
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.5 0.655222 0 0 0.655222 1 0.929318 0 NaN 0.415719 PriorPredictor %Data% %Data% %Output% 99 0 0 maml.exe TrainTest test=%Data% tr=PriorPredictor dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% out=%Output% seed=1
0.5 0.655222 0 0 0.655222 1 0.929318 0 0 0.415719 PriorPredictor %Data% %Data% %Output% 99 0 0 maml.exe TrainTest test=%Data% tr=PriorPredictor dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% out=%Output% seed=1

Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Negative precision: 0.656163 (0.0263)
Negative recall: 1.000000 (0.0000)
Log-loss: 0.935104 (0.0247)
Log-loss reduction: -0.009590 (0.0001)
F1 Score: NaN (NaN)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.418968 (0.0212)

---------------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
PriorPredictor
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.5 0.656163 0 0 0.656163 1 0.935104 -0.00959 NaN 0.418968 PriorPredictor %Data% %Output% 99 0 0 maml.exe CV tr=PriorPredictor threads=- dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% seed=1
0.5 0.656163 0 0 0.656163 1 0.935104 -0.00959 0 0.418968 PriorPredictor %Data% %Output% 99 0 0 maml.exe CV tr=PriorPredictor threads=- dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% seed=1

Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Negative precision: 0.655222 (0.0000)
Negative recall: 1.000000 (0.0000)
Log-loss: 0.929318 (0.0000)
Log-loss reduction: 0.000000 (0.0000)
F1 Score: NaN (0.0000)
F1 Score: 0.000000 (0.0000)
AUPRC: 0.415719 (0.0000)

---------------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
PriorPredictor
AUC Accuracy Positive precision Positive recall Negative precision Negative recall Log-loss Log-loss reduction F1 Score AUPRC Learner Name Train Dataset Test Dataset Results File Run Time Physical Memory Virtual Memory Command Line Settings
0.5 0.655222 0 0 0.655222 1 0.929318 0 NaN 0.415719 PriorPredictor %Data% %Data% %Output% 99 0 0 maml.exe TrainTest test=%Data% tr=PriorPredictor dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% out=%Output% seed=1
0.5 0.655222 0 0 0.655222 1 0.929318 0 0 0.415719 PriorPredictor %Data% %Data% %Output% 99 0 0 maml.exe TrainTest test=%Data% tr=PriorPredictor dout=%Output% loader=Text{col=Label:BL:0 col=Features:~} data=%Data% out=%Output% seed=1