Scrubbing SDCA learners #2825
Scrubbing SDCA learners #2825
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2825 +/- ##
==========================================
+ Coverage 71.69% 71.7% +<.01%
==========================================
Files 809 810 +1
Lines 142444 142473 +29
Branches 16109 16111 +2
==========================================
+ Hits 102130 102156 +26
- Misses 35889 35891 +2
- Partials 4425 4426 +1
|
|
|
||
| /// <summary> | ||
| /// Whether to shuffle data at every epoch (default true). |
There was a problem hiding this comment.
epoch [](start = 49, length = 5)
iteration? or it's two different things? #Resolved
There was a problem hiding this comment.
retaining the latest documentation updates being done in parallel
In reply to: 261895645 [](ancestors = 261895645)
| /// <summary> | ||
| /// Convergence check frequency (in terms of <see cref="NumberOfIterations"/>). Set as negative or zero for not checking at all. | ||
| /// If left blank, it defaults to check after every <see cref="NumberOfThreads"/> iterations. | ||
| /// </summary> | ||
| [Argument(ArgumentType.AtMostOnce, HelpText = "Convergence check frequency (in terms of number of iterations). Set as negative or zero for not checking at all. If left blank, it defaults to check after every 'numThreads' iterations.", NullName = "<Auto>", ShortName = "checkFreq")] | ||
| public int? CheckFrequency; |
There was a problem hiding this comment.
CheckFrequency [](start = 24, length = 14)
Maybe ConvergenceCheckFrequency?
Also can you copy this documentation across other CheckFrequency options?
| [TGUI(Label = "Initial Learning Rate (for SGD)")] | ||
| public double InitLearningRate = Defaults.InitLearningRate; | ||
| public double InitialLearningRate = Defaults.InitialLearningRate; |
There was a problem hiding this comment.
InitialLearningRate [](start = 26, length = 19)
So it works this way: in the beginning of each iteration we set learning rate to this one, and then with going through examples we decrease it. But in each iteration it starts with this one.
So technically it's initial learning rate, but it remain same for each iteration....
Can you look other example of InitialLearningRate, and are they have same pattern or not?
I would probably just call it LearningRate. #Resolved
There was a problem hiding this comment.
I do not see other examples of InitialLearningRate in the other learners.
To me the names makes sense, esp. since it does indicate that this learning rate is adaptive in nature. We can add more documentation as to why this helps . @shmoradims
In reply to: 261897003 [](ancestors = 261897003)
| @@ -26,7 +26,7 @@ namespace Microsoft.ML.Trainers | |||
| { | |||
| // SDCA linear multiclass trainer. | |||
| /// <include file='doc.xml' path='doc/members/member[@name="SDCA"]/*' /> | |||
| public class SdcaMultiClassTrainer : SdcaTrainerBase<SdcaMultiClassTrainer.Options, MulticlassPredictionTransformer<MulticlassLogisticRegressionModelParameters>, MulticlassLogisticRegressionModelParameters> | |||
| public sealed class SdcaMultiClassTrainer : SdcaTrainerBase<SdcaMultiClassTrainer.Options, MulticlassPredictionTransformer<MulticlassLogisticRegressionModelParameters>, MulticlassLogisticRegressionModelParameters> | |||
There was a problem hiding this comment.
sealed [](start = 11, length = 6)
How I can put thumbs up emoji?
Anyway thanks for catching this! #Resolved
| int maxIterations = SgdBinaryTrainer.Options.Defaults.MaxIterations, | ||
| double initLearningRate = SgdBinaryTrainer.Options.Defaults.InitLearningRate, | ||
| float l2Weight = SgdBinaryTrainer.Options.Defaults.L2Weight) | ||
| int maxIterations = SgdBinaryTrainer.Options.Defaults.NumberOfIterations, |
There was a problem hiding this comment.
maxIterations [](start = 16, length = 13)
Should you reflect the change in the extensions as well?
|
@abgoswam Any chance you plan to update and merge this one? |
ghost
locked as resolved and limited conversation to collaborators
Fixes #2616 . Related #2613