TrainersName pattern (Discussion)

[Ivanidzo4ka]

#2172 follow up on this one.
So right now we have mix of trainers names.
I want to standardize them.

We have following zoo of naming patterns:
LogisticRegression
SdcaMultiClassTrainer
SgdNonCalibratedBinaryTrainer
LinearSvmTrainer
LightGbmMulticlassTrainer
MulticlassLogisticRegression

My proposal is following:

{AlgoName}(optional){Calibrated/NonCalibrated}{TypeOfTask}Trainer

Where AlgoName is full name without abbreviations (SDCA->StochasticDualCoordinateAscent, Linearsvm ->LinearSupportVectorMachines) with exception of LightGBM.

I would also prefer to explicitly specify TypeOfTask even if algorithm exist only for one type. (Which would create weird abominations like OneVersusAllMulticlassTrainer, but i'm fine with that)

Does that sound good for you?
@sfilipi @TomFinley @eerhardt @yaeldekel

[wschin]

Maybe {AlgoName}{**ModelName**}(optional){Calibrated/NonCalibrated}{TypeOfTask}Trainer?

[Ivanidzo4ka]

What ModelName means? Can you give example?

[wschin]

StochasticGradient-FieldAwareFactorizationMachine-Calibrated-BinaryClassification
AlternatingNewton-FieldAwareFactorizationMachine-Calibrated-BinaryClassification

[abgoswam]

This is also related to #2172 . The trainer estimator Class names we finally decide upon should also match the corresponding MLContext names

So if a user has to specify an SDCA binary trainer with advanced options, they would have to say something like :

// Pipeline.
var pipeline =
ml.BinaryClassification.Trainers.StochasticDualCoordinateAscentNonCalibratedBinaryTrainer(
new StochasticDualCoordinateAscentNonCalibratedBinaryTrainer.Options { NumThreads = 1 }
);

It may be overly verbose. Thoughts ?

[TomFinley]

ml.BinaryClassification.Trainers.StochasticDualCoordinateAscentNonCalibratedBinaryTrainer(
new StochasticDualCoordinateAscentNonCalibratedBinaryTrainer.Options { NumThreads = 1 }
);

It may be overly verbose. Thoughts ?

Right. It's not even StochasticDualCoordinateAscentNonCalibratedBinaryTrainer, if we also take the advice to incorporate the type of model it becomes, I think, StochasticDualCoordinateAscentLinearNonCalibratedBinaryTrainer. Likewise with AlternatingNewtonFieldAwareFactorizationMachineCalibratedBinaryClassificationTrainer. I think being principled is fine, but this is clearly nuts. I also don't see any harm in OneVersusAllTrainer.

I think naming things calibrated or not is fine if there is actually a distinction. But, I would pick one to identify as being calibrated or uncalibrated. If a class of models always produces calibrated or always non-calibrated models, no need to call attention to it. In any case it will be obvious from the return value and the generic parameters.

LogisticRegressionBinaryClassificationTrainer might be fine. That's not too long.

I know we're asking for hard and fast rules, but I think this is one of those cases where you just have to accept that if having a uniform procedure for naming would lead to a ridiculous situations like identifier names greater than 80 characters, we need to accept that a slightly irregular situation is better than being perfectly regular. If it's any consolation, sklearn gets by just fine without having these elaborate, byzantine names.

[eerhardt]

(thoughts from a non-data scientist):

1. Some of these names sound like they should be options/parameters instead of whole new methods/classes. Especially Calibrated vs. Non-calibrated.
2. For the method names, there is no point in having the TypeOfTask nor the Trainer suffix. We are already in the context of "task" and "trainers" by the time we've reached the method: mlContext.BinaryClassification.Trainers.StochasticDualCoordinateAscent(

[abgoswam]

Here's a proposal

• For MLContext name, we keep only the {AlgoName}. As @eerhardt noted above, by the time we reach the method we already have all the necessary info, and do not have to repeat the same again.

• For the Class name, we use the patter {AlgoName}(optional){Calibrated/NonCalibrated}{TypeOfTask}Trainer. The class name is slightly more verbose.

Note : that users would would have to use the class name only when they want to use the trainer estimators with advanced Options.

• It does mean that the exact match of MLContext name and Class name desired in The trainer name types should follow the names used in the contexts #2172 will not be there. There will be a prefix match of the {AlgoName}

(non-standard usage currently)	Proposed MLContext Name	Proposed Class Name
LightGbmRegressorTrainer	LightGbm	LightGbmRegressionTrainer
LightGbmBinaryTrainer	LightGbm	LightGbmBinaryClassificationTrainer
LightGbmRankingTrainer	LightGbm	LightGbmRankingTrainer
LightGbmMulticlassTrainer	LightGbm	LightGbmMulticlassClassificationTrainer
FastTreeBinaryClassificationTrainer	FastTree	FastTreeBinaryClassificationTrainer
FastTreeRegressionTrainer	FastTree	FastTreeRegressionTrainer
FastTreeRankingTrainer	FastTree	FastTreeRankingTrainer
FastTreeTweedieTrainer	FastTreeTweedie	FastTreeTweedieRegressionTrainer
FastForestRegression	FastForest	FastForestRegressionTrainer
FastForestClassification	FastForest	FastForestBinaryClassificationTrainer
MatrixFactorizationTrainer	MatrixFactorization	MatrixFactorizationRecommendationTrainer
BinaryClassificationGamTrainer	GeneralizedAdditiveModel	GeneralizedAdditiveModelBinaryClassificationTrainer
RegressionGamTrainer	GeneralizedAdditiveModel	GeneralizedAdditiveModelRegressionTrainer
LogisticRegression	LogisticRegression	LogisticRegressionBinaryClassificationTrainer
MulticlassLogisticRegression	LogisticRegression	LogisticRegressionMulticlassClassificationTrainer
AveragedPerceptronTrainer	AveragedPerceptron	AveragedPerceptronBinaryClassificationTrainer
LinearSvmTrainer	LinearSvm	LinearSvmBinaryClassificationTrainer
OnlineGradientDescentTrainer	OnlineGradientDescent	OnlineGradientDescentRegressionTrainer
PoissonRegression	PoissonRegression	PoissonRegressionTrainer
SdcaBinaryTrainer	StochasticDualCoordinateAscent	StochasticDualCoordinateAscentBinaryClassificationTrainer
SdcaNonCalibratedBinaryTrainer	StochasticDualCoordinateAscentNonCalibrated	StochasticDualCoordinateAscentNonCalibratedBinaryClassificationTrainer
SdcaMultiClassTrainer	StochasticDualCoordinateAscent	StochasticDualCoordinateAscentMulticlassClassificationTrainer
SdcaRegressionTrainer	StochasticDualCoordinateAscent	StochasticDualCoordinateAscentRegressionTrainer
KMeansPlusPlusTrainer	KMeans	KMeansPlusPlusClusteringTrainer
OrdinaryLeastSquaresRegressionTrainer	OrdinaryLeastSquares	OrdinaryLeastSquaresRegressionTrainer

CurrentName	Proposed MLContext Name	Proposed Class Name
FieldAwareFactorizationMachineBinaryClassificationTrainer	FieldAwareFactorizationMachine	FieldAwareFactorizationMachineBinaryClassificationTrainer
SymbolicStochasticGradientDescentClassificationTrainer	SymbolicStochasticGradientDescent	SymbolicStochasticGradientDescentBinaryClassificationTrainer

Base Class	Proposed New Name
SdcaTrainerBase (base class)	StochasticDualCoordinateAscentTrainerBase
LbfgsTrainerBase (base class)	LbfgsTrainerBase
GamTrainerBase (base class)	GeneralizedAdditiveModelTrainerBase

EDITS Proposed:

• {AlgoName} for KMeans in MLContext should be KMeansPlusPlus.

[eerhardt]

That proposal looks good @abgoswam. Here are some questions/comments:

1. When do we decide when we should use acronyms and when we don't? We have Svm, Gbm and Lbfgs above. But some things that use acronyms elsewhere are spelled out - for example we have a SymSgdNative assembly, but are spelling out SymbolicStochasticGradientDescent above.

2. KMeans vs. KMeansPlusPlusClusteringTrainer - what is the AlgoName? KMeans or KMeansPlusPlus? We should probably be consistent here.

[abgoswam]

1. For acronyms, the table above uses acronyms when the acronym is itself seems to be more standard than using the full name

• LightGbm comes from the name of the library itself
• SVM also is a very standard acronym
• For SymSGD is less convincing . While SGD is standard acronym but Sym isnt. We can either rename it to SymbolicSGD or use the full name SymbolicStochasticGradientDescent . Do we have a preference ?

2. The AlgoName should be KMeansPlusPlus (thats the algo our internal repo points to) . Thanks for pointing is out. I have noted this in the "Edits Proposed" section of the table.

[eerhardt]

I don't have a huge preference. I was mostly just trying to get what the policy is. If we go by "This is the algo's standard name - ex. LightGBM", then I think that is a fine policy.