Multiclass LightGBM bug

[yaeldekel]

LightGBM trainer has two non-readonly fields called _numClass and _tlcNumClass. The second one is used to determine the number of predictors in the OVA predictor. However, the value of _tlcNumClass is only updated once, so if Fit is called again on the same estimator, it might give the wrong number of classes.

[antoniovs1029]

I've just taken a quick look to this. To do it, I modified the Multiclass LightGbm sample, and added code so to Fit again the pipeline with a dataview that has labels 1-4 (whereas the original dataview used to fit the pipeline the first time, had labels 1-3). As a result, when fitting again the pipeline _tlcNumClass was still set to be "3" (instead of 4), and when printing the metrics, only 3 labels were taken into account, ignoring the last one.

I believe this is the issue @yaeldekel is describing, right?

If this is the issue, wouldn't this be solved by simply moving the initialization of _tlcNumClass out of this if statement, so that it would always be initialized when fitting the pipeline?

[justinormont]

What does calling Fit() twice do?

If it's not LightGBM's task=refit, can we expose it?

This may serve the needs of AutoML, which is looking for streamable trees. This would let us fit the tree structure on ~10 to 100GB of data, then stream the whole dataset (TBs) to refit the leaf node values. There's a similar option by using TreeFeat + linear model.
/cc @daholste

[yaeldekel]

The estimators are intended to be stateless, so calling Fit() twice should produce exactly the same result as defining two estimators and calling Fit() once on each of them (except, perhaps, for any randomness used during training).
Regarding LightGBM refit, is it capable of doing something that cannot be done using TreeFeat + linear model? If the answer is yes, could you open a new issue for it?