Feature Importance with ML.NET

[WladdGorshenin]

Dear ML.NET team and community members,

I'm so excited about ML.NET. It helps me easily integrate ML capabilities in a C# projects.
But as evolving project it lacks documentation and code examples. Therefore I'd like to ask the following question.

My current project requires not only prediction but reasoning behind it as well. I tried my approach with decision trees in Python/Sklearn and have proved my PoC. Now I'm going to implement the same approach with ML.NET and I'd like to know:

• what is the best way to derive feature importance out of a trained tree/forest?
• what is the best way to implement a method similar to DecisionTreeClassifier.decision_path with ML.NET?

[Zruty0]

@WladdGorshenin ,

1. After we train a tree ensemble model, the trainer produces the 'model summary', which includes aggregated per-feature gains. These proved a useful proxy to 'feature importance'.

Currently it's a bit of a chore to extract the summary post-training, but it's definitely possible.
You can take a look at a complete example, where we inspect the topology of the tree among other things:
https://github.com/dotnet/machinelearning/pull/653/files#diff-d36b6bf4d2fcf5366387069ff79b95a5

treePredictor.GetSummaryInKeyValuePairs() is a method that you can call to extract the per-feature aggregated gains.

In addition to this artifact of training, we are also planning to enable some more 'explainability' features: namely:

• permutation feature importance: it's a model-agnostic analysis tool that tries to assess which features the model is more sensitive to
• per-example feature gains: for any given example and a model (tree ensemble or linear), we can give a signed 'feature impact' of each feature to the score of that example. Note that this analysis is per-example, whereas the above is for the dataset as a whole.

These features await their porting to ML.NET, and @GalOshri would like to know how much value would you put in them.

[WladdGorshenin]

Hi @Zruty0 , thank you for answering the first point. I'm working on it. Could you please give me any hint on the second point (what is the best way to implement a method similar to DecisionTreeClassifier.decision_path with ML.NET?)

[klausmh]

+1 for adding permutation feature importance and per-example feature gains to ML.NET. That is something we would need.

[Zruty0]

@GalOshri , could you please consolidate all the requests for feature importance in one issue, and close the others?

[GalOshri]

@Zruty0 I looked through some of the explainability issues and will close some of the duplicates but others are worth keeping open for more open-ended discussion.

This issue refers to specific components that need to be moved to ML.NET (permutation feature importance and per-example feature gains). Can we try to schedule this for 0.8?

[shauheen]

closing this as we shipped this functionality in 0.8, feel free to reopen if still not completely addressed.