Bring back support for ConfusionMatrix in the new API

[abgoswam]

While updating tests from the old AP to the new API, we found that the new API does not support ConfusionMatrix in the evaluation output.

machinelearning/src/Microsoft.ML.Data/Evaluators/MultiClassClassifierEvaluator.cs

Lines 506 to 520 in 21d64aa

	public MultiClassClassifierMetrics Evaluate(IDataView data, string label, string score, string predictedLabel)
	{
	Host.CheckValue(data, nameof(data));
	Host.CheckNonEmpty(label, nameof(label));
	Host.CheckNonEmpty(score, nameof(score));
	Host.CheckNonEmpty(predictedLabel, nameof(predictedLabel));
	var roles = new RoleMappedData(data, opt: false,
	RoleMappedSchema.ColumnRole.Label.Bind(label),
	RoleMappedSchema.CreatePair(MetadataUtils.Const.ScoreValueKind.Score, score),
	RoleMappedSchema.CreatePair(MetadataUtils.Const.ScoreValueKind.PredictedLabel, predictedLabel));
	var resultDict = ((IEvaluator)this).Evaluate(roles);
	Host.Assert(resultDict.ContainsKey(MetricKinds.OverallMetrics));
	var overall = resultDict[MetricKinds.OverallMetrics];

In the Legacy API, there was support for ConfusionMatrix

machinelearning/src/Microsoft.ML.Data/Evaluators/MultiClassClassifierEvaluator.cs

Lines 1031 to 1049 in 21d64aa

	public static partial class Evaluate
	{
	[TlcModule.EntryPoint(Name = "Models.ClassificationEvaluator", Desc = "Evaluates a multi class classification scored dataset.")]
	public static CommonOutputs.ClassificationEvaluateOutput MultiClass(IHostEnvironment env, MultiClassMamlEvaluator.Arguments input)
	{
	Contracts.CheckValue(env, nameof(env));
	var host = env.Register("EvaluateMultiClass");
	host.CheckValue(input, nameof(input));
	EntryPointUtils.CheckInputArgs(host, input);
	MatchColumns(host, input, out string label, out string weight, out string name);
	IMamlEvaluator evaluator = new MultiClassMamlEvaluator(host, input);
	var data = new RoleMappedData(input.Data, label, null, null, weight, name);
	var metrics = evaluator.Evaluate(data);
	var warnings = ExtractWarnings(host, metrics);
	var overallMetrics = ExtractOverallMetrics(host, metrics, evaluator);
	var perInstanceMetrics = evaluator.GetPerInstanceMetrics(data);
	var confusionMatrix = ExtractConfusionMatrix(host, metrics);

We should add back the checks for ConfusionMatrix in the following tests :

• TrainAndPredictIrisModelTest
• TrainAndPredictIrisModelWithStringLabelTest

@codemzs @Ivanidzo4ka

[bartczernicki]

+1 this is really silly that we don't have a Confusion Matrix output. I realize this can be calculated on your own, but this one of the fundamental performance constructs that get reported in most ML software/libraries.

[Ivanidzo4ka]

We have one month before we lock our code in preparation for 1.0 release. It's a nice thing to have, but we just don't have time to focus on this right now.
Unless you think it's a game changer and necessary to have.

[bartczernicki]

@Ivanidzo4ka I completely understand there is 1.0 pressure. I think having core performance conventions and explain-ability aligned with basic ML software should be prioritized for post 1.0.