Skip to content

Simplify post-prediction schema info to make user-friendly prediction's info such as list of ranked predicted labels/categories #2278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CESARDELATORRE opened this issue Jan 28, 2019 · 4 comments
Labels
enhancement New feature or request P2 Priority of the issue for triage purpose: Needs to be fixed at some point.

Comments

@CESARDELATORRE
Copy link
Contributor

CESARDELATORRE commented Jan 28, 2019

CONTEXT:

In this PR (and probably, available in 0.10), we're allowing the user to get some schema info after predicting:

#2250

That way a user can get, for example, get the list of the predicted labels in a multi-class classification and related that to the list/array of scores.

That's very useful for business cases where you want to automatically predict/assign a product to multiple categories, for example, instead of predicting a single category/label.

However, the current user's code needs to be like the following, using not user-friendly types such as VBuffer, etc.:

            // Slot names on top of Score column represent original labels for i-th value in Score array.
            VBuffer<ReadOnlyMemory<char>> slotNames = default;
            engine.OutputSchema[nameof(IrisPrediction.Score)].GetSlotNames(ref slotNames);
            // Key names represent original values for PredictedLabel column.
            VBuffer<ReadOnlyMemory<char>> keys = default;
            engine.OutputSchema[nameof(IrisPrediction.PredictedLabel)].GetKeyValues(ref keys);

            Assert.True(slotNames.GetItemOrDefault(0).ToString() == "Iris-setosa");
            Assert.True(slotNames.GetItemOrDefault(1).ToString() == "Iris-versicolor");
            Assert.True(slotNames.GetItemOrDefault(2).ToString() == "Iris-virginica");

REQUEST:

The ask is to simplify it and provide types and simpler API with user-friendly data-types that are familiar for regular .NET developers.

Also, a more straightforward way to relate the list of labels with their related scores. Maybe returning a single array/list with both concepts coming along (labels-scores) instead of separated arrays/lists?


Related info

This topic is related with this original issue, too:
Multi-class classification returning ranked list of possible labels): #2233

@CESARDELATORRE CESARDELATORRE changed the title Simplify post-prediction schema info to make user-friendly predictions such as list of ranked predicted labels/categories Simplify post-prediction schema info to make user-friendly prediction's info such as list of ranked predicted labels/categories Jan 28, 2019
@CESARDELATORRE CESARDELATORRE added enhancement New feature or request and removed enhancement New feature or request labels Jan 28, 2019
@prathyusha12345
Copy link

@Ivanidzo4ka Is anyone working on simplifying this code for 0.11? Or not yet?
Thanks,

@Ivanidzo4ka
Copy link
Contributor

@prathyusha12345
I know we have proposal regarding Metadata itself: #2622 but I didn't have time to look on it yet.

@Ivanidzo4ka Ivanidzo4ka removed their assignment Jun 3, 2019
@aljones007
Copy link

Is anyone looking into this yet? Just wondering if there is a simple way of getting the multi-labels out. Thanks.

@gvashishtha gvashishtha added the enhancement New feature or request label Nov 7, 2019
@gvashishtha gvashishtha self-assigned this Nov 7, 2019
@gvashishtha
Copy link
Contributor

@aljones007 it is in our backlog, and we hope to make some progress on this soon.

@gvashishtha gvashishtha added the P2 Priority of the issue for triage purpose: Needs to be fixed at some point. label Jan 9, 2020
@gvashishtha gvashishtha removed their assignment Jan 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request P2 Priority of the issue for triage purpose: Needs to be fixed at some point.
Projects
None yet
Development

No branches or pull requests

5 participants