-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Add a note that NDCG returns values between 0 and 100 #2637
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Would it be possible to move the range to 0 to 1.0? This would be inline with most of our other metrics, and the industry standard. |
It's definitely possible, but it's a breaking change for a lot of folks out there. @TomFinley what's your thought? |
It would be more consistent to do as @justinormont suggests -- we phrase no other metric as a percentage, as far as I am aware. At the same time I know that the people that use this metrics are more accustomed to having it expressed as a percentage. (Which I assume is what you mean by a breaking change.) But of course, the custom of some people can be adjusted and changed, whereas if we have an inconsistency now we are probably going to commit to it forever. This strikes me as unattractive. This leads me to prefer the 0 to 1 range. It seems logically defensible and consistent, though I understand it is more work than the documentation change, and might annoy some people in the short term. If you decide you agree @rogancarr please change this to be under API and project 13 appropriately, since it will then represent a breaking change. |
@TomFinley @justinormont Great! Let's move it from 0 to 1 and add a note to the API docs. I'll self-assign and send this in shortly. Maybe we could write up a doc for internal users with a summary of small changes. |
I found two other measures being scaled to percentages: The relative information gains for Binary Classification |
Just a quick note. I implemented the change, and about 41 tests fail. We'll have to remake baselines for a bunch of tests, so it'll be a rather big PR. |
NDCG, the normalized discounted cumulative gain metric that we often use in ranking, ranges in value between 0 and 100 in ML.NET. In the wide world, NDCG is often considered to be between 0 and 1 (e.g. see Wikipedia). We should add a note to the ranking metrics object that the values range between 0 and 100 so that we don't confuse people who are new to the toolkit.
The text was updated successfully, but these errors were encountered: