Skip to content

Include confidence intervals for non-probabilistic scores #1906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rogancarr opened this issue Dec 18, 2018 · 2 comments
Closed

Include confidence intervals for non-probabilistic scores #1906

rogancarr opened this issue Dec 18, 2018 · 2 comments
Labels
enhancement New feature or request

Comments

@rogancarr
Copy link
Contributor

rogancarr commented Dec 18, 2018

For probabilistic classifiers, we get a sense for the confidence of a model's output from the probability. For other types of scorers (e.g. regression), we just get a raw score out. Oftentimes, consumers of ML models want to know how confident a model is in its prediction. Thus, it would be nice to output confidence intervals for predictions.

There are methods to generate confidence intervals for ML models, but those usually require a lot of hand tuning and a background in statistics. It would be nice to automate such techniques so that anybody could use them.

Related to #511

@rogancarr
Copy link
Contributor Author

(And of course, we may want confidence intervals on the predicted probabilities too....)

@codemzs
Copy link
Member

codemzs commented Jun 30, 2019

not on the roadmap for near future.

@codemzs codemzs closed this as completed Jun 30, 2019
@ghost ghost locked as resolved and limited conversation to collaborators Mar 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants