Skip to content

Add metadata of bias and fairness metrics to models #1912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rogancarr opened this issue Dec 18, 2018 · 2 comments · Fixed by #6539
Closed

Add metadata of bias and fairness metrics to models #1912

rogancarr opened this issue Dec 18, 2018 · 2 comments · Fixed by #6539
Labels
enhancement New feature or request

Comments

@rogancarr
Copy link
Contributor

Currently, we can use ML.NET to evaluate the performance of a model. However, when the model is passed to a third party, any metrics must be passed separately.

If you consider the bias and fairness metrics** to be properties of the model, then it makes sense to include them in the model. I expect this to be helpful in deployment, productionization, debugging, etc. Plus it would be nice to have properties of the model visible in an IDE and accessible programatically.

Related to #511
Related to #1911
Related to #1908

** e.g. over a dataset representative of the expected distribution of data to be seen by the model

@tauheedul
Copy link
Contributor

Thank you, this is an important feature.
You may be interested to learn Microsoft published a paper in ICML 2018 covering Machine Learning for Fair Decisions

Please check out: A Reductions Approach to Fair Classification

arXiv: https://arxiv.org/abs/1803.02453
Source: https://github.com/Microsoft/fairlearn

@codemzs
Copy link
Member

codemzs commented Jun 30, 2019

thanks but not on the roadmap for the near future.

@codemzs codemzs closed this as completed Jun 30, 2019
@ghost ghost locked as resolved and limited conversation to collaborators Mar 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants