Skip to content

Is it possible to integrate a metric of imblearn as a scorer? #786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
moelliDo opened this issue Feb 22, 2020 · 1 comment
Open

Is it possible to integrate a metric of imblearn as a scorer? #786

moelliDo opened this issue Feb 22, 2020 · 1 comment
Labels

Comments

@moelliDo
Copy link

I would like to use the geometric mean as a metric to optimize classification models trained with the fit() method.
I thought that the autosklearn.metrics.make_scorer() would allow to define the geometric mean as a scorer like:
classifier.fit(X_train, y_train, feat_type=feat_type, metric=autosklearn.metrics.make_scorer("gm", imblearn.metrics.geometric_mean_score))
as the imblearn-package is fully compatible with sklearn.

However, after the model has been fitted, the sprint statistics indicate, that the definition as I did it does not seem to be working:

auto-sklearn results:
Dataset name: 6b31930a65e59cca700a5844fbab91a0
Metric: gm
Best validation score: 0.000000
Number of target algorithm runs: 187
Number of successful target algorithm runs: 82
Number of crashed target algorithm runs: 74
Number of target algorithms that exceeded the time limit: 18
Number of target algorithms that exceeded the memory limit: 13

Is it somehow possible to define the geometric mean as a metric to optimize the model?

@github-actions
Copy link
Contributor

github-actions bot commented May 5, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs for the next 7 days. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants