Eddie Bergman: Fix prediction fails with MOO ensemble and dummy is best (#1518)

Author: Github Actions

development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip

@@ -155,7 +155,7 @@ View the models found by auto-sklearn
 
               rank  ensemble_weight           type      cost  duration
     model_id                                                          
-    2            1              1.0  random_forest  0.447294  3.712529
+    2            1              1.0  random_forest  0.447294  3.239878
 
 
 
@@ -182,9 +182,16 @@ Print the final ensemble constructed by auto-sklearn
 
  .. code-block:: none
 
-    /home/runner/work/auto-sklearn/auto-sklearn/autosklearn/automl.py:2152: UserWarning: No models in the ensemble. Kindly provide an ensemble class.
-      warnings.warn(
-    {}
+    {   2: {   'balancing': Balancing(random_state=1),
+               'classifier': <autosklearn.pipeline.components.classification.ClassifierChoice object at 0x7fbe500c3fa0>,
+               'cost': 0.4472941828699525,
+               'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe4aaaad00>,
+               'ensemble_weight': 1.0,
+               'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe500c3280>,
+               'model_id': 2,
+               'rank': 1,
+               'sklearn_classifier': RandomForestClassifier(max_features=15, n_estimators=512, n_jobs=1,
+                           random_state=1, warm_start=True)}}
 
 
 
@@ -257,7 +264,7 @@ Get the Score of the final ensemble
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  24.663 seconds)
+   **Total running time of the script:** ( 0 minutes  32.104 seconds)
 
 
 .. _sphx_glr_download_examples_20_basic_example_multilabel_classification.py:

development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip

@@ -123,10 +123,9 @@ View the models found by auto-sklearn
 
  .. code-block:: none
 
-              rank  ensemble_weight           type      cost   duration
-    model_id                                                           
-    2            1             0.96  random_forest  0.109310   2.524796
-    18           2             0.04  random_forest  0.157617  27.725271
+              rank  ensemble_weight              type          cost  duration
+    model_id                                                                 
+    28           1              1.0  gaussian_process  1.718292e-08  3.467824
 
 
 
@@ -153,9 +152,17 @@ Print the final ensemble constructed by auto-sklearn
 
  .. code-block:: none
 
-    /home/runner/work/auto-sklearn/auto-sklearn/autosklearn/automl.py:2152: UserWarning: No models in the ensemble. Kindly provide an ensemble class.
-      warnings.warn(
-    {}
+    {   28: {   'cost': 1.71829196426998e-08,
+                'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe4fe3f970>,
+                'ensemble_weight': 1.0,
+                'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe4fe8ee80>,
+                'model_id': 28,
+                'rank': 1,
+                'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe50675820>,
+                'sklearn_regressor': GaussianProcessRegressor(alpha=1.0000000000000018e-08,
+                             kernel=RBF(length_scale=[1, 1, 1, 1, 1]),
+                             n_restarts_optimizer=10, normalize_y=True,
+                             random_state=1)}}
 
 
 
@@ -183,7 +190,7 @@ Get the Score of the final ensemble
 
  .. code-block:: none
 
-    R2 score: 0.8771877459908536
+    R2 score: 0.9999998896841648
 
 
 
@@ -436,7 +443,7 @@ Get the configuration space
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 1 minutes  54.937 seconds)
+   **Total running time of the script:** ( 1 minutes  55.016 seconds)
 
 
 .. _sphx_glr_download_examples_20_basic_example_multioutput_regression.py:

development/_images/sphx_glr_example_inspect_predictions_001.png

@@ -122,12 +122,14 @@ View the models found by auto-sklearn
 
  .. code-block:: none
 
-              rank  ensemble_weight               type      cost  duration
-    model_id                                                              
-    31           1             0.54     ard_regression  0.428169  0.910553
-    25           2             0.24                sgd  0.436679  0.777051
-    29           3             0.18     ard_regression  0.493390  0.759394
-    7            4             0.04  gradient_boosting  0.518673  1.351315
+              rank  ensemble_weight               type      cost   duration
+    model_id                                                               
+    25           1             0.46                sgd  0.436679   0.682268
+    29           2             0.04   gaussian_process  0.445373  13.078738
+    6            3             0.30     ard_regression  0.455042   0.696445
+    27           4             0.12     ard_regression  0.462249   0.678429
+    11           5             0.02      random_forest  0.507400   9.128839
+    7            6             0.06  gradient_boosting  0.518673   1.157540
 
 
 
@@ -154,9 +156,74 @@ Print the final ensemble constructed by auto-sklearn
 
  .. code-block:: none
 
-    /home/runner/work/auto-sklearn/auto-sklearn/autosklearn/automl.py:2152: UserWarning: No models in the ensemble. Kindly provide an ensemble class.
-      warnings.warn(
-    {}
+    {   6: {   'cost': 0.4550418898836528,
+               'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe4b698310>,
+               'ensemble_weight': 0.3,
+               'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe4a87edc0>,
+               'model_id': 6,
+               'rank': 1,
+               'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe4a87e2b0>,
+               'sklearn_regressor': ARDRegression(alpha_1=0.0003701926442639788, alpha_2=2.2118001735899097e-07,
+                  copy_X=False, lambda_1=1.2037591637980971e-06,
+                  lambda_2=4.358378124977852e-09,
+                  threshold_lambda=1136.5286041327277, tol=0.021944240404849075)},
+        7: {   'cost': 0.5186726734789994,
+               'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe4a10c130>,
+               'ensemble_weight': 0.06,
+               'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe4de46130>,
+               'model_id': 7,
+               'rank': 2,
+               'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe4de46bb0>,
+               'sklearn_regressor': HistGradientBoostingRegressor(l2_regularization=1.8428972335335263e-10,
+                                  learning_rate=0.012607824914758717, max_iter=512,
+                                  max_leaf_nodes=10, min_samples_leaf=8,
+                                  n_iter_no_change=0, random_state=1,
+                                  validation_fraction=None, warm_start=True)},
+        11: {   'cost': 0.5073997164657239,
+                'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe50068550>,
+                'ensemble_weight': 0.02,
+                'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe633f2ca0>,
+                'model_id': 11,
+                'rank': 3,
+                'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe633f29a0>,
+                'sklearn_regressor': RandomForestRegressor(bootstrap=False, criterion='mae',
+                          max_features=0.6277363920171745, min_samples_leaf=6,
+                          min_samples_split=15, n_estimators=512, n_jobs=1,
+                          random_state=1, warm_start=True)},
+        25: {   'cost': 0.43667876507897496,
+                'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe50688f70>,
+                'ensemble_weight': 0.46,
+                'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe4a9a55e0>,
+                'model_id': 25,
+                'rank': 4,
+                'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe4a9a5430>,
+                'sklearn_regressor': SGDRegressor(alpha=0.0006517033225329654, epsilon=0.012150149892783745,
+                 eta0=0.016444224834275295, l1_ratio=1.7462342366289323e-09,
+                 loss='epsilon_insensitive', max_iter=16, penalty='elasticnet',
+                 power_t=0.21521743568582094, random_state=1,
+                 tol=0.002431731981071206, warm_start=True)},
+        27: {   'cost': 0.4622486119001967,
+                'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe500a31c0>,
+                'ensemble_weight': 0.12,
+                'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe4a0e5550>,
+                'model_id': 27,
+                'rank': 5,
+                'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe4a0e54f0>,
+                'sklearn_regressor': ARDRegression(alpha_1=2.7664515192592053e-05, alpha_2=9.504988116581138e-07,
+                  copy_X=False, lambda_1=6.50650698230178e-09,
+                  lambda_2=4.238533890074848e-07,
+                  threshold_lambda=78251.58542976103, tol=0.0007301343236220855)},
+        29: {   'cost': 0.44537254042256413,
+                'data_preprocessor': <autosklearn.pipeline.components.data_preprocessing.DataPreprocessorChoice object at 0x7fbe4de625e0>,
+                'ensemble_weight': 0.04,
+                'feature_preprocessor': <autosklearn.pipeline.components.feature_preprocessing.FeaturePreprocessorChoice object at 0x7fbe4a34a2e0>,
+                'model_id': 29,
+                'rank': 6,
+                'regressor': <autosklearn.pipeline.components.regression.RegressorChoice object at 0x7fbe4a34a520>,
+                'sklearn_regressor': GaussianProcessRegressor(alpha=0.323250809620855,
+                             kernel=RBF(length_scale=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
+                             n_restarts_optimizer=10, normalize_y=True,
+                             random_state=1)}}
 
 
 
@@ -190,8 +257,8 @@ predicting the data mean has an R2 score of 0.
 
  .. code-block:: none
 
-    Train R2 score: 0.5891379293923084
-    Test R2 score: 0.3966515401967873
+    Train R2 score: 0.5965240258552345
+    Test R2 score: 0.39560603803897343
 
 
 
@@ -236,7 +303,7 @@ the true value).
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 1 minutes  59.783 seconds)
+   **Total running time of the script:** ( 1 minutes  59.300 seconds)
 
 
 .. _sphx_glr_download_examples_20_basic_example_regression.py:

development/_images/sphx_glr_example_inspect_predictions_002.png

@@ -5,14 +5,14 @@
 
 Computation times
 =================
-**06:19.631** total execution time for **examples_20_basic** files:
+**06:24.818** total execution time for **examples_20_basic** files:
 
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``)                       | 02:00.249 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``)                               | 01:59.300 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``)                               | 01:59.783 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``)                       | 01:58.398 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_multioutput_regression.py` (``example_multioutput_regression.py``)       | 01:54.937 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_multioutput_regression.py` (``example_multioutput_regression.py``)       | 01:55.016 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:24.663 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:32.104 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+