Skip to content

Commit 218b908

Browse files
author
Github Actions
committed
Eddie Bergman: Fix timeouts related to metalearnings tests (#1508)
1 parent c892b18 commit 218b908

File tree

75 files changed

+2908
-2076
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+2908
-2076
lines changed

‎development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip

0 Bytes
Binary file not shown.

‎development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip

0 Bytes
Binary file not shown.

‎development/_images/sphx_glr_example_inspect_predictions_001.png

622 Bytes
Loading

‎development/_images/sphx_glr_example_inspect_predictions_002.png

3.92 KB
Loading

‎development/_images/sphx_glr_example_inspect_predictions_003.png

8.15 KB
Loading

‎development/_images/sphx_glr_example_inspect_predictions_thumb.png

345 Bytes
Loading

‎development/_images/sphx_glr_example_multi_objective_001.png

216 Bytes
Loading

‎development/_images/sphx_glr_example_multi_objective_thumb.png

-88 Bytes
Loading

‎development/_images/sphx_glr_example_pandas_train_test_001.png

-1.09 KB
Loading

‎development/_images/sphx_glr_example_pandas_train_test_thumb.png

-1.05 KB
Loading

‎development/_images/sphx_glr_example_regression_001.png

513 Bytes
Loading

‎development/_images/sphx_glr_example_regression_thumb.png

-185 Bytes
Loading

‎development/_sources/examples/20_basic/example_classification.rst.txt

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -121,29 +121,29 @@ View the models found by auto-sklearn
121121

122122
.. code-block:: none
123123
124-
rank ensemble_weight type cost duration
125-
model_id
126-
7 1 0.08 extra_trees 0.014184 2.094769
127-
16 3 0.02 gradient_boosting 0.021277 1.468260
128-
21 2 0.04 extra_trees 0.021277 1.980973
129-
2 4 0.04 random_forest 0.028369 2.374208
130-
3 11 0.10 mlp 0.028369 1.425260
131-
6 10 0.08 mlp 0.028369 1.552391
132-
10 9 0.02 random_forest 0.028369 2.654753
133-
11 12 0.12 random_forest 0.028369 2.757030
134-
13 7 0.02 gradient_boosting 0.028369 1.988821
135-
14 8 0.02 mlp 0.028369 2.801952
136-
19 6 0.06 extra_trees 0.028369 3.659178
137-
26 5 0.06 extra_trees 0.028369 3.013445
138-
8 15 0.02 random_forest 0.035461 2.724722
139-
12 13 0.02 gradient_boosting 0.035461 1.756777
140-
17 14 0.02 gradient_boosting 0.035461 2.275373
141-
9 16 0.02 extra_trees 0.042553 2.556095
142-
25 17 0.04 adaboost 0.042553 2.790631
143-
24 19 0.12 random_forest 0.070922 2.419922
144-
29 18 0.02 decision_tree 0.070922 1.460724
145-
20 20 0.06 passive_aggressive 0.078014 1.035407
146-
30 21 0.02 extra_trees 0.141844 2.177066
124+
rank ensemble_weight type cost duration
125+
model_id
126+
7 1 0.06 extra_trees 0.014184 1.679163
127+
16 3 0.04 gradient_boosting 0.021277 1.235884
128+
21 2 0.06 extra_trees 0.021277 1.564166
129+
2 4 0.04 random_forest 0.028369 1.836669
130+
3 5 0.06 mlp 0.028369 1.038364
131+
14 9 0.04 mlp 0.028369 1.775494
132+
19 10 0.02 extra_trees 0.028369 3.151361
133+
22 8 0.04 gradient_boosting 0.028369 1.246145
134+
26 7 0.02 extra_trees 0.028369 2.508754
135+
31 6 0.10 extra_trees 0.028369 1.874738
136+
5 14 0.02 random_forest 0.035461 2.130856
137+
8 13 0.04 random_forest 0.035461 2.098375
138+
12 12 0.02 gradient_boosting 0.035461 1.460034
139+
17 11 0.02 gradient_boosting 0.035461 1.813402
140+
9 15 0.10 extra_trees 0.042553 1.974389
141+
30 16 0.12 liblinear_svc 0.042553 1.085129
142+
15 17 0.02 mlp 0.049645 3.767453
143+
28 18 0.02 mlp 0.056738 1.079536
144+
29 19 0.08 extra_trees 0.056738 1.903980
145+
33 20 0.02 mlp 0.085106 1.310837
146+
32 21 0.06 extra_trees 0.106383 3.348995
147147
148148
149149
@@ -207,7 +207,7 @@ Get the Score of the final ensemble
207207
208208
.. rst-class:: sphx-glr-timing
209209

210-
**Total running time of the script:** ( 2 minutes 2.560 seconds)
210+
**Total running time of the script:** ( 2 minutes 0.249 seconds)
211211

212212

213213
.. _sphx_glr_download_examples_20_basic_example_classification.py:

‎development/_sources/examples/20_basic/example_multilabel_classification.rst.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ View the models found by auto-sklearn
155155
156156
rank ensemble_weight type cost duration
157157
model_id
158-
2 1 1.0 random_forest 0.447294 4.474277
158+
2 1 1.0 random_forest 0.447294 3.712529
159159
160160
161161
@@ -257,7 +257,7 @@ Get the Score of the final ensemble
257257
258258
.. rst-class:: sphx-glr-timing
259259

260-
**Total running time of the script:** ( 0 minutes 16.641 seconds)
260+
**Total running time of the script:** ( 0 minutes 24.663 seconds)
261261

262262

263263
.. _sphx_glr_download_examples_20_basic_example_multilabel_classification.py:

‎development/_sources/examples/20_basic/example_multioutput_regression.rst.txt

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -123,9 +123,10 @@ View the models found by auto-sklearn
123123

124124
.. code-block:: none
125125
126-
rank ensemble_weight type cost duration
127-
model_id
128-
2 1 1.0 random_forest 0.183452 3.042198
126+
rank ensemble_weight type cost duration
127+
model_id
128+
2 1 0.96 random_forest 0.109310 2.524796
129+
18 2 0.04 random_forest 0.157617 27.725271
129130
130131
131132
@@ -182,7 +183,7 @@ Get the Score of the final ensemble
182183

183184
.. code-block:: none
184185
185-
R2 score: 0.8258957909196227
186+
R2 score: 0.8771877459908536
186187
187188
188189
@@ -435,7 +436,7 @@ Get the configuration space
435436
436437
.. rst-class:: sphx-glr-timing
437438

438-
**Total running time of the script:** ( 1 minutes 54.648 seconds)
439+
**Total running time of the script:** ( 1 minutes 54.937 seconds)
439440

440441

441442
.. _sphx_glr_download_examples_20_basic_example_multioutput_regression.py:

‎development/_sources/examples/20_basic/example_regression.rst.txt

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -122,14 +122,12 @@ View the models found by auto-sklearn
122122

123123
.. code-block:: none
124124
125-
rank ensemble_weight type cost duration
126-
model_id
127-
25 1 0.38 sgd 0.436679 0.930268
128-
6 2 0.40 ard_regression 0.455042 0.968481
129-
27 3 0.02 ard_regression 0.462249 0.924505
130-
29 4 0.16 ard_regression 0.493390 0.881591
131-
11 5 0.02 random_forest 0.507400 10.770000
132-
7 6 0.02 gradient_boosting 0.518673 1.657472
125+
rank ensemble_weight type cost duration
126+
model_id
127+
31 1 0.54 ard_regression 0.428169 0.910553
128+
25 2 0.24 sgd 0.436679 0.777051
129+
29 3 0.18 ard_regression 0.493390 0.759394
130+
7 4 0.04 gradient_boosting 0.518673 1.351315
133131
134132
135133
@@ -192,8 +190,8 @@ predicting the data mean has an R2 score of 0.
192190

193191
.. code-block:: none
194192
195-
Train R2 score: 0.5845958689841146
196-
Test R2 score: 0.3969087782855073
193+
Train R2 score: 0.5891379293923084
194+
Test R2 score: 0.3966515401967873
197195
198196
199197
@@ -238,7 +236,7 @@ the true value).
238236

239237
.. rst-class:: sphx-glr-timing
240238

241-
**Total running time of the script:** ( 2 minutes 2.359 seconds)
239+
**Total running time of the script:** ( 1 minutes 59.783 seconds)
242240

243241

244242
.. _sphx_glr_download_examples_20_basic_example_regression.py:

‎development/_sources/examples/20_basic/sg_execution_times.rst.txt

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,14 @@
55

66
Computation times
77
=================
8-
**06:16.208** total execution time for **examples_20_basic** files:
8+
**06:19.631** total execution time for **examples_20_basic** files:
99

1010
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
11-
| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``) | 02:02.560 | 0.0 MB |
11+
| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``) | 02:00.249 | 0.0 MB |
1212
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
13-
| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``) | 02:02.359 | 0.0 MB |
13+
| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``) | 01:59.783 | 0.0 MB |
1414
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
15-
| :ref:`sphx_glr_examples_20_basic_example_multioutput_regression.py` (``example_multioutput_regression.py``) | 01:54.648 | 0.0 MB |
15+
| :ref:`sphx_glr_examples_20_basic_example_multioutput_regression.py` (``example_multioutput_regression.py``) | 01:54.937 | 0.0 MB |
1616
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
17-
| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:16.641 | 0.0 MB |
17+
| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:24.663 | 0.0 MB |
1818
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+

‎development/_sources/examples/40_advanced/example_calc_multiple_metrics.rst.txt

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -151,47 +151,49 @@ Get the Score of the final ensemble
151151

152152
.. code-block:: none
153153
154-
Accuracy score 0.958041958041958
154+
Accuracy score 0.951048951048951
155155
################################################################################
156156
Metric results
157157
rank_test_scores param_classifier:__choice__ mean_test_score metric_balanced_accuracy metric_precision metric_recall metric_f1 metric_custom_error
158158
4 random_forest 0.971631 0.969533 0.977528 0.977528 0.977528 0.028369
159159
4 mlp 0.971631 0.961538 0.956989 1.000000 0.978022 0.028369
160-
24 mlp 0.943262 0.935069 0.945055 0.966292 0.955556 0.056738
161-
14 random_forest 0.964539 0.959918 0.966667 0.977528 0.972067 0.035461
160+
25 mlp 0.943262 0.935069 0.945055 0.966292 0.955556 0.056738
161+
15 random_forest 0.964539 0.959918 0.966667 0.977528 0.972067 0.035461
162162
4 mlp 0.971631 0.961538 0.956989 1.000000 0.978022 0.028369
163163
1 extra_trees 0.985816 0.984767 0.988764 0.988764 0.988764 0.014184
164-
14 random_forest 0.964539 0.963915 0.977273 0.966292 0.971751 0.035461
165-
19 extra_trees 0.957447 0.954300 0.966292 0.966292 0.966292 0.042553
164+
15 random_forest 0.964539 0.963915 0.977273 0.966292 0.971751 0.035461
165+
20 extra_trees 0.957447 0.954300 0.966292 0.966292 0.966292 0.042553
166166
4 random_forest 0.971631 0.969533 0.977528 0.977528 0.977528 0.028369
167167
4 random_forest 0.971631 0.969533 0.977528 0.977528 0.977528 0.028369
168-
14 gradient_boosting 0.964539 0.963915 0.977273 0.966292 0.971751 0.035461
168+
15 gradient_boosting 0.964539 0.963915 0.977273 0.966292 0.971751 0.035461
169169
4 gradient_boosting 0.971631 0.965536 0.967033 0.988764 0.977778 0.028369
170170
4 mlp 0.971631 0.965536 0.967033 0.988764 0.977778 0.028369
171-
21 mlp 0.950355 0.948682 0.965909 0.955056 0.960452 0.049645
171+
23 mlp 0.950355 0.948682 0.965909 0.955056 0.960452 0.049645
172172
2 gradient_boosting 0.978723 0.975151 0.977778 0.988764 0.983240 0.021277
173-
14 gradient_boosting 0.964539 0.959918 0.966667 0.977528 0.972067 0.035461
174-
14 random_forest 0.964539 0.959918 0.966667 0.977528 0.972067 0.035461
173+
15 gradient_boosting 0.964539 0.959918 0.966667 0.977528 0.972067 0.035461
174+
15 random_forest 0.964539 0.959918 0.966667 0.977528 0.972067 0.035461
175175
4 extra_trees 0.971631 0.969533 0.977528 0.977528 0.977528 0.028369
176-
27 passive_aggressive 0.921986 0.894231 0.890000 1.000000 0.941799 0.078014
176+
29 passive_aggressive 0.921986 0.894231 0.890000 1.000000 0.941799 0.078014
177177
2 extra_trees 0.978723 0.975151 0.977778 0.988764 0.983240 0.021277
178178
4 gradient_boosting 0.971631 0.965536 0.967033 0.988764 0.977778 0.028369
179-
21 mlp 0.950355 0.940687 0.945652 0.977528 0.961326 0.049645
180-
25 random_forest 0.929078 0.923833 0.943820 0.943820 0.943820 0.070922
181-
19 adaboost 0.957447 0.950303 0.956044 0.977528 0.966667 0.042553
179+
23 mlp 0.950355 0.940687 0.945652 0.977528 0.961326 0.049645
180+
28 random_forest 0.929078 0.923833 0.943820 0.943820 0.943820 0.070922
181+
20 adaboost 0.957447 0.950303 0.956044 0.977528 0.966667 0.042553
182182
4 extra_trees 0.971631 0.965536 0.967033 0.988764 0.977778 0.028369
183-
29 mlp 0.368794 0.500000 0.000000 0.000000 0.000000 0.631206
184-
21 extra_trees 0.950355 0.952679 0.976744 0.943820 0.960000 0.049645
185-
25 decision_tree 0.929078 0.935825 0.975904 0.910112 0.941860 0.070922
186-
28 extra_trees 0.858156 0.807692 0.816514 1.000000 0.898990 0.141844
183+
31 mlp 0.368794 0.500000 0.000000 0.000000 0.000000 0.631206
184+
25 mlp 0.943262 0.947061 0.976471 0.932584 0.954023 0.056738
185+
25 extra_trees 0.943262 0.931072 0.935484 0.977528 0.956044 0.056738
186+
20 liblinear_svc 0.957447 0.962295 0.988235 0.943820 0.965517 0.042553
187+
4 extra_trees 0.971631 0.969533 0.977528 0.977528 0.977528 0.028369
188+
30 extra_trees 0.893617 0.887748 0.920455 0.910112 0.915254 0.106383
187189
188190
189191
190192
191193
192194
.. rst-class:: sphx-glr-timing
193195

194-
**Total running time of the script:** ( 2 minutes 5.113 seconds)
196+
**Total running time of the script:** ( 1 minutes 58.693 seconds)
195197

196198

197199
.. _sphx_glr_download_examples_40_advanced_example_calc_multiple_metrics.py:

‎development/_sources/examples/40_advanced/example_debug_logging.rst.txt

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -175,11 +175,11 @@ Build and fit a classifier
175175
176176
/home/runner/work/auto-sklearn/auto-sklearn/autosklearn/data/target_validator.py:187: UserWarning: Fitting transformer with a pandas series which has the dtype category. Inverse transform may not be able preserve dtype when converting to np.ndarray
177177
warnings.warn(
178-
2022-06-17 12:51:31,331 - Client-EnsembleBuilder - INFO - DummyFuture: ([{'Timestamp': Timestamp('2022-06-17 12:51:31.305701'), 'ensemble_optimization_score': 0.47853535353535354, 'ensemble_test_score': 0.47434292866082606}], 50)/SingleThreadedClient() Started Ensemble builder job at 2022.06.17-12.51.31 for iteration 0.
179-
2022-06-17 12:51:36,660 - Client-EnsembleBuilder - INFO - DummyFuture: ([{'Timestamp': Timestamp('2022-06-17 12:51:36.639349'), 'ensemble_optimization_score': 0.9684343434343434, 'ensemble_test_score': 0.9824780976220275}], 50)/SingleThreadedClient() Started Ensemble builder job at 2022.06.17-12.51.36 for iteration 1.
178+
2022-06-17 13:53:35,251 - Client-EnsembleBuilder - INFO - DummyFuture: ([{'Timestamp': Timestamp('2022-06-17 13:53:35.233185'), 'ensemble_optimization_score': 0.9886363636363636, 'ensemble_test_score': 0.9899874843554443}], 50)/SingleThreadedClient() Started Ensemble builder job at 2022.06.17-13.53.35 for iteration 0.
179+
2022-06-17 13:53:39,428 - Client-EnsembleBuilder - INFO - DummyFuture: ([{'Timestamp': Timestamp('2022-06-17 13:53:39.409255'), 'ensemble_optimization_score': 0.98989898989899, 'ensemble_test_score': 0.9924906132665833}], 50)/SingleThreadedClient() Started Ensemble builder job at 2022.06.17-13.53.39 for iteration 1.
180180
tmp_folder/space.json
181+
tmp_folder/AutoML(1):de4ca5f7-ee44-11ec-87ca-495084c38962.log
181182
tmp_folder/distributed.log
182-
tmp_folder/AutoML(1):3060aec3-ee3c-11ec-87f8-bd77db06b4a1.log
183183
tmp_folder/.auto-sklearn
184184
tmp_folder/smac3-output
185185
@@ -189,7 +189,7 @@ Build and fit a classifier
189189
190190
.. rst-class:: sphx-glr-timing
191191

192-
**Total running time of the script:** ( 0 minutes 20.245 seconds)
192+
**Total running time of the script:** ( 0 minutes 15.227 seconds)
193193

194194

195195
.. _sphx_glr_download_examples_40_advanced_example_debug_logging.py:

0 commit comments

Comments
 (0)