Only 2 phases:
- Feed-back phase: You get to practice (with code or result submission) on 5 datasets and get immediate feed-back on a leaderboard.
- AutoML phase: The code gets blind tested on 5 new datasets.
The setting is similar to AutoML1, so you can get a head start by studying what was done.
AutoML1
The First ChaLearn AutoML challenge included 5 rounds, 30 datasets (available for download), and $30,000 in prizes. The goal was to design the perfect machine learning “black box” capable of performing all model selection and hyper-parameter tuning without any human intervention. Post-challenge submissions can be made on the clone websites:
In phase Final4, we also had a GPU track.
RESULTS
We have released a short analysis of the challenge results for the ICML 2016 workshop on AutoML.
The top ranking participants described briefly their methods in the FACT SHEETS and wrote blogs:
WINNERS
The competition started December 8, 2014 and ended May 1, 2016.
Phase |
Ended |
Winners |
Final 0 |
02/14/15 |
1. ideal.intel.analytics [code not released]
2. abhishek4: Abhishek Thakur (Berlin, Germany) [code][paper][slides]
3. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code][paper][slides] |
AutoML 1 |
02/15/15 |
1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code][paper][slides]
2. jrl44: James Loyd (Cambridge, UK) [code]
3. tadejs: Tadej Stajner (Ljubljana, Slovenia) [code][paper][slides]
|
Final 1 |
06/14/15 |
1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. ideal.intel.analytics [code not released]
3. amsl.intel.com: Victor Kocheganov (Nizhny Novgorog, Russia) [code] |
AutoML 2 |
06/15/15 |
1. backstreet.bayes: James Loyd (Cambridge, UK) [code] [slides]
2. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
3. matthias.vonrohr [code not released] |
Final 2 |
11/14/15 |
1. ideal.intel.analytics [code not released][slides]
2. djajetic: Damir Jajetic (Croatia) [code]
3. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany)
[code][paper][supplementary material][slides] |
AutoML 3 |
11/15/15 |
1. djajetic: Damir Jajetic (Croatia) [code]
No other qualifiable winner. |
Final 3 |
2/19/16 |
1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajetic: Damir Jajetic (Croatia) [code]
3. ideal.intel.analytics [code not released] |
AutoML 4 |
2/20/16 |
1. aad_freiburg: (first place tie) Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajetic: (first place tie) Damir Jajetic (Croatia) [code]
3. marc.boulle [code not released] |
Final 4 CPU track
|
5/1/16 |
1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. ideal.intel.analytics [code not released]
3. abhishek4: Abhishek Thakur (Berlin, Germany) [code] |
Final 4 GPU track |
5/1/16 |
1. abhhishek-GPU: Abhishek Thakur (Berlin, Germany) [code]
2. djajetic-GPU: (second place tie) Damir Jajetic (Croatia) [code][fact_sheet]
3. aad_freiburg-GPU: (second place tie) Frank Hutter and collaborators (Freiburg, Germany) [code][paper] |
AutoML 5 |
5/1/16 |
1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajetic: Damir Jajetic (Croatia) [code]
3. postech.mlg_exbrain: Kim Jungtaek and collaborators (Republic of Korea)[code][fact_sheet] |
|
|
|