Winning Software

AutoML2

pakdd2018

Only 2 phases:
  1. Feed-back phase: You get to practice (with code or result submission) on 5 datasets and get immediate feed-back on a leaderboard.
  2. AutoML phase: The code gets blind tested on 5 new datasets.
The setting is similar to AutoML1, so you can get a head start by studying what was done.


AutoML1

http://codalab.org/AutoML 

The First ChaLearn AutoML challenge included 5 rounds, 30 datasets (available for download), and $30,000 in prizes. The goal was to design the perfect machine learning “black box” capable of performing all model selection and hyper-parameter tuning without any human intervention. Post-challenge submissions can be made on the clone websites:

In phase Final4, we also had a GPU track.

RESULTS
We have released a short analysis of the challenge results for the ICML 2016 workshop on AutoML.
The results of top ranking participants of the first round were discussed at the AutoML workshop at ICML 2015. See summary paper. A more detailed paper was presented at IJCNN 2015. The NIPS 2016 [slides] give an update. We also have published the results of a short survey on methods used (fact sheets). Thanks for citing our papers, see citations for the BibTex entries.

The top ranking participants described briefly their methods in the FACT SHEETS and wrote blogs:
WINNERS

The competition started December 8, 2014 and ended May 1, 2016. 
 Phase  Ended  Winners
Final 0 02/14/15 1. ideal.intel.analytics [code not released]
2. abhishek4: Abhishek Thakur (Berlin, Germany) [code][paper][slides]
3. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code][paper][slides]
 AutoML 1 02/15/15 1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code][paper][slides]
2. jrl44James Loyd (Cambridge, UK) [code
3. tadejs: Tadej Stajner (Ljubljana, Slovenia) [code][paper][slides]
 Final 1 06/14/15 1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. ideal.intel.analytics [code not released]
3. amsl.intel.com: Victor Kocheganov (Nizhny Novgorog, Russia) [code]
AutoML 2 06/15/15 1. backstreet.bayes: James Loyd (Cambridge, UK) [code] [slides]
 
2aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
3. matthias.vonrohr [code not released]
 Final 2 11/14/15 1. ideal.intel.analytics [code not released][slides]
2. djajetic: Damir Jajetic (Croatia) [code]
3. aad_freiburgFrank Hutter and collaborators (Freiburg, Germany)
 [code][paper][supplementary material][slides]
AutoML 3 11/15/15 1. djajeticDamir Jajetic (Croatia) [code]
No other qualifiable winner.
 Final 3  2/19/16  1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajeticDamir Jajetic (Croatia) [code]
3. ideal.intel.analytics [code not released]
AutoML 4  2/20/16  1. aad_freiburg: (first place tie) Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajetic(first place tie) Damir Jajetic (Croatia) [code]
3. marc.boulle [code not released]
 Final 4 CPU track

 5/1/16   1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
 2. ideal.intel.analytics [code not released]
 3. abhishek4: Abhishek Thakur (Berlin, Germany) [code]
 Final 4 GPU track   5/1/16  1. abhhishek-GPU: Abhishek Thakur (Berlin, Germany) [code]  2. djajetic-GPU(second place tie) Damir Jajetic (Croatia) [code][fact_sheet]
 3. aad_freiburg-GPU: (second place tie) Frank Hutter and collaborators (Freiburg, Germany) [code][paper]
AutoML 5   5/1/16  1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
 2. djajeticDamir Jajetic (Croatia) [code]
 
3. postech.mlg_exbrain: Kim Jungtaek and collaborators (Republic of Korea)[code][fact_sheet]


Comments