LAST DAYS! Joins us June 3rd at PAKDD conference


A new competition with CODE submission.
Only 2 phases:
  1. Feed-back phase: You get to practice (with code or result submission) on 5 datasets and get immediate feed-back on a leaderboard.
  2. AutoML phase: The code gets blind tested on 5 new datasets.
The setting is similar to AutoML1, so you can get a head start by studying what was done



The First ChaLearn AutoML challenge included 5 rounds, 30 datasets (available for download), and $30,000 in prizes. The goal was to design the perfect machine learning “black box” capable of performing all model selection and hyper-parameter tuning without any human intervention. Post-challenge submissions can be made on the clone websites:

In phase Final4, we also had a GPU track.

We have released a short analysis of the challenge results for the ICML 2016 workshop on AutoML.
The results of top ranking participants of the first round were discussed at the AutoML workshop at ICML 2015. See summary paper. A more detailed paper was presented at IJCNN 2015. The NIPS 2016 [slides] give an update. We also have published the results of a short survey on methods used (fact sheets). Thanks for citing our papers, see citations for the BibTex entries.

The top ranking participants described briefly their methods in the FACT SHEETS and wrote blogs:

The competition started December 8, 2014 and ended May 1, 2016. 
 Phase Ended Winners
Final 002/14/151. ideal.intel.analytics [code not released]
2. abhishek4: Abhishek Thakur (Berlin, Germany) [code][paper][slides]
3. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code][paper][slides]
 AutoML 102/15/151. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code][paper][slides]
2. jrl44James Loyd (Cambridge, UK) [code
3. tadejs: Tadej Stajner (Ljubljana, Slovenia) [code][paper][slides]
 Final 106/14/151. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. ideal.intel.analytics [code not released]
3. amsl.intel.com: Victor Kocheganov (Nizhny Novgorog, Russia) [code]
AutoML 206/15/151. backstreet.bayes: James Loyd (Cambridge, UK) [code] [slides]
2aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
3. matthias.vonrohr [code not released]
 Final 211/14/151. ideal.intel.analytics [code not released][slides]
2. djajetic: Damir Jajetic (Croatia) [code]
3. aad_freiburgFrank Hutter and collaborators (Freiburg, Germany)
 [code][paper][supplementary material][slides]
AutoML 311/15/151. djajeticDamir Jajetic (Croatia) [code]
No other qualifiable winner.
 Final 3 2/19/16 1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajeticDamir Jajetic (Croatia) [code]
3. ideal.intel.analytics [code not released]
AutoML 4 2/20/16 1. aad_freiburg: (first place tie) Frank Hutter and collaborators (Freiburg, Germany) [code]
2. djajetic(first place tie) Damir Jajetic (Croatia) [code]
3. marc.boulle [code not released]
 Final 4 CPU track

 5/1/16  1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
 2. ideal.intel.analytics [code not released]
 3. abhishek4: Abhishek Thakur (Berlin, Germany) [code]
 Final 4 GPU track  5/1/16 1. abhhishek-GPU: Abhishek Thakur (Berlin, Germany) [code]  2. djajetic-GPU(second place tie) Damir Jajetic (Croatia) [code][fact_sheet]
 3. aad_freiburg-GPU: (second place tie) Frank Hutter and collaborators (Freiburg, Germany) [code][paper]
AutoML 5  5/1/16 1. aad_freiburg: Frank Hutter and collaborators (Freiburg, Germany) [code]
 2. djajeticDamir Jajetic (Croatia) [code]
3. postech.mlg_exbrain: Kim Jungtaek and collaborators (Republic of Korea)[code][fact_sheet]


December 9, 2017:
We will announce AutoML2 at PAKDD at the CiML workshop.
June 21, 2016: Microsoft published a BLOG post.
June 24, 2016: We have an accepted workshop at ICML 2016.
[challenge analysis paper].
July 26, 2016: We have a talk at WCCI, in the competition track.

March 2016: As AutoML is near its end, we are starting to prepare a Life Long Learning challenge in machine learning.

December 11-12 2015: We had a great NIPS workshop.

The results of the AutoML4 phase are out!

We re-implemented old challenges in Codalab [NIPS 2003][KDDcup 2009, small, large] and created one for [MNIST data]

Subpages (1): Eugene Tuv
Isabelle Guyon,
Sep 30, 2016, 9:09 AM
Isabelle Guyon,
Sep 30, 2016, 10:38 AM
Isabelle Guyon,
Sep 30, 2016, 9:10 AM
Isabelle Guyon,
Sep 30, 2016, 9:10 AM
Isabelle Guyon,
May 17, 2016, 5:14 PM
Isabelle Guyon,
May 3, 2016, 11:42 AM
Isabelle Guyon,
May 23, 2016, 2:05 PM
Isabelle Guyon,
Jun 2, 2016, 8:49 AM
Isabelle Guyon,
Oct 13, 2016, 2:41 PM
Isabelle Guyon,
Aug 28, 2015, 11:46 AM
Isabelle Guyon,
Feb 8, 2015, 9:05 AM