An Adaptive and Near Parameter-free Evolutionary Computation Approach Towards True Automation in AutoML

A common claim of evolutionary computation methods is that they can achieve good results without the need for human intervention. However, one criticism of this is that there are still hyperparameters which must be tuned in order to achieve good performance. In this work, we propose a near parameter-free genetic programming approach, which adapts the hyperparameter values throughout evolution without ever needing to be specified manually. We apply this to the area of automated machine learning (by extending TPOT), to produce pipelines which can effectively be claimed to be free from human input, and show that the results are competitive with existing state-of-the-art which use hand-selected hyperparameter values. Pipelines begin with a randomly chosen estimator and evolve to competitive pipelines automatically. This work moves towards a truly automated approach to AutoML.

[1]  D. Thierens.  Adaptive mutation rate control schemes in genetic algorithms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  Kalyanmoy Deb,et al.  Towards estimating nadir objective vector using evolutionary approaches , 2006, GECCO.

[3]  Kevin Leyton-Brown,et al.  Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms , 2012, KDD.

[4]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[5]  A. Newell,et al.  Phyllotactic patterns on plants. , 2004, Physical review letters.

[6]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[7]  Demis Hassabis,et al.  Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm , 2017, ArXiv.

[8]  Carlos M. Fonseca,et al.  An Improved Dimension-Sweep Algorithm for the Hypervolume Indicator , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[9]  Elena Marchiori,et al.  Evolutionary Algorithms with On-the-Fly Population Size Adjustment , 2004, PPSN.

[10]  Marcus Randall.  Near Parameter Free Ant Colony Optimisation , 2004, ANTS Workshop.

[11]  Benjamin Evans,et al.  Population-based Ensemble Learning with Tree Structures for Classification , 2019 .

[12]  Zbigniew Michalewicz,et al.  GAVaPS-a genetic algorithm with varying population size , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[13]  Randal S. Olson,et al.  Automating Biomedical Data Science Through Tree-Based Pipeline Optimization , 2016, EvoApplications.

[14]  Joaquin Vanschoren,et al.  GAMA: Genetic Automated Machine learning Assistant , 2019, J. Open Source Softw..

[15]  Frank Hutter,et al.  Towards Further Automation in AutoML , 2018 .

[16]  Kenneth A. De Jong,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods on the Choice of the Offspring Population Size in Evolutionary Algorithms on the Choice of the Offspring Population Size in Evolutionary Algorithms , 2004 .

[17]  Anton V. Eremeev,et al.  Comparing evolutionary algorithms to the (1+1)-EA , 2008, Theor. Comput. Sci..

[18]  Cláudio F. Lima,et al.  A review of adaptive population sizing schemes in genetic algorithms , 2005, GECCO '05.

[19]  ChunXia Zhao,et al.  Particle swarm optimization with adaptive population size and its application , 2009, Appl. Soft Comput..

[20]  Serafina Cuomo,et al.  Fibonacci’s Liber Abaci: A Translation into Modern English of Leonardo Pisano’s Book of Calculation , 2003 .

[21]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[22]  Zbigniew Michalewicz,et al.  Parameter Setting in Evolutionary Algorithms , 2007, Studies in Computational Intelligence.

[23]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[24]  Thomas Koshy,et al.  Fibonacci and Lucas Numbers With Applications , 2018 .

[25]  Irene Moser,et al.  A Systematic Literature Review of Adaptive Parameter Control Methods for Evolutionary Algorithms , 2016, ACM Comput. Surv..

[26]  The Fibonacci Sequence: Nature’s Little Secret , 2014 .

[27]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[28]  Lars Kotthoff,et al.  Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA , 2017, J. Mach. Learn. Res..

[29]  Xin Yao,et al.  A Large Population Size Can Be Unhelpful in Evolutionary Algorithms a Large Population Size Can Be Unhelpful in Evolutionary Algorithms , 2022 .

[30]  Randal S. Olson,et al.  Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science , 2016, GECCO.

[31]  Jaume Bacardit.  Applications of evolutionary computation: 19th European conference, Evoapplications 2016 Porto, Portugal, March 30 – April 1, 2016 proceedings, part II , 2016 .

[32]  James J. Filliben,et al.  Determining Relative Importance and Effective Settings for Genetic Algorithm Control Parameters , 2015, Evolutionary Computation.

[33]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[34]  Lisa M. Sullivan,et al.  Common Statistical Pitfalls in Basic Science Research , 2016, Journal of the American Heart Association.

[35]  Bernd Bischl,et al.  An Open Source AutoML Benchmark , 2019, ArXiv.

[36]  Hidefumi Sawai,et al.  Parameter-Free Genetic Algorithm Inspired by "Disparity Theory of Evolution" , 1998, PPSN.