Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimization of DNNs. Neuroevolution is a term, which describes these processes of automated configuration and training of DNNs using EAs. While many works exist in the literature, no comprehensive surveys currently exist focusing exclusively on the strengths and limitations of using neuroevolution approaches in DNNs. Absence of such surveys can lead to a disjointed and fragmented field preventing DNNs researchers potentially adopting neuroevolutionary methods in their own research, resulting in lost opportunities for wider application within real-world deep learning problems. This article presents a comprehensive survey, discussion, and evaluation of the state-of-the-art in using EAs for architectural configuration and training of DNNs. This article highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions. Impact Statement—The concept of deep learning originated from the study of artificial neural networks (ANNs). ANNs have achieved extraordinary results in a variety of diverse application areas. Numerous methods have been applied to the architectural configuration and learning or training of artificial DNN and these methods play a crucial role in the success or failure of the DNN for most problems and applications. Recently, EAs have been gaining momentum as a computationally feasible method (called neuroevolution) for the automated configuration and learning or training of DNNs. This article reviews over 170 recent scientific papers describing how major EAs paradigms are being applied by researchers to the configuration and optimization of multiple DNNs. By articulating a clear understanding of the context, state-of-the-art, and feasibility of Neuroevolution, researchers in AI, EAs, and DNN will benefit from this article. The impact of this article comes from contributing toward enhancing research capacity, knowledge, and skills for researchers currently working in neuroevolution and actively engaging those considering becoming involved in this area.

[1]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[2]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[3]  M. Kimura.  Evolutionary Rate at the Molecular Level , 1968, Nature.

[4]  John Holland,et al.  Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology , 1975 .

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[8]  Ingo Rechenberg,et al.  Evolution Strategy: Nature’s Way of Optimization , 1989 .

[9]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[12]  David B. Fogel,et al.  Alternative Neural Network Training Methods , 1995, IEEE Expert.

[13]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[14]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[15]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[16]  Michael O'Neill,et al.  Grammatical Evolution: Evolving Programs for an Arbitrary Language , 1998, EuroGP.

[17]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[18]  C. A. Coello Coello,et al.  A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques , 1999, Knowledge and Information Systems.

[19]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[20]  T. Jukes,et al.  The neutral theory of molecular evolution. , 2000, Genetics.

[21]  Julian Francis Miller,et al.  Cartesian genetic programming , 2000, GECCO '10.

[22]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[23]  Jürgen Schmidhuber,et al.  LSTM recurrent networks learn simple context-free and context-sensitive languages , 2001, IEEE Trans. Neural Networks.

[24]  Martin Mandischer.  A comparison of evolution strategies and backpropagation for neural network training , 2002, Neurocomputing.

[25]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[26]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[27]  Peter J. Fleming,et al.  Evolutionary algorithms in control systems engineering: a survey , 2002 .

[28]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[29]  John R. Koza,et al.  Genetic Programming IV: Routine Human-Competitive Machine Intelligence , 2003 .

[30]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[31]  P. Lennie.  Receptive fields , 2003, Current Biology.

[32]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[33]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[34]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[35]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[36]  Riccardo Poli,et al.  An empirical investigation of how and why neutrality affects evolutionary search , 2006, GECCO '06.

[37]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[38]  C. A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Computational Intelligence Magazine.

[39]  David Zhang,et al.  A Direct Evolutionary Feature Extraction Algorithm for Classifying High Dimensional Data , 2006, AAAI.

[40]  Riccardo Poli,et al.  Some Steps Towards Understanding How Neutrality Affects Evolutionary Search , 2006, PPSN.

[41]  Riccardo Poli,et al.  On the effects of bit-wise neutrality on fitness distance correlation, phenotypic mutation rates and problem hardness , 2007, FOGA'07.

[42]  Yoshua Bengio,et al.  An empirical evaluation of deep architectures on problems with many factors of variation , 2007, ICML '07.

[43]  Riccardo Poli,et al.  The Effects of Constant Neutrality on Performance and Problem Hardness in GP , 2008, EuroGP.

[44]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[45]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[46]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[47]  Edgar Galván López.  An analysis of the effects of neutrality on problem hardness for evolutionary algorithms , 2009 .

[48]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[49]  Riccardo Poli,et al.  An Empirical Investigation of How Degree Neutrality Affects GP Search , 2009, MICAI.

[50]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[51]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[52]  John R. Koza,et al.  Human-competitive results produced by genetic programming , 2010, Genetic Programming and Evolvable Machines.

[53]  Frank Sehnke,et al.  Parameter-exploring policy gradients , 2010, Neural Networks.

[54]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[55]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[56]  Kenneth O. Stanley,et al.  Novelty Search and the Problem with Objectives , 2011 .

[57]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[58]  Riccardo Poli,et al.  Neutrality in evolutionary algorithms… What do we know? , 2011, Evol. Syst..

[59]  Tara N. Sainath,et al.  Deep Belief Networks using discriminative features for phone recognition , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[60]  Razvan Pascanu,et al.  Learning Algorithms for the Classification Restricted Boltzmann Machine , 2012, J. Mach. Learn. Res..

[61]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[62]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[63]  Riccardo Poli,et al.  The Effects of Constant and Bit-Wise Neutrality on Problem Hardness, Fitness Distance Correlation and Phenotypic Mutation Rates , 2012, IEEE Transactions on Evolutionary Computation.

[64]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[65]  Christian Igel,et al.  An Introduction to Restricted Boltzmann Machines , 2012, CIARP.

[66]  Pierre Baldi,et al.  Autoencoders, Unsupervised Learning, and Deep Architectures , 2011, ICML Unsupervised and Transfer Learning.

[67]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[68]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[69]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[70]  Xiaogang Wang,et al.  Hybrid Deep Learning for Face Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[71]  Iddo Greental,et al.  Genetic algorithms for evolving deep neural networks , 2014, GECCO.

[72]  Daniel Cownden,et al.  Random feedback weights support learning in deep neural networks , 2014, ArXiv.

[73]  Xue-wen Chen,et al.  Big Data Deep Learning: Challenges and Perspectives , 2014, IEEE Access.

[74]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[75]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[76]  A. E. Eiben,et al.  From evolutionary computation to the evolution of things , 2015, Nature.

[77]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[78]  Shinji Watanabe,et al.  Structure discovery of deep neural network based on evolutionary algorithms , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[79]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[80]  Meng Wang,et al.  Multimodal Deep Autoencoder for Human Pose Recovery , 2015, IEEE Transactions on Image Processing.

[81]  Alexandros Agapitos,et al.  Deep evolution of image representations for handwritten digit recognition , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[82]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[83]  Yi Shang,et al.  EvoAE -- A New Evolutionary Method for Training Autoencoders for Deep Learning Networks , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[84]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[85]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[86]  Andrew V. Goldberg,et al.  Route Planning in Transportation Networks , 2015, Algorithm Engineering.

[87]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[88]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[89]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[90]  Kenneth O. Stanley,et al.  Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks , 2016, GECCO.

[91]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[92]  Erik Ostermann.  How To Solve It Modern Heuristics , 2016 .

[93]  David Pfau,et al.  Convolution by Evolution: Differentiable Pattern Producing Networks , 2016, GECCO.

[94]  Gang Liu,et al.  Image classification with stacked restricted boltzmann machines and evolutionary function array classification voter , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[95]  Sophia Blau,et al.  Numerical Optimization Of Computer Models , 2016 .

[96]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[97]  Ye-Hoon Kim,et al.  NEMO : Neuro-Evolution with Multiobjective Optimization of Deep Neural Network for Speed and Accuracy , 2017 .

[98]  Emmanuel Dufourq,et al.  EDEN: Evolutionary deep networks for efficient machine learning , 2017, 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech).

[99]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[100]  Alan L. Yuille,et al.  Genetic CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[101]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[102]  Quoc V. Le,et al.  Neural Architecture Search with Reinforcement Learning , 2016, ICLR.

[103]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[104]  Geoffrey J. Gordon,et al.  DeepArchitect: Automatically Designing and Training Deep Architectures , 2017, ArXiv.

[105]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[106]  Junjie Yan,et al.  Practical Network Blocks Design with Q-Learning , 2017, ArXiv.

[107]  Masanori Suganuma,et al.  A genetic programming approach to designing convolutional neural network architectures , 2017, GECCO.

[108]  Travis Desell,et al.  Large scale evolution of convolutional neural networks using volunteer computing , 2017, GECCO.

[109]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[110]  Kenneth O. Stanley,et al.  Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning , 2017, ArXiv.

[111]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[112]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[113]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[114]  Frank Hutter,et al.  Simple And Efficient Architecture Search for Convolutional Neural Networks , 2017, ICLR.

[115]  Bin Wang,et al.  A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification , 2018, Australasian Conference on Artificial Intelligence.

[116]  Xiaogang Wang,et al.  Structure Learning for Deep Neural Networks Based on Multiobjective Optimization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[117]  Amar Phanishayee,et al.  Benchmarking and Analyzing Deep Neural Network Training , 2018, 2018 IEEE International Symposium on Workload Characterization (IISWC).

[118]  Li Fei-Fei,et al.  Progressive Neural Architecture Search , 2017, ECCV.

[119]  Masanori Suganuma,et al.  Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search , 2018, ICML.

[120]  Kirthevasan Kandasamy,et al.  Neural Architecture Search with Bayesian Optimisation and Optimal Transport , 2018, NeurIPS.

[121]  Yong Yu,et al.  Efficient Architecture Search by Network Transformation , 2017, AAAI.

[122]  Oriol Vinyals,et al.  Hierarchical Representations for Efficient Architecture Search , 2017, ICLR.

[123]  Valery Naranjo,et al.  EvoDeep: A new evolutionary approach for automatic Deep Neural Networks parametrisation , 2018, J. Parallel Distributed Comput..

[124]  Rui Liu,et al.  Effective long short-term memory with differential evolution algorithm for electricity price prediction , 2018, Energy.

[125]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[126]  Kagan Tumer,et al.  Evolution-Guided Policy Gradient in Reinforcement Learning , 2018, NeurIPS.

[127]  Nuno Lourenço,et al.  DENSER: deep evolutionary network structured representation , 2018, Genetic Programming and Evolvable Machines.

[128]  Theodore Lim,et al.  SMASH: One-Shot Model Architecture Search through HyperNetworks , 2017, ICLR.

[129]  Jakub Nalepa,et al.  Genetically-trained deep neural networks , 2018, GECCO.

[130]  Nuno Lourenço,et al.  Evolving the Topology of Large Scale Deep Neural Networks , 2018, EuroGP.

[131]  Quoc V. Le,et al.  The Evolved Transformer , 2019, ICML.

[132]  Mauro Castelli,et al.  Explorations of the Semantic Learning Machine Neuroevolution Algorithm: Dynamic Training Data Use, Ensemble Construction Methods, and Deep Learning Perspectives , 2019, GPTP.

[133]  Kalyanmoy Deb,et al.  NSGA-Net: neural architecture search using multi-objective genetic algorithm , 2018, GECCO.

[134]  Alok Aggarwal,et al.  Regularized Evolution for Image Classifier Architecture Search , 2018, AAAI.

[135]  Risto Miikkulainen,et al.  Designing neural networks through neuroevolution , 2019, Nat. Mach. Intell..

[136]  Marius Lindauer,et al.  Best Practices for Scientific Research on Neural Architecture Search , 2019, ArXiv.

[137]  Haizhou Li,et al.  A Cost-Sensitive Deep Belief Network for Imbalanced Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[138]  Zhang Yi,et al.  Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations , 2017, IEEE Transactions on Evolutionary Computation.

[139]  Elliot Meyerson,et al.  Evolving Deep Neural Networks , 2017, Artificial Intelligence in the Age of Neural Networks and Brain Computing.

[140]  Jun Liang,et al.  Discriminant autoencoder for feature extraction in fault diagnosis , 2019, Chemometrics and Intelligent Laboratory Systems.

[141]  Jen Jen Chung,et al.  Neuroevolution of a Modular Memory-Augmented Neural Network for Deep Memory Problems , 2019, Evolutionary Computation.

[142]  Anna Sergeevna Bosman,et al.  Evolutionary Neural Architecture Search for Image Restoration , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[143]  Ashraf Darwish,et al.  A survey of swarm and evolutionary computing approaches for deep learning , 2019, Artificial Intelligence Review.

[144]  Travis Desell,et al.  Investigating recurrent neural network memory structures using neuro-evolution , 2019, GECCO.

[145]  Frank Hutter,et al.  Neural Architecture Search , 2019, Automated Machine Learning.

[146]  Mengjie Zhang,et al.  Evolving Deep Convolutional Neural Networks for Image Classification , 2017, IEEE Transactions on Evolutionary Computation.

[147]  Mengjie Zhang,et al.  Completely Automated CNN Architecture Design Based on Blocks , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[148]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[149]  Antonio J. Rivera,et al.  EvoAAA: An evolutionary methodology for automated neural autoencoder architecture search , 2020, Integr. Comput. Aided Eng..

[150]  DEVDAN: Deep Evolving Denoising Autoencoder , 2019, Neurocomputing.

[151]  P. Alam.  ‘L’ , 2021, Composites Engineering: An A–Z Guide.

[152]  Suely Oliveira,et al.  Distributed Evolution of Deep Autoencoders , 2020, SAI.

[153]  C. Goerick,et al.  Evolution Strategies: an Alternative to Gradient Based Learning , 2022 .