Learning to Evolve

Evolution and learning are two of the fundamental mechanisms by which life adapts in order to survive and to transcend limitations. These biological phenomena inspired successful computational methods such as evolutionary algorithms and deep learning. Evolution relies on random mutations and on random genetic recombination. Here we show that learning to evolve, i.e. learning to mutate and recombine better than at random, improves the result of evolution in terms of fitness increase per generation and even in terms of attainable fitness. We use deep reinforcement learning to learn to dynamically adjust the strategy of evolutionary algorithms to varying circumstances. Our methods outperform classical evolutionary algorithms on combinatorial and continuous optimization problems.

[1]  Mehmet Polat Saka,et al.  Optimum design of nonlinear steel frames with semi-rigid connections using a genetic algorithm , 2001 .

[2]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[3]  Vitaly Levdik,et al.  Time Limits in Reinforcement Learning , 2017, ICML.

[4]  Martijn C. Schut,et al.  Reinforcement Learning for Online Control of Evolutionary Algorithms , 2006, ESOA.

[5]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[6]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[7]  Arina Buzdalova,et al.  Adaptive Parameter Selection in Evolutionary Algorithms by Reinforcement Learning with Dynamic Discretization of Parameter Range , 2016, GECCO.

[8]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[9]  Pratyusha Rakshit,et al.  DE-TDQL: An adaptive memetic algorithm , 2012, 2012 IEEE Congress on Evolutionary Computation.

[10]  S.D. Muller,et al.  Step size adaptation in evolution strategies using reinforcement learning , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[11]  Ramasamy Panneerselvam,et al.  A Study of Crossover Operators for Genetic Algorithm and Proposal of a New Crossover Operator to Solve Open Shop Scheduling Problem , 2016 .

[12]  Zbigniew Michalewicz,et al.  Adaptation in evolutionary computation: a survey , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[13]  Sebastian Scherer,et al.  Improving Stochastic Policy Gradients in Continuous Control with Deep Reinforcement Learning using the Beta Distribution , 2017, ICML.

[14]  Gregory S. Hornby,et al.  An Evolved Antenna for Deployment on NASA's Space Technology 5 Mission , 2004 .

[15]  Mark Hoogendoorn,et al.  Generic parameter control with reinforcement learning , 2014, GECCO.

[16]  Risto Miikkulainen,et al.  Designing neural networks through neuroevolution , 2019, Nat. Mach. Intell..

[17]  Arina Buzdalova,et al.  Choosing Best Fitness Function with Reinforcement Learning , 2011, 2011 10th International Conference on Machine Learning and Applications and Workshops.

[18]  Arina Buzdalova,et al.  Improved Selection of Auxiliary Objectives Using Reinforcement Learning in Non-stationary Environment , 2014, 2014 13th International Conference on Machine Learning and Applications.

[19]  Qingfu Zhang,et al.  Adaptive Operator Selection With Bandits for a Multiobjective Evolutionary Algorithm Based on Decomposition , 2014, IEEE Transactions on Evolutionary Computation.

[20]  Carlo Poloni,et al.  Optimization of wind turbine positioning in large windfarms by means of a genetic algorithm , 1994 .

[21]  Richard M. Everson,et al.  Controlling Genetic Algorithms With Reinforcement Learning , 2002, GECCO.