Evolving Neural Networks through a Reverse Encoding Tree

NeuroEvolution is one of the most competitive evolutionary learning strategies for designing novel neural networks for use in specific tasks, such as logic circuit design and digital gaming. However, the application of benchmark methods such as the NeuroEvolution of Augmenting Topologies (NEAT) remains a challenge, in terms of their computational cost and search time inefficiency. This paper advances a method which incorporates a type of topological edge coding, named Reverse Encoding Tree (RET), for evolving scalable neural networks efficiently. Using RET, two types of approaches – NEAT with Binary search encoding (Bi-NEAT) and NEAT with Golden-Section search encoding (GS-NEAT) – have been designed to solve problems in benchmark continuous learning environments such as logic gates, Cartpole, and Lunar Lander, and tested against classical NEAT and FS-NEAT as baselines. Additionally, we conduct a robustness test to evaluate the resilience of the proposed NEAT approaches. The results show that the two proposed approaches deliver improved performance, characterized by (1) a higher accumulated reward within a finite number of time steps; (2) using fewer episodes to solve problems in targeted environments, and (3) maintaining adaptive robustness under noisy perturbations, which outperform the baselines in all tested cases. Our analysis also demonstrates that RET expends potential future research directions in dynamic environments. Code is available from https://github.com/HaolingZHANG/ReverseEncodingTree.

[1]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[2]  Kenneth O. Stanley,et al.  Safe mutations for deep and recurrent neural networks through output gradients , 2017, GECCO.

[3]  Thomas Bäck,et al.  Genetic Algorithms and Evolution Strategies - Similarities and Differences , 1990, PPSN.

[4]  Lee Spector,et al.  Evolving Graphs and Networks with Edge Encoding: Preliminary Report , 1996 .

[5]  J. Kiefer,et al.  Sequential minimax search for a maximum , 1953 .

[6]  Risto Miikkulainen,et al.  Evolving Reusable Neural Modules , 2004, GECCO.

[7]  Ponnuthurai N. Suganthan,et al.  Real-parameter evolutionary multimodal optimization - A survey of the state-of-the-art , 2011, Swarm Evol. Comput..

[8]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[9]  Jesper Tegnér,et al.  Learning Functions in Large Networks requires Modularity and produces Multi-Agent Dynamics , 2018, ArXiv.

[10]  Javad Alikhani Koupaei,et al.  A new optimization algorithm based on chaotic maps and golden section search method , 2016, Eng. Appl. Artif. Intell..

[11]  Kenneth O. Stanley,et al.  Compositional Pattern Producing Networks : A Novel Abstraction of Development , 2007 .

[12]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[13]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[14]  Hector Zenil,et al.  Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning , 2018, ArXiv.

[15]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[16]  Risto Miikkulainen,et al.  Designing neural networks through neuroevolution , 2019, Nat. Mach. Intell..

[17]  Erkan Tanyildizi,et al.  A novel optimization method for solving constrained and unconstrained problems: modified Golden Sine Algorithm , 2018, Turkish J. Electr. Eng. Comput. Sci..

[18]  Joshua Evan Auerbach,et al.  Evolving complete robots with CPPN-NEAT: the utility of recurrent connections , 2011, GECCO '11.

[19]  Gonzalo Navarro,et al.  k2-Trees for Compact Web Graph Representation , 2009, SPIRE.

[20]  Chin-Teng Lin,et al.  A review of clustering techniques and developments , 2017, Neurocomputing.

[21]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[22]  Ling Huang,et al.  Fast approximate spectral clustering , 2009, KDD.

[23]  Anthony M. Zador,et al.  A critique of pure learning and what artificial neural networks can learn from animal brains , 2019, Nature Communications.

[24]  Chin-Hui Lee,et al.  Enhanced Adversarial Strategically-Timed Attacks Against Deep Reinforcement Learning , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[27]  Yen-Ching Chang,et al.  N-Dimension Golden Section Search: Its Variants and Limitations , 2009, 2009 2nd International Conference on Biomedical Engineering and Informatics.

[28]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.

[29]  Li Luo,et al.  Fibonacci multi-modal optimization algorithm in noisy environment , 2020, Appl. Soft Comput..

[30]  Jean-Baptiste Mouret,et al.  Does Aligning Phenotypic and Genotypic Modularity Improve the Evolution of Neural Networks? , 2016, GECCO.

[31]  Marcus Vinicius dos Santos,et al.  Toward an estimation of distribution algorithm for the evolution of artificial neural networks , 2010, C3S2E '10.

[32]  Christian Igel,et al.  Neuroevolution for reinforcement learning using evolution strategies , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[33]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[34]  Frédéric Gruau,et al.  Genetic synthesis of Boolean neural networks with a cell rewriting developmental process , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[35]  Alagan Anpalagan,et al.  Anti-Jamming Communications Using Spectrum Waterfall: A Deep Reinforcement Learning Approach , 2017, IEEE Communications Letters.

[36]  Mengjie Zhang,et al.  Blocky Net: A New NeuroEvolution Method , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[37]  Jianwei Huang,et al.  Complex Networks from Simple Rewrite Systems , 2012, ArXiv.

[38]  Ingrid Oliveros,et al.  Search for Global Maxima in Multimodal Functions by Applying Numerical Optimization Algorithms: A Comparison between Golden Section and Simulated Annealing , 2019, Comput..

[39]  Risto Miikkulainen,et al.  Automatic feature selection in neuroevolution , 2005, GECCO '05.

[40]  Armin Gruen,et al.  High-resolution 3D modelling and visualization of Mount Everest , 2002 .

[41]  Tor Lattimore,et al.  Behaviour Suite for Reinforcement Learning , 2019, ICLR.

[42]  S. Henikoff,et al.  Position-based sequence weights. , 1994, Journal of molecular biology.

[43]  Rui Wang,et al.  VINE: an open source interactive data visualization tool for neuroevolution , 2018, GECCO.

[44]  Jon S. Knapp,et al.  Natural Evolution Speciation for NEAT , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[45]  Percy Liang,et al.  Generalized Binary Search For Split-Neighborly Problems , 2018, AISTATS.

[46]  Andreas Krause,et al.  Approximate K-Means++ in Sublinear Time , 2016, AAAI.

[47]  Robert Wall Emerson,et al.  Causation and Pearson's Correlation Coefficient , 2015 .

[48]  Apinan Aurasopon.  An improved local search involving bee colony optimization using lambda iteration combined with a golden section search method to solve an economic dispatch problem , 2019 .