HyperENTM: Evolving Scalable Neural Turing Machines through HyperNEAT

Recent developments within memory-augmented neural networks have solved sequential problems requiring long-term memory, which are intractable for traditional neural networks. However, current approaches still struggle to scale to large memory sizes and sequence lengths. In this paper we show how access to memory can be encoded geometrically through a HyperNEAT-based Neural Turing Machine (HyperENTM). We demonstrate that using the indirect HyperNEAT encoding allows for training on small memory vectors in a bit-vector copy task and then applying the knowledge gained from such training to speed up training on larger size memory vectors. Additionally, we demonstrate that in some instances, networks trained to copy bit-vectors of size 9 can be scaled to sizes of 1,000 without further training. While the task in this paper is simple, these results could open up the problems amendable to networks with external memories to problems with larger memory vectors and theoretically unbounded memory sizes.

[1]  O. Sporns.  Network Analysis , Complexity , and Brain Function , 2002 .

[2]  Olaf Sporns,et al.  Networks analysis, complexity, and brain function , 2002 .

[3]  Quoc V. Le,et al.  HyperNetworks , 2016, ICLR.

[4]  Sebastian Risi,et al.  Continual and One-Shot Learning Through Neural Networks with Dynamic External Memory , 2017, EvoApplications.

[5]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[6]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[7]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[8]  Josh Bongard,et al.  Evolving modular genetic regulatory networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[9]  Kenneth O. Stanley,et al.  Indirect Encoding of Neural Networks for Scalable Go , 2010, PPSN.

[10]  Kenneth O. Stanley,et al.  Autonomous Evolution of Topographic Regularities in Artificial Neural Networks , 2010, Neural Computation.

[11]  Sebastian Risi,et al.  An Enhanced Hypercube-Based Encoding for Evolving the Placement, Density, and Connectivity of Neurons , 2012, Artificial Life.

[12]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[13]  Risto Miikkulainen,et al.  A Taxonomy for Artificial Embryogeny , 2003, Artificial Life.

[14]  Sebastian Risi,et al.  A unified approach to evolving plasticity and neural geometry , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[15]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[16]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[17]  Kenneth O. Stanley,et al.  Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning , 2017, ArXiv.

[19]  Kenneth O. Stanley,et al.  Constraining connectivity to encourage modularity in HyperNEAT , 2011, GECCO '11.

[20]  Emil Juul Jacobsen,et al.  Evolving Neural Turing Machines , 2015 .

[21]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[22]  Charles Ofria,et al.  Evolving coordinated quadruped gaits with the HyperNEAT generative encoding , 2009, 2009 IEEE Congress on Evolutionary Computation.

[23]  Sebastian Risi,et al.  Evolving Neural Turing Machines for Reward-based Learning , 2016, GECCO.

[24]  Kenneth O. Stanley,et al.  On the Performance of Indirect Encoding Across the Continuum of Regularity , 2011, IEEE Transactions on Evolutionary Computation.

[25]  Jimmy Secretan,et al.  Picbreeder: evolving pictures collaboratively online , 2008, CHI.

[26]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[27]  Kenneth O. Stanley,et al.  Evolving a Single Scalable Controller for an Octopus Arm with a Variable Number of Segments , 2010, PPSN.

[28]  Jordan B. Pollack,et al.  Creating High-Level Components with a Generative Representation for Body-Brain Evolution , 2002, Artificial Life.

[29]  Sebastian Risi,et al.  1D Printing of Recyclable Robots , 2017, IEEE Robotics and Automation Letters.

[30]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[31]  Joel Lehman,et al.  Evolving policy geometry for scalable multiagent learning , 2010, AAMAS.

[32]  Kenneth O. Stanley,et al.  Compositional Pattern Producing Networks : A Novel Abstraction of Development , 2007 .

[33]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.