Evolving autonomous learning in cognitive networks

There are two common approaches for optimizing the performance of a machine: genetic algorithms and machine learning. A genetic algorithm is applied over many generations whereas machine learning works by applying feedback until the system meets a performance threshold. These methods have been previously combined, particularly in artificial neural networks using an external objective feedback mechanism. We adapt this approach to Markov Brains, which are evolvable networks of probabilistic and deterministic logic gates. Prior to this work MB could only adapt from one generation to the other, so we introduce feedback gates which augment their ability to learn during their lifetime. We show that Markov Brains can incorporate these feedback gates in such a way that they do not rely on an external objective feedback signal, but instead can generate internal feedback that is then used to learn. This results in a more biologically accurate model of the evolution of learning, which will enable us to study the interplay between evolution and learning and could be another step towards autonomously learning machines.

[1]  Alan D. Blair,et al.  Evolving Plastic Neural Networks for Online Learning: Review and Future Directions , 2012, Australasian Conference on Artificial Intelligence.

[2]  Robert T. Pennock,et al.  The evolutionary origin of complex features , 2003, Nature.

[3]  Randal S. Olson,et al.  Markov Brains: A Technical Introduction , 2017, ArXiv.

[4]  Mauro Santos,et al.  Phenotypic plasticity, the Baldwin effect, and the speeding up of evolution: the computational roots of an illusion. , 2014, Journal of theoretical biology.

[5]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[6]  Jürgen Schmidhuber,et al.  Evolving Modular Fast-Weight Networks for Control , 2005, ICANN.

[7]  Ghassan Kawas Kaleh,et al.  Joint parameter estimation and symbol detection for linear or nonlinear unknown channels , 1994, IEEE Trans. Commun..

[8]  Arend Hintze,et al.  Rewards, Risks, and Reaching the Right Strategy: Evolutionary Paths From Heuristics to Optimal Decisions , 2018, Evolutionary Behavioral Sciences.

[9]  M. Egas,et al.  How Adaptive Learning Affects Evolution: Reviewing Theory on the Baldwin Effect , 2011, Evolutionary Biology.

[10]  Sebastian Risi,et al.  Evolving Neural Turing Machines for Reward-based Learning , 2016, GECCO.

[11]  Christof Koch,et al.  The Minimal Complexity of Adapting Agents Increases with Fitness , 2012, ALIFE.

[12]  Wei Wu,et al.  Boundedness and convergence of batch back-propagation algorithm with penalty for feedforward neural networks , 2012, Neurocomputing.

[13]  Paul M. Baggenstoss.  A modified Baum-Welch algorithm for hidden Markov models with multiple observation spaces , 2001, IEEE Trans. Speech Audio Process..

[14]  W. Ma,et al.  Changing concepts of working memory , 2014, Nature Neuroscience.

[15]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[16]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[17]  Jean-Baptiste Mouret,et al.  On the relationships between synaptic plasticity and generative systems , 2011, GECCO '11.

[18]  S. Risi,et al.  Continual Learning through Evolvable Neural Turing Machines , 2016 .

[19]  Arend Hintze,et al.  Integrated Information Increases with Fitness in the Evolution of Animats , 2011, PLoS Comput. Biol..

[20]  Arend Hintze,et al.  Predator confusion is sufficient to evolve swarming behaviour , 2012, Journal of The Royal Society Interface.

[21]  H. Eichenbaum,et al.  Consolidation and Reconsolidation: Two Lives of Memories? , 2011, Neuron.

[22]  R. French.  Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[23]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[24]  Arend Hintze,et al.  Evolution of an artificial visual cortex for image recognition , 2013, ECAL.

[25]  E. Kandel,et al.  The Molecular and Systems Biology of Memory , 2014, Cell.

[26]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[27]  Jean-Baptiste Mouret,et al.  Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills , 2015, PLoS Comput. Biol..

[28]  Arend Hintze,et al.  Evolution of Autonomous Hierarchy Formation and Maintenance , 2014, ALIFE.

[29]  Arend Hintze,et al.  The Evolution of Representation in Simple Cognitive Networks , 2012, Neural Computation.

[30]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[31]  D. Stephens,et al.  Reliability, uncertainty, and costs in the evolution of animal learning , 2016, Current Opinion in Behavioral Sciences.

[32]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[33]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[34]  Sebastian Risi,et al.  A unified approach to evolving plasticity and neural geometry , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[35]  Arend Hintze,et al.  Evolution of Integrated Causal Structures in Animats Exposed to Environments of Increasing Complexity , 2014, PLoS Comput. Biol..

[36]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[37]  Arend Hintze,et al.  Information-theoretic neuro-correlates boost evolution of cognitive systems , 2015, Entropy.

[38]  A. Clark.  Being There: Putting Brain, Body, and World Together Again , 1996 .

[39]  L. Nadel,et al.  Update on Memory Systems and Processes , 2011, Neuropsychopharmacology.

[40]  J. Baldwin.  A New Factor in Evolution , 1896, The American Naturalist.

[41]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[42]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[43]  Dario Floreano,et al.  Evolution of Adaptive Synapses: Robots with Fast Adaptive Behavior in New Environments , 2001, Evolutionary Computation.

[44]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[45]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[46]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  L. Squire,et al.  The cognitive neuroscience of human memory since H.M. , 2011, Annual review of neuroscience.

[48]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[49]  Dario Floreano,et al.  Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios , 2008, ALIFE.

[50]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[51]  Ron Meir,et al.  The Effect of Learning on the Evolution of Asexual Populations , 1990, Complex Syst..

[52]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[53]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[54]  Lee Spector,et al.  Genetic Programming and Autoconstructive Evolution with the Push Programming Language , 2002, Genetic Programming and Evolvable Machines.

[55]  J. Gabrieli.  Cognitive neuroscience of human memory. , 1998, Annual review of psychology.

[56]  Risto Miikkulainen,et al.  Evolving adaptive neural networks with and without adaptive synapses , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[57]  Kenneth O. Stanley,et al.  Autonomous Evolution of Topographic Regularities in Artificial Neural Networks , 2010, Neural Computation.

[58]  Emil Juul Jacobsen,et al.  Evolving Neural Turing Machines , 2015 .

[59]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[60]  W. Abraham,et al.  Memory retention – the synaptic stability versus plasticity dilemma , 2005, Trends in Neurosciences.

[61]  Arend Hintze,et al.  Computational evolution of decision-making strategies , 2015, CogSci.

[62]  Christoph Adami,et al.  Distributed under Creative Commons Cc-by 4.0 the Evolution of Logic Circuits for the Purpose of Protein Contact Map Prediction , 2022 .