Breeding novel solutions in the brain: a model of Darwinian neurodynamics

Background: The fact that surplus connections and neurons are pruned during development is well established. We complement this selectionist picture by a proof-of-principle model of evolutionary search in the brain, that accounts for new variations in theory space. We present a model for Darwinian evolutionary search for candidate solutions in the brain. Methods: We combine known components of the brain – recurrent neural networks (acting as attractors), the action selection loop and implicit working memory – to provide the appropriate Darwinian architecture. We employ a population of attractor networks with palimpsest memory. The action selection loop is employed with winners-share-all dynamics to select for candidate solutions that are transiently stored in implicit working memory. Results: We document two processes: selection of stored solutions and evolutionary search for novel solutions. During the replication of candidate solutions attractor networks occasionally produce recombinant patterns, increasing variation on which selection can act. Combinatorial search acts on multiplying units (activity patterns) with hereditary variation and novel variants appear due to (i) noisy recall of patterns from the attractor networks, (ii) noise during transmission of candidate solutions as messages between networks, and, (iii) spontaneously generated, untrained patterns in spurious attractors. Conclusions: Attractor dynamics of recurrent neural networks can be used to model Darwinian search. The proposed architecture can be used for fast search among stored solutions (by selection) and for evolutionary search when novel candidate solutions are generated in successive iterations. Since all the suggested components are present in advanced nervous systems, we hypothesize that the brain could implement a truly evolutionary combinatorial search system, capable of generating novel variants.

[1]  Eörs Szathmáry,et al.  The Neuronal Replicator Hypothesis , 2010, Neural Computation.

[2]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[3]  D. Muller,et al.  Structural plasticity: mechanisms and contribution to developmental psychiatric disorders , 2014, Front. Neuroanat..

[4]  Tomoki Fukai,et al.  A Simple Neural Network Exhibiting Selective Activation of Neuronal Ensembles: From Winner-Take-All to Winners-Share-All , 1997, Neural Computation.

[5]  Michael P. Kilgard,et al.  Cortical Map Plasticity Improves Learning but Is Not Necessary for Improved Performance , 2011, Neuron.

[6]  L. Nadel,et al.  Memory consolidation, retrograde amnesia and the hippocampal complex , 1997, Current Opinion in Neurobiology.

[7]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[8]  J. Changeux,et al.  A theory of the epigenesis of neuronal networks by selective stabilization of synapses. , 1973, Proceedings of the National Academy of Sciences of the United States of America.

[9]  K. Oberauer Access to information in working memory: exploring the focus of attention. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[10]  T. Sejnowski,et al.  Irresistible environment meets immovable neurons , 1997, Behavioral and Brain Sciences.

[11]  Lee Spector,et al.  Genetic Programming for Reward Function Search , 2010, IEEE Transactions on Autonomous Mental Development.

[12]  M. Shanahan A spiking neuron model of cortical broadcast and competition , 2008, Consciousness and Cognition.

[13]  W. Calvin The brain as a Darwin Machine , 1987, Nature.

[14]  Michael P. Kilgard,et al.  Harnessing plasticity to understand learning and treat disease , 2012, Trends in Neurosciences.

[15]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[16]  V. Mountcastle Modality and topographic properties of single neurons of cat's somatic sensory cortex. , 1957, Journal of neurophysiology.

[17]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[18]  Bettina Appel,et al.  Thirty-five years of research into ribozymes and nucleic acid catalysis: where do we stand today? , 2016, F1000Research.

[19]  H. P. de Vladar,et al.  Neuronal boost to evolutionary dynamics , 2015, Interface Focus.

[20]  Eörs Szathmáry,et al.  Natural Selection in the Brain , 2010 .

[21]  A. E. Hirsh,et al.  The application of statistical physics to evolutionary biology. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Eörs Szathmáry,et al.  Selectionist and Evolutionary Approaches to Brain Function: A Critical Appraisal , 2012, Front. Comput. Neurosci..

[23]  E T Rolls,et al.  Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network , 1992, Hippocampus.

[24]  Y. Iwasa,et al.  Free fitness that always increases in evolution. , 1988, Journal of theoretical biology.

[25]  J. Silvanto,et al.  How is working memory content consciously experienced? The ‘conscious copy’ model of WM introspection , 2015, Neuroscience & Biobehavioral Reviews.

[26]  Hagai Bergman,et al.  Stepping out of the box: information processing in the neural networks of the basal ganglia , 2001, Current Opinion in Neurobiology.

[27]  John O. Campbell Universal Darwinism As a Process of Bayesian Inference , 2016, Front. Syst. Neurosci..

[28]  P. Husbands,et al.  Evolvable Neuronal Paths: A Novel Basis for Information and Search in the Brain , 2011, PloS one.

[29]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[30]  H. P. de Vladar,et al.  The statistical mechanics of a polygenic character under stabilizing selection, mutation and drift , 2010, Journal of The Royal Society Interface.

[31]  H. Wigström,et al.  Onset Characteristics of Long‐Term Potentiation in the Guinea‐Pig Hippocampal CA1 Region in Vitro , 1989, The European journal of neuroscience.

[32]  R. Yuste,et al.  Imprinting and recalling cortical ensembles , 2016, Science.

[33]  Karl J. Friston,et al.  The Functional Anatomy of Time: What and When in the Brain , 2016, Trends in Cognitive Sciences.

[34]  S. Herculano‐Houzel,et al.  Changing numbers of neuronal and non-neuronal cells underlie postnatal brain growth in the rat , 2009, Proceedings of the National Academy of Sciences.

[35]  P. Caroni,et al.  Structural plasticity upon learning: regulation and functions , 2012, Nature Reviews Neuroscience.

[36]  Noah D. Goodman,et al.  Theory learning as stochastic search in the language of thought , 2012 .

[37]  Lynn Nadel,et al.  Commentary — reconsolidation: Memory traces revisited , 2000, Nature Reviews Neuroscience.

[38]  Bartlett W. Mel,et al.  Cortical rewiring and information storage , 2004, Nature.

[39]  Juha Silvanto,et al.  Reappraising the relationship between working memory and conscious awareness , 2014, Trends in Cognitive Sciences.

[40]  Richard Dawkins,et al.  The Evolution of Evolvability , 1987, ALIFE.

[41]  L. Bianchi,et al.  The DEG/ENaC cation channel protein UNC-8 drives activity-dependent synapse removal in remodeling GABAergic neurons , 2016, eLife.

[42]  F. Crépel,et al.  Use‐dependent changes in synaptic efficacy in rat prefrontal neurons in vitro. , 1990, The Journal of physiology.

[43]  Haim Sompolinsky,et al.  Computational neuroscience: beyond the local circuit , 2014, Current Opinion in Neurobiology.

[44]  Abigail Morrison,et al.  Corticostriatal circuit mechanisms of value-based action selection: Implementation of reinforcement learning algorithms and beyond , 2016, Behavioural Brain Research.

[45]  S Dehaene,et al.  A neuronal model of a global workspace in effortful cognitive tasks. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[46]  P. Adams Hebb and Darwin. , 1998, Journal of theoretical biology.

[47]  Chrisantha Fernando,et al.  Copying and Evolution of Neuronal Topology , 2008, PloS one.

[48]  Thomas Jansen,et al.  A building-block royal road where crossover is provably essential , 2007, GECCO '07.

[49]  C. Shalizi Dynamics of Bayesian Updating with Dependent Data and Misspecified Models , 2009, 0901.1342.

[50]  J. Changeux Neuronal man : the biology of mind , 1985 .

[51]  P. Rakić,et al.  Elimination of neurons from the rhesus monkey's lateral geniculate nucleus during development , 1988, The Journal of comparative neurology.

[52]  D. O'Leary Development of connectional diversity and specificity in the mammalian brain by the pruning of collateral projections , 1992, Current Opinion in Neurobiology.

[53]  M. Kimura The Neutral Theory of Molecular Evolution: Introduction , 1983 .

[54]  A. Kolodkin,et al.  Mechanisms and molecules of neuronal wiring: a primer. , 2011, Cold Spring Harbor perspectives in biology.

[55]  M. Eigen Selforganization of matter and the evolution of biological macromolecules , 1971, Naturwissenschaften.

[56]  B. Baars,et al.  The Timing of the Cognitive Cycle , 2011, PloS one.

[57]  G. Xi,et al.  Human Cortex Development: Estimates of Neuronal Numbers Indicate Major Loss Late During Gestation , 1996, Journal of neuropathology and experimental neurology.

[58]  K. Svoboda,et al.  Experience-dependent structural synaptic plasticity in the mammalian brain , 2009, Nature Reviews Neuroscience.

[59]  A. Aertsen,et al.  Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding , 2010, Nature Reviews Neuroscience.

[60]  Florentin Wörgötter,et al.  Time scales of memory, learning, and plasticity , 2012, Biological Cybernetics.