A Developmental Model for the Evolution of Artificial Neural Networks

We present a model of decentralized growth and development for artificial neural networks (ANNs), inspired by developmental biology and the physiology of nervous systems. In this model, each individual artificial neuron is an autonomous unit whose behavior is determined only by the genetic information it harbors and local concentrations of substrates. The chemicals and substrates, in turn, are modeled by a simple artificial chemistry. While the system is designed to allow for the evolution of complex networks, we demonstrate the power of the artificial chemistry by analyzing engineered (handwritten) genomes that lead to the growth of simple networks with behaviors known from physiology. To evolve more complex structures, a Java-based, platform-independent, asynchronous, distributed genetic algorithm (GA) has been implemented that allows users to participate in evolutionary experiments via the World Wide Web.

[1]  N. Pierce.  Origin of Species , 1914, Nature.

[2]  W. Pitts.  The linear theory of neuron networks: The dynamic problem , 1942 .

[3]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[4]  Aristid Lindenmayer,et al.  Mathematical Models for Cellular Interactions in Development , 1968 .

[5]  R. Britten,et al.  Gene regulation for higher cells: a theory. , 1969, Science.

[6]  R. Britten,et al.  Repetitive and Non-Repetitive DNA Sequences and a Speculation on the Origins of Evolutionary Novelty , 1971, The Quarterly Review of Biology.

[7]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  T. Poggio,et al.  Biophysics of Computation: Neurons, Synapses and Membranes , 1984 .

[11]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[12]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[13]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[14]  Robert M. Farber,et al.  How Neural Nets Work , 1987, NIPS.

[15]  Raoul Tawel.  Does the Neuron "Learn" Like the Synapse? , 1988, NIPS.

[16]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[17]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[18]  Richard Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[19]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[20]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[21]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[22]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[23]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[24]  Przemyslaw Prusinkiewicz,et al.  The Algorithmic Beauty of Plants , 1990, The Virtual Laboratory.

[25]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[26]  Stephen José Hanson,et al.  What connectionist models learn: Learning and representation in connectionist networks , 1990, Behavioral and Brain Sciences.

[27]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[28]  G. Mani,et al.  Learning by gradient descent in function space , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[29]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[30]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[31]  Charles E. Taylor,et al.  Artificial Life II , 1991 .

[32]  Bernd Fritzke.  Growing Cell Structures – a Self-organizing Network in k Dimensions , 1992 .

[33]  Frédéric Gruau,et al.  Genetic synthesis of Boolean neural networks with a cell rewriting developmental process , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[34]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[35]  T. Hattori.  Conceptual history of the nigrostriatal dopamine system , 1993, Neuroscience Research.

[36]  Frédéric Gruau,et al.  The cellular development of neural networks: the interaction of learning and evolution , 1993 .

[37]  Dragan Cvetkovic,et al.  Genetische Algorithmen , 1993, Künstliche Intell..

[38]  James M. Bower,et al.  The Book of GENESIS , 1994, Springer New York.

[39]  Frank Dellaert,et al.  Toward an evolvable model of development for autonomous agent synthesis , 1994 .

[40]  Karl Sims,et al.  Evolving 3D Morphology and Behavior by Competition , 1994, Artificial Life.

[41]  Andreas Zell,et al.  Simulation neuronaler Netze , 1994 .

[42]  Jari Vaario,et al.  Modelling adaptive self-organization , 1994 .

[43]  Christopher G. Langton,et al.  Artificial Life , 2019, Philosophical Posthumanism.

[44]  Moshe Sipper,et al.  An Introduction To Articial Life , 1995 .

[45]  G Theraulaz,et al.  Coordination in Distributed Building , 1995, Science.

[46]  Kurt W. Fleischer,et al.  A Multiple-Mechanism Developmental Model for Defining Self-Organizing Geometric Structures , 1995 .

[47]  Katsunori Shimohara,et al.  On Formation of Structures , 1995, ECAL.

[48]  Hiroaki Kitano.  Cell Differentiation and Neurogenesis in Evolutionary Large Scale Chaos , 1995, ECAL.

[49]  Olivier Michel.  An Artificial Life Approach for the Synthesis of Autonomous Agents , 1995, Artificial Evolution.

[50]  X. Beristain.  Essentials of neural science and behavior , 1996 .

[51]  O. Michel.  Experiences en neuroethologie artificielle evots : une methodologie evolutionniste appliquee en robotique mobile , 1996 .

[52]  J. G. Taylor,et al.  Concepts for Neural Networks: A Survey , 1997 .

[53]  C. Adami,et al.  Introduction To Artificial Life , 1997, IEEE Trans. Evol. Comput..

[54]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[55]  P. Jonas,et al.  Corelease of two fast neurotransmitters at a central synapse. , 1998, Science.

[56]  Hiroaki Kitano.  Building Complex Systems Using Developmental Process: An Engineering Approach , 1998, ICES.

[57]  Margaret R. Thomson,et al.  Vertebrate genome evolution and the zebrafish gene map , 1998, Nature Genetics.

[58]  Wolfgang Banzhaf,et al.  Artificial ChemistriesA Review , 2001, Artificial Life.

[59]  Stuart A. Kauffman,et al.  ORIGINS OF ORDER , 2019, Origins of Order.