Compositional Pattern Producing Networks : A Novel Abstraction of Development

Natural DNA can encode complexity on an enormous scale. Researchers are attempting to achieve the same representational efficiency in computers by implementing developmental encodings, i.e. encodings that map the genotype to the phenotype through a process of growth from a small starting point to a mature form. A major challenge in in this effort is to find the right level of abstraction of biological development to capture its essential properties without introducing unnecessary inefficiencies. In this paper, a novel abstraction of natural development, called Compositional Pattern Producing Networks (CPPNs), is proposed. Unlike currently accepted abstractions such as iterative rewrite systems and cellular growth simulations, CPPNs map to the phenotype without local interaction, that is, each individual component of the phenotype is determined independently of every other component. Results produced with CPPNs through interactive evolution of two-dimensional images show that such an encoding can nevertheless produce structural motifs often attributed to more conventional developmental abstractions, suggesting that local interaction may not be essential to the desirable properties of natural encoding in the way that is usually assumed.

[1]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[2]  A. Lindenmayer Mathematical models for cellular interactions in development. II. Simple and branching filaments with two-sided inputs. , 1968, Journal of theoretical biology.

[3]  A. Lindenmayer Mathematical models for cellular interactions in development. I. Filaments with one-sided inputs. , 1968, Journal of theoretical biology.

[4]  Jeffrey W. Roberts,et al.  遺伝子の分子生物学 = Molecular biology of the gene , 1970 .

[5]  Aristid Lindenmayer,et al.  Adding Continuous Components to L-Systems , 1974, L Systems.

[6]  H. Meinhardt Models of biological pattern formation , 1982 .

[7]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[8]  R. Dawkins The Blind Watchmaker , 1986 .

[9]  J E Darnell,et al.  Speculations on the early course of evolution. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[11]  Przemyslaw Prusinkiewicz,et al.  The Algorithmic Beauty of Plants , 1990, The Virtual Laboratory.

[12]  A. M. Turing,et al.  The chemical basis of morphogenesis , 1952, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[13]  Karl Sims,et al.  Artificial evolution for computer graphics , 1991, SIGGRAPH.

[14]  D. Parisi,et al.  Growing neural networks , 1991 .

[15]  David H. Sharp,et al.  A connectionist model of development. , 1991, Journal of theoretical biology.

[16]  Joshua R. Smith Designing Biomorphs with an Interactive Genetic Algorithm , 1991, ICGA.

[17]  Stephen Todd,et al.  Evolutionary Art and Computers , 1992 .

[18]  T. Wiesel,et al.  Receptive field dynamics in adult primary visual cortex , 1992, Nature.

[19]  A. Cangelosi,et al.  Cell division and migration in a 'genotype' for neural networks (Cell division and migration in neural networks) , 1993 .

[20]  Byoung-Tak Zhang,et al.  Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor , 1993, Complex Syst..

[21]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[22]  Jon McCormack,et al.  Interactive evolution of L-System grammars for computer graphics modelling , 1993 .

[23]  David G. Green,et al.  Complex Systems: From Biology to Computation , 1993 .

[24]  Lee Altenberg,et al.  Evolving better representations through selective genome growth , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[25]  Shumeet Baluja,et al.  Towards Automated Artificial Evolution for Computer-generated Images , 1993, Connect. Sci..

[26]  Karl Sims,et al.  Evolving 3d morphology and behavior by competition , 1994 .

[27]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[28]  N. Jakobi Harnessing Morphogenesis Csrp 423 , 1995 .

[29]  D. Curtis,et al.  nanos is an evolutionarily conserved organizer of anterior-posterior polarity. , 1995, Development.

[30]  David B. Fogel,et al.  Evolving Neural Control Systems , 1995, IEEE Expert.

[31]  Robert G. Reynolds,et al.  Morphogenic Evolutionary Computations: Introduction, Issues and Examples , 1995 .

[32]  S. Carroll Homeotic genes and the evolution of arthropods and chordates , 1995, Nature.

[33]  Peter J. Angeline,et al.  Morphogenic Evolutionary Computations: Introduction, Issues and Example , 1995, Evolutionary Programming.

[34]  F. Dellaert TOWARD A BIOLOGICALLY DEFENSIBLE MODEL OF DEVELOPMENT , 1995 .

[35]  Maja J. Matarić,et al.  A Developmental Model for the Evolution of Complete Autonomous Agents , 1996 .

[36]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[37]  Phil Husbands,et al.  Two Applications of Genetic Algorithms to Component Design , 1996, Evolutionary Computing, AISB Workshop.

[38]  R. Raff Understanding Evolution: The Next Step. (Book Reviews: The Shape of Life. Genes, Development, and the Evolution of Animal Form.) , 1996 .

[39]  Riccardo,et al.  Evolution of the Topology and the Weightsof Neural Networks using GeneticProgramming with a Dual RepresentationJo , 1997 .

[40]  David W. Opitz,et al.  Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies , 1997, J. Artif. Intell. Res..

[41]  Stewart W. Wilson,et al.  From Animals to Animats 5. Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior , 1997 .

[42]  P. Lijnzaad,et al.  A physical map of 30,000 human genes. , 1998, Science.

[43]  T. Schnier,et al.  Evolved Representations and Their Use in Computational Creativity , 1998 .

[44]  Kunihiko Kaneko,et al.  Emergence of Multicellular Organisms with Dynamic Differentiation and Spatial Pattern , 1997, Artificial Life.

[45]  Xin Yao,et al.  Towards designing artificial neural networks by evolution , 1998 .

[46]  Y L Wang,et al.  Zebrafish hox clusters and vertebrate genome evolution. , 1998, Science.

[47]  Andrew P. Martin Increasing Genomic Complexity by Gene Duplication and the Origin of Vertebrates , 1999, The American Naturalist.

[48]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[49]  A. Force,et al.  Preservation of duplicate genes by complementary, degenerative mutations. , 1999, Genetics.

[50]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[51]  Peter J. Bentley,et al.  Three Ways to Grow Designs: A Comparison of Evolved Embryogenies for a Design Problem , 1999 .

[52]  G. Greenfield Evolving Expressions and Art by Choice , 2000, Leonardo.

[53]  Christoph Adami,et al.  A Developmental Model for the Evolution of Artificial Neural Networks , 2000, Artificial Life.

[54]  Julian Francis Miller,et al.  Cartesian genetic programming , 2000, GECCO '10.

[55]  Christopher G. Langton,et al.  Artificial Life III , 2000 .

[56]  Sung-Bae Cho,et al.  A 3D modeling system for creative design , 2001, Proceedings 15th International Conference on Information Networking.

[57]  S. Lall,et al.  Conservation and divergence in molecular mechanisms of axis formation. , 2001, Annual review of genetics.

[58]  Maciej Komosinski,et al.  Comparison of Different Genotype Encodings for Simulated Three-Dimensional Agents , 2002, Artificial Life.

[59]  Gregory S. Hornby,et al.  Body-brain co-evolution using L-systems as a generative encoding , 2001 .

[60]  Gregory S. Hornby,et al.  The advantages of generative grammatical encodings for physical design , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[61]  R. Pfeifer,et al.  Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny , 2001 .

[62]  Hideyuki Takagi,et al.  Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation , 2001, Proc. IEEE.

[63]  G. Ragsdell Systems , 2002, Economics of Visual Art.

[64]  Jordan B. Pollack,et al.  Creating High-Level Components with a Generative Representation for Body-Brain Evolution , 2002, Artificial Life.

[65]  Josh Bongard,et al.  Evolving modular genetic regulatory networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[66]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[67]  Ian C. Parmee,et al.  Improving problem definition through interactive evolutionary computation , 2002, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[68]  R. Beer,et al.  20 – A developmental model for the evolution of complete autonomous agents , 2003 .

[69]  Risto Miikkulainen,et al.  A Taxonomy for Artificial Embryogeny , 2003, Artificial Life.

[70]  Wolfgang Banzhaf,et al.  Advances in Artificial Life , 2003, Lecture Notes in Computer Science.

[71]  Julian Francis Miller,et al.  Evolving Developmental Programs for Adaptation, Morphogenesis, and Self-Repair , 2003, ECAL.

[72]  N. Jakobi 21 – Harnessing morphogenesis , 2003 .

[73]  Peter J. Bentley,et al.  Exploring Reaction-Diffusion and Pattern Formation , 2003 .

[74]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[75]  Kenneth O. Stanley and Joseph Reisinger and Risto Miikkulainen,et al.  Exploiting Morphological Conventions for Genetic Reuse , 2004 .

[76]  D. Federici Using Embryonic Stages to increase the evolvability of development , 2004 .

[77]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[78]  Daniel Roggen,et al.  Multi-cellular Development: Is There Scalability and Robustness to Gain? , 2004, PPSN.

[79]  S. Vijayakumar,et al.  Evolving a neurocontroller through a process of embryogeny , 2004 .

[80]  Julian Francis Miller,et al.  Evolving a Self-Repairing, Self-Regulating, French Flag Organism , 2004, GECCO.

[81]  Risto Miikkulainen,et al.  Towards an empirical measure of evolvability , 2005, GECCO '05.

[82]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[83]  Tim Andersen,et al.  A biologically derived approach to tissue modeling. , 2005, Studies in health technology and informatics.

[84]  Nicholas J. Radcliffe,et al.  Genetic set recombination and its application to neural network topology optimisation , 1993, Neural Computing & Applications.

[85]  T. Metzinger The evolution of evolvability Ruth Garret Millikan Varieties of Meaning: The 2002 Jean Nicod Lectures , 2005, Trends in Cognitive Sciences.

[86]  D. McCandless Fundamental neuroscience , 1997, Metabolic Brain Disease.

[87]  Peter Eggenberger-Hotz Evolving Morphologies of Simulated 3d Organisms Based on Differential Gene Expression , 2007 .

[88]  A. Barr,et al.  A Simulation Testbed for the Study of Multicellular Development: The Multiple Mechanisms of Morphogenesis , 2008 .

[89]  Stuart A. Kauffman,et al.  ORIGINS OF ORDER , 2019, Origins of Order.