Defining and simulating open-ended novelty: requirements, guidelines, and challenges

The open-endedness of a system is often defined as a continual production of novelty. Here we pin down this concept more fully by defining several types of novelty that a system may exhibit, classified as variation, innovation, and emergence. We then provide a meta-model for including levels of structure in a system’s model. From there, we define an architecture suitable for building simulations of open-ended novelty-generating systems and discuss how previously proposed systems fit into this framework. We discuss the design principles applicable to those systems and close with some challenges for the community.

[1]  Barry McMullin,et al.  Architectures for Self-reproduction: Abstractions, Realisations and a Research Program , 2012, ALIFE.

[2]  Wolfgang Banzhaf,et al.  Meta-Evolution in Graph GP , 1999, EuroGP.

[3]  Timothy J. Taylor,et al.  From artificial evolution to artificial life , 1999 .

[4]  Patrick Suppes.  A comparison of the meaning and uses of models in mathematics and the empirical sciences , 1960 .

[5]  P. Anderson.  More is different. , 1972, Science.

[6]  Noam Chomsky,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[7]  Andrew N. Pargellis,et al.  Digital Life Behavior in the Amoeba World , 2000, Artificial Life.

[8]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[9]  Steen Rasmussen,et al.  The coreworld: emergence and evolution of cooperative structures in a computational chemistry , 1990 .

[10]  Carlo C. Maley,et al.  Four steps toward open-ended evolution , 1999 .

[11]  Tomonori Hasegawa,et al.  On the evolution of genotype-phenotype mapping: exploring viability in the Avida articial life system , 2015 .

[12]  R. Lenski,et al.  Historical contingency and the evolution of a key innovation in an experimental population of Escherichia coli , 2008 .

[13]  K. Ruiz-Mirazo,et al.  A Universal Definition of Life: Autonomy and Open-Ended Evolution , 2004, Origins of life and evolution of the biosphere.

[14]  Carl F. Craver.  The Mosaic Unity of Neuroscience , 2007 .

[15]  Eckehard Olbrich,et al.  Comparison between Different Methods of Level Identification , 2014, Adv. Complex Syst..

[16]  Lee Spector,et al.  Towards Practical Autoconstructive Evolution: Self-Evolution of Problem-Solving Genetic Programming Systems , 2011 .

[17]  T. Ray.  Evolution , Ecology and Optimization of Digital Organisms , 1992 .

[18]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[19]  Philippe Huneman,et al.  Determinism, predictability and open-ended evolution: lessons from computational emergence , 2010, Synthese.

[20]  Alvaro Moreno,et al.  Autonomy in evolution: from minimal to complex life , 2012, Synthese.

[21]  Mark A. Bedau,et al.  Can Unrealistic Computer Models Illuminate Theoretical Biology , 1999 .

[22]  D. Hilbert.  Mathematical Problems , 2019, Mathematics: People · Problems · Results.

[23]  C. H. Waddington.  Paradigm for an Evolutionary Process , 2008 .

[24]  W. Wimsatt.  The Ontology of Complex Systems: Levels of Organization, Perspectives, and Causal Thickets , 1994 .

[25]  Kenneth O. Stanley,et al.  Identifying Necessary Conditions for Open-Ended Evolution through the Artificial Life World of Chromaria , 2014, ALIFE.

[26]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[27]  Wolfgang Banzhaf,et al.  Artificial ChemistriesA Review , 2001, Artificial Life.

[28]  Peter J. Bentley,et al.  Evolving Fractal Gene Regulatory Networks for Robot Control , 2003, ECAL.

[29]  Russell K. Standish,et al.  Open-Ended Artificial Evolution , 2002, Int. J. Comput. Intell. Appl..

[30]  R. Soare.  Recursively enumerable sets and degrees , 1987 .

[31]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[32]  Eörs Szathmáry,et al.  The Major Transitions in Evolution , 1997 .

[33]  J. Hartmanis,et al.  On the Computational Complexity of Algorithms , 1965 .

[34]  Bruce Edmonds,et al.  Meta-Genetic Programming: Co-evolving the Operators of Variation , 2001 .

[35]  Susan Stepney,et al.  Simulation as a Scientific Instrument , 2012 .

[36]  E. Schrödinger.  What is life? : the physical aspect of the living cell , 1944 .

[37]  N. Moran,et al.  Extreme genome reduction in symbiotic bacteria , 2011, Nature Reviews Microbiology.

[38]  SAGAInman HarveyCSRP.  Species Adaptation Genetic Algorithms: A Basis for a Continuing SAGA , 1992 .

[39]  Mark A. Bedau,et al.  Towards a comparison of evolutionary creativity in biological and cultural evolution , 2002 .

[40]  M. Bedau.  Measurement of Evolutionary Activity, Teleology, and Life , 1996 .

[41]  Wolfgang Banzhaf,et al.  Basic Concepts of Artificial Chemistries , 2015 .

[42]  Arthur Koestler,et al.  Beyond Atomism and Holism—the Concept of the Holon , 2015 .

[43]  Jose D. Fernández,et al.  Emergent Diversity in an Open-Ended Evolving Virtual Community , 2012, Artificial Life.

[44]  Richard E. Lenski,et al.  Epistasis and Allele Specificity in the Emergence of a Stable Polymorphism in Escherichia coli , 2014, Science.

[45]  Anneke Kleppe,et al.  MDA explained - the Model Driven Architecture: practice and promise , 2003, Addison Wesley object technology series.

[46]  M. Sachs.  The Open Universe: An Argument for Indeterminism , 1985 .

[47]  Conor Ryan,et al.  Long-term evolutionary dynamics in heterogeneous cellular automata , 2013, GECCO '13.

[48]  Susan Stepney,et al.  CoSMoS special issue editorial , 2015, Natural Computing.

[49]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[50]  Marco Tomassini,et al.  A phylogenetic, ontogenetic, and epigenetic view of bio-inspired hardware systems , 1997, IEEE Trans. Evol. Comput..

[51]  Bérénice Batut,et al.  Reductive genome evolution at both ends of the bacterial population size spectrum , 2014, Nature Reviews Microbiology.

[52]  Francis Heylighen,et al.  A brain in a vat cannot break out: why the singularity must be extended, embedded and embodied , 2012 .

[53]  R. Harré.  Models in Science. , 1978 .

[54]  Cailin O'Connor,et al.  Simulation and Similarity: Using Models to Understand the World , 2016 .

[55]  Kenneth O. Stanley,et al.  Beyond Open-endedness: Quantifying Impressiveness , 2012, ALIFE.

[56]  Ernesto Costa,et al.  Step evolution: Improving the performance of open-ended evolution simulations , 2013, 2013 IEEE Symposium on Artificial Life (ALife).

[57]  Robert I. Soare,et al.  Recursively enumerable sets and degrees - a study of computable functions and computability generated sets , 1987, Perspectives in mathematical logic.

[58]  Thomas S. Ray,et al.  An Approach to the Synthesis of Life , 1991 .

[59]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1998 .

[60]  Elwyn R. Berlekamp,et al.  Winning Ways for Your Mathematical Plays, Volume 2 , 2003 .

[61]  Carol E. Cleland,et al.  The Nature of Life , 1996 .

[62]  Stefano Nolfi,et al.  Toward open-ended evolutionary robotics: evolving elementary robotic units able to self-assemble and self-reproduce , 2004, Connect. Sci..

[63]  F. J. Odling-Smee,et al.  Niche Construction: The Neglected Process in Evolution , 2003 .

[64]  C. Titus Brown,et al.  Evolutionary Learning in the 2D Artificial Life System "Avida" , 1994, adap-org/9405003.

[65]  E. Berlekamp,et al.  Winning Ways for Your Mathematical Plays , 1983 .

[66]  S. Hartmann,et al.  Models in Science , 2006 .

[67]  Kiel Moe.  Science in the Age of Computer Simulation , 2015 .

[68]  K. Ruiz-Mirazo,et al.  Enabling conditions for ‘open-ended evolution’ , 2007 .

[69]  David Sloan Wilson,et al.  BIOLOGICAL COMMUNITIES AS FUNCTIONALLY ORGANIZED UNITS , 1997 .

[70]  Peer Vries,et al.  Why the West rules – for now: the patterns of history and what they reveal about the future , 2012 .

[71]  Alastair Channon.  Passing the ALife Test: Activity Statistics Classify Evolution in Geb as Unbounded , 2001, ECAL.

[72]  Eric Winsberg,et al.  Science in the Age of Computer Simulation , 2010 .

[73]  J. Foster.  Computational genetics: Evolutionary computation , 2001, Nature Reviews Genetics.

[74]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[75]  Kunihiko Kaneko.  Chaos as a source of complexity and diversity in evolution , 1993 .

[76]  George Kampis,et al.  Evolvability of Natural and Artificial Systems , 2011, FET.

[77]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[78]  Susan Stepney,et al.  Energy as a driver of diversity in open-ended evolution , 2011, ECAL.

[79]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[80]  Susan Stepney,et al.  When does a physical system compute? , 2013, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[81]  Lee Spector,et al.  Genetic Programming and Autoconstructive Evolution with the Push Programming Language , 2002, Genetic Programming and Evolvable Machines.

[82]  N. Packard,et al.  Transitions from Nonliving to Living Matter , 2004, Science.

[83]  P. Suppes.  A comparison of the meaning and uses of models in mathematics and the empirical sciences , 1960, Synthese.

[84]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[85]  Susan Stepney,et al.  Reflecting on open-ended evolution , 2011, ECAL.

[86]  Emil L. Post.  Recursively enumerable sets of positive integers and their decision problems , 1944 .

[87]  E. Szathmáry.  Toward major evolutionary transitions theory 2.0 , 2015, Proceedings of the National Academy of Sciences.

[88]  L. Buss,et al.  The evolution of individuality , 1987 .

[89]  Anikó Ekárt,et al.  Genetic algorithms in computer aided design , 2003, Comput. Aided Des..

[90]  Susan Stepney,et al.  Specification of the Stringmol chemical programming language version 0 . 2 Technical Report Number YCS-2010-458 , 2010 .

[91]  M. Bedau.  Can Biological Teleology Be Naturalized , 1991 .

[92]  Susan Stepney,et al.  The CoSMoS Process, Version 0.1: A Process for the Modelling and Simulation of Complex Systems , 2010 .

[93]  Declan Baugh,et al.  Implementing von Neumann’s architecture for machine self reproduction within the tierra artificial life platform to investigate evolvable genotype-phenotype mappings , 2015 .

[94]  Master Gardener,et al.  Mathematical games: the fantastic combinations of john conway's new solitaire game "life , 1970 .

[95]  Paul W. Rendell,et al.  Turing Universality of the Game of Life , 2002, Collision-Based Computing.

[96]  A. Church.  An Unsolvable Problem of Elementary Number Theory , 1936 .

[97]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[98]  Susan Stepney,et al.  Maximizing the Adjacent Possible in Automata Chemistries , 2016, Artificial Life.

[99]  David Lane,et al.  HIERARCHY, COMPLEXITY, SOCIETY , 2006 .

[100]  George F R Ellis,et al.  Top-down causation and emergence: some comments on mechanisms , 2012, Interface Focus.

[101]  Doron Lancet,et al.  Is There an Optimal Level of Open-Endedness in Prebiotic Evolution? , 2012, Origins of Life and Evolution of Biospheres.

[102]  John S. McCaskill,et al.  Open Problems in Artificial Life , 2000, Artificial Life.

[103]  Nils Aall Barricelli,et al.  Numerical testing of evolution theories , 1963 .

[104]  Klaudia Frankfurter.  Computers And Intractability A Guide To The Theory Of Np Completeness , 2016 .

[105]  Alastair Channon,et al.  Improving and still passing the ALife test: component-normalised activity statistics classify evolution in geb as unbounded , 2002 .

[106]  R. Rosen.  Life Itself: A Comprehensive Inquiry Into the Nature, Origin, and Fabrication of Life , 1991 .

[107]  B. Rensch,et al.  Evolution above the species level , 1959 .

[108]  G. Beslon,et al.  A long-term evolutionary pressure on the amount of noncoding DNA. , 2007, Molecular biology and evolution.

[109]  Jeffrey E. Barrick,et al.  Genome evolution and adaptation in a long-term experiment with Escherichia coli , 2009, Nature.

[110]  Milton H. Saier,et al.  A stochastic automaton shows how enzyme assemblies may contribute to metabolic efficiency , 2007, BMC Systems Biology.

[111]  Chrystopher L. Nehaniv,et al.  What Software Evolution and Biological Evolution Don't Have in Common , 2006, 2006 Second International IEEE Workshop on Software Evolvability (SE'06).

[112]  Susan Stepney,et al.  CoSMoS process , models , and metamodels , 2011 .

[113]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[114]  N. Packard,et al.  A classification of long-term evolutionary dynamics , 1998 .

[115]  BanzhafWolfgang,et al.  Artificial chemistriesa review , 2001 .

[116]  Tim J. Hutton,et al.  Evolvable Self-Replicating Molecules in an Artificial Chemistry , 2002, Artificial Life.

[117]  E. Winfree,et al.  Robust self-replication of combinatorial information via crystal growth and scission , 2012, Proceedings of the National Academy of Sciences.

[118]  K. Holsinger.  The neutral theory of molecular evolution , 2004 .

[119]  Paul Humphreys,et al.  Extending Ourselves: Computational Science, Empiricism, and Scientific Method , 2004 .

[120]  Robert I. Damper,et al.  Towards the evolutionary emergence of increasingly complex advantageous behaviours , 2000, Int. J. Syst. Sci..

[121]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[122]  Jeffrey E. Barrick,et al.  Genome dynamics during experimental evolution , 2013, Nature Reviews Genetics.

[123]  Simon J. Hickinbotham,et al.  A quantitative measure of non-neutral evolutionary activity for systems that exhibit intrinsic fitness , 2012, ALIFE.

[124]  Nicholas Mark Gotts.  Ramifying Feedback Networks, Cross-Scale Interactions, and Emergent Quasi Individuals in Conway's Game of Life , 2009, Artificial Life.

[125]  Carl F. Craver.  A Field‐Guide to Levels , 2007 .