Quality Diversity: A New Frontier for Evolutionary Computation

While evolutionary computation and evolutionary robotics take inspiration from nature, they have long focused mainly on problems of performance optimization. Yet evolution in nature can be interpreted as more nuanced than a process of simple optimization. In particular, natural evolution is a divergent search that optimizes locally within each niche as it simultaneously diversifies. This tendency to discover both quality and diversity at the same time differs from many of the conventional algorithms of machine learning, and also thereby suggests a different foundation for inferring the approach of greatest potential for evolutionary algorithms. In fact, several recent evolutionary algorithms called quality diversity (QD) algorithms(e.g. novelty search with local competition and MAP-Elites) have drawn inspiration from this more nuanced view, aiming to fill a space of possibilities with the best possible example of each type of achievable behavior. The result is a new class of algorithms that return an archive of diverse, high-quality behaviors in a single run. The aim in this paper is to study the application of QD algorithms in challenging environments (in particular complex mazes) to establish their best practices for ambitious domains in the future. In addition to providing insight into cases when QD succeeds and fails, a new approach is investigated that hybridizes multiple views of behaviors (called behavior characterizations) in the same run, which succeeds in overcoming some of the challenges associated with searching for QD with respect to a behavior characterization that is not necessarily sufficient for generating both quality and diversity at the same time.

[1]  E. W. Adams,et al.  Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting , 1962 .

[2]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  Inman Harvey,et al.  Explorations in Evolutionary Robotics , 1993, Adapt. Behav..

[8]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[9]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[10]  Thomas Bäck,et al.  Evolutionary computation: comments on the history and current state , 1997, IEEE Trans. Evol. Comput..

[11]  D. Floreano,et al.  Evolutionary Robotics: The Biology,Intelligence,and Technology , 2000 .

[12]  Gregory S. Hornby,et al.  The advantages of generative grammatical encodings for physical design , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[13]  Jordan B. Pollack,et al.  Creating High-Level Components with a Generative Representation for Body-Brain Evolution , 2002, Artificial Life.

[14]  Josh Bongard,et al.  Evolving modular genetic regulatory networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[15]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[16]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[17]  Risto Miikkulainen,et al.  A Taxonomy for Artificial Embryogeny , 2003, Artificial Life.

[18]  Russell K. Standish,et al.  Open-Ended Artificial Evolution , 2002, Int. J. Comput. Intell. Appl..

[19]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[20]  Kenneth O. Stanley,et al.  Compositional Pattern Producing Networks : A Novel Abstraction of Development , 2007 .

[21]  Mark A. Bedau.  The arrow of complexity hypothesis , 2008, ALIFE.

[22]  Francisco Fernández de Vega,et al.  Discovering Several Robot Behaviors through Speciation , 2008, EvoWorkshops.

[23]  Kenneth O. Stanley,et al.  Exploiting Open-Endedness to Solve Problems Through the Search for Novelty , 2008, ALIFE.

[24]  Charles E. Hughes,et al.  How novelty search escapes the deceptive trap of learning to learn , 2009, GECCO.

[25]  Stéphane Doncieux,et al.  Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity , 2009, 2009 IEEE Congress on Evolutionary Computation.

[26]  Ben Jones,et al.  Novelty of behaviour as a basis for the neuro-evolution of operant reward learning , 2009, GECCO.

[27]  Heather Goldsby,et al.  Automatically discovering properties that specify the latent behavior of UML models , 2010, MODELS'10.

[28]  S. Casper.  Book Review: Mind as machine: a history of cognitive science , 2010, Medical History.

[29]  Markus Olhofer,et al.  Towards Directed Open-Ended Search by a Novelty Guided Evolution Strategy , 2010, PPSN.

[30]  Peter Krcah.  Solving deceptive tasks in robot body-brain co-evolution by searching for behavioral novelty , 2010, ISDA 2010.

[31]  Charles E. Hughes,et al.  Evolving plastic neural networks with novelty search , 2010, Adapt. Behav..

[32]  Malcolm I. Heywood,et al.  Novelty-Based Fitness: An Evaluation under the Santa Fe Trail , 2010, EuroGP.

[33]  Kenneth O. Stanley,et al.  Revising the evolutionary computation abstraction: minimal criteria novelty search , 2010, GECCO '10.

[34]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[35]  Francisco Fernández de Vega,et al.  Speciation in Behavioral Space for Evolutionary Robotics , 2011, J. Intell. Robotic Syst..

[36]  Kenneth O. Stanley,et al.  Evolving a diversity of virtual creatures through novelty search and local competition , 2011, GECCO '11.

[37]  Jean-Baptiste Mouret.  Novelty-Based Multiobjectivization , 2011 .

[38]  Shimon Whiteson,et al.  Critical factors in the performance of novelty search , 2011, GECCO '11.

[39]  Kenneth O. Stanley.  Why Evolutionary Robotics Will Matter , 2011 .

[40]  Kenneth O. Stanley,et al.  On the deleterious effects of a priori objectives on evolution and representation , 2011, GECCO '11.

[41]  Stéphane Doncieux,et al.  Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study , 2012, Evolutionary Computation.

[42]  Julian Togelius,et al.  Enhancements to constrained novelty search: two-population novelty search for generating game content , 2013, GECCO '13.

[43]  Antoine Cully,et al.  Behavioral repertoire learning in robotics , 2013, GECCO '13.

[44]  Sebastian Risi,et al.  Confronting the challenge of learning a flexible neural controller for a diversity of morphologies , 2013, GECCO '13.

[45]  Julian Togelius,et al.  Transforming Exploratory Creativity with DeLeNoX, , 2021, ICCC.

[46]  Leonardo Trujillo,et al.  Searching for novel regression functions , 2013, 2013 IEEE Congress on Evolutionary Computation.

[47]  Kenneth O. Stanley,et al.  Indirectly Encoded Sodarace for Artificial Life , 2013, ECAL.

[48]  Leonardo Trujillo,et al.  Searching for novel clustering programs , 2013, GECCO '13.

[49]  Anders Lyhne Christensen,et al.  Evolution of swarm robotics systems with novelty search , 2013, Swarm Intelligence.

[50]  Sebastian Risi,et al.  Single-unit pattern generators for quadruped locomotion , 2013, GECCO '13.

[51]  Anders Lyhne Christensen,et al.  Generic behaviour similarity measures for evolutionary swarm robotics , 2013, GECCO '13.

[52]  Kenneth O. Stanley,et al.  Evolvability Is Inevitable: Increasing Evolvability without the Pressure to Adapt , 2013, PloS one.

[53]  Jeff Clune,et al.  Novelty search creates robots with general skills for exploration , 2014, GECCO.

[54]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[55]  Jean-Baptiste Mouret,et al.  Illuminating search spaces by mapping elites , 2015, ArXiv.

[56]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Kenneth O. Stanley,et al.  Confronting the Challenge of Quality Diversity , 2015, GECCO.

[58]  Jason Yosinski,et al.  Innovation Engines: Automated Creativity and Improved Stochastic Optimization via Deep Learning , 2015, GECCO.

[59]  Kenneth O. Stanley,et al.  Unsupervised Feature Learning through Divergent Discriminative Feature Accumulation , 2014, AAAI.

[60]  Anders Lyhne Christensen,et al.  Devising Effective Novelty Search Algorithms: A Comprehensive Empirical Study , 2015, GECCO.

[61]  A. E. Eiben,et al.  Evolutionary Robotics: What, Why, and Where to , 2015, Front. Robot. AI.

[62]  Sebastian Risi,et al.  Creative Generation of 3D Objects with Deep Learning and Innovation Engines , 2016, ICCC.