Guiding evolutionary search towards innovative solutions

The main goal of this work is to develop a method that, operating on top of an Evolutionary Algorithm, increases its likeliness of finding innovative solutions. This likeliness is laid out to be increased with the diversity of the solutions found, provided that they are of sufficient quality. The developed method needs to be applicable in a scenario in which the search is required to be started from a single, fixed solution. Therefore, a scheme is envisioned in which the search is performed in a sequential fashion, zooming in on a locally-optimal solution, and then exploring for a new potentially high-quality region based on a memory of solutions encountered earlier in the search. Two exploration criteria, one using an archive of earlier solutions as memory and the other deriving from a surrogate model trained on earlier solutions, were established to be worthwhile for integration into quality-based search. The resulting schemes were applied to a real-world airfoil optimization task, showing both to perform better than the baseline method of multiple standard optimization runs. The model-based approach delivers the best results, in the sense that it finds more solutions, more diverse solutions, and better-quality solutions than the baseline method.

[1]  Howard J. Hamilton,et al.  Measuring the interestingness of discovered knowledge: A principled approach , 2003, Intell. Data Anal..

[2]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[3]  Wei Chen,et al.  A non‐stationary covariance‐based Kriging method for metamodelling in engineering design , 2007 .

[4]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[5]  Bernhard Sendhoff,et al.  Aerodynamic Shape Optimisation using Evolution Strategies , 2002 .

[6]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[7]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[8]  Lothar Thiele,et al.  Maximizing population diversity in single-objective optimization , 2011, GECCO '11.

[9]  Bernhard Sendhoff,et al.  Three dimensional evolutionary aerodynamic design optimization with CMA-ES , 2005, GECCO '05.

[10]  Pierre-Yves Oudeyer,et al.  Active learning of inverse models with intrinsically motivated goal exploration in robots , 2013, Robotics Auton. Syst..

[11]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[12]  Michael T. M. Emmerich,et al.  On Quality Indicators for Black-Box Level Set Approximation , 2013, EVOLVE.

[13]  John S. Gero,et al.  Curious agents and situated design evaluations , 2004, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[14]  Rob Saunders,et al.  Designing for Interest and Novelty , 2001 .

[15]  Bernhard Sendhoff,et al.  Model-guided Evolution Strategies for Dynamically Balancing Exploration and Exploitation , 2011 .

[16]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[17]  Bruno Crémilleux,et al.  A Unified View of Objective Interestingness Measures , 2007, MLDM.

[18]  Jurgen Schmidhuber,et al.  Artificial curiosity with planning for autonomous perceptual and cognitive development , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[19]  Ofer M. Shir,et al.  Adaptive Niche Radii and Niche Shapes Approaches for Niching with the CMA-ES , 2010, Evolutionary Computation.

[20]  Bin Li,et al.  An evolution strategy assisted by an ensemble of local Gaussian process models , 2013, GECCO '13.

[21]  Tom Schaul,et al.  Artificial curiosity for autonomous space exploration , 2011 .

[22]  A. Solow,et al.  Measuring biological diversity , 2006, Environmental and Ecological Statistics.

[23]  Abraham Silberschatz,et al.  What Makes Patterns Interesting in Knowledge Discovery Systems , 1996, IEEE Trans. Knowl. Data Eng..

[24]  Bernhard Sendhoff,et al.  Novelty and interestingness measures for design-space exploration , 2013, GECCO '13.

[25]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[26]  Todd Hester,et al.  TEXPLORE: Temporal Difference Reinforcement Learning for Robots and Time-Constrained Domains , 2013, Studies in Computational Intelligence.

[27]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[28]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[29]  Balaji Padmanabhan,et al.  A Belief-Driven Method for Discovering Unexpected Patterns , 1998, KDD.

[30]  Michèle Sebag,et al.  Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy , 2012, GECCO '12.

[31]  Markus Olhofer,et al.  Towards Directed Open-Ended Search by a Novelty Guided Evolution Strategy , 2010, PPSN.

[32]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[33]  Lothar Thiele,et al.  Defining and Optimizing Indicator-Based Diversity Measures in Multiobjective Search , 2010, PPSN.

[34]  Faustino J. Gomez,et al.  Novelty-based restarts for evolution strategies , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[35]  Bernhard Sendhoff,et al.  Learning-Guided Exploration in Airfoil Optimization , 2013, IDEAL.

[36]  Jurgen Schmidhuber,et al.  Intrinsically motivated neuroevolution for vision-based reinforcement learning , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[37]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[38]  Risto Miikkulainen,et al.  Effective diversity maintenance in deceptive domains , 2013, GECCO '13.

[39]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[40]  Warren B. Powell,et al.  Reinforcement Learning and Its Relationship to Supervised Learning , 2004 .

[41]  Kenneth O. Stanley,et al.  Exploiting Open-Endedness to Solve Problems Through the Search for Novelty , 2008, ALIFE.

[42]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[43]  Rob Saunders,et al.  Curious Design Agents and Artificial Creativity - A Synthetic Approach to the Study of Creative Behaviour , 2001 .

[44]  Sebastian Thrun,et al.  Exploration in active learning , 1998 .

[45]  Toshiyuki Arima,et al.  A Numerical Investigation of Transonic Axial Compressor Rotor Flow Using a Low Reynolds Number k-ε Turbulence Model , 1997 .

[46]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[47]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[48]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[49]  Murray S. Davis,et al.  That's Interesting! , 1971 .

[50]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[51]  Tom Schaul,et al.  Exploring parameter space in reinforcement learning , 2010, Paladyn J. Behav. Robotics.

[52]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[53]  Jean-Baptiste Mouret Novelty-Based Multiobjectivization , 2011 .

[54]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[55]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[56]  Jürgen Schmidhuber,et al.  Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes , 2008, ABiALS.

[57]  M. Stein Statistical Interpolation of Spatial Data: Some Theory for Kriging , 1999 .

[58]  D. Ackley A connectionist machine for genetic hillclimbing , 1987 .

[59]  Faustino J. Gomez,et al.  When Novelty Is Not Enough , 2011, EvoApplications.

[60]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[61]  Jack P. C. Kleijnen,et al.  Kriging in Multi-response Simulation, including a Monte Carlo Laboratory (Replaced by 2014-012) , 2012 .

[62]  D. Dennis,et al.  SDO : A Statistical Method for Global Optimization , 1997 .

[63]  Rosanna Garcia,et al.  A critical look at technological innovation typology and innovativeness terminology: a literature review , 2002 .

[64]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[65]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[66]  Howard J. Hamilton,et al.  Interestingness measures for data mining: A survey , 2006, CSUR.

[67]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[68]  John S. and Saunders Robert Gero,et al.  Constructed Representations and Their Functions in Computational Models of Designing , 2000 .

[69]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[70]  Tom Schaul,et al.  Curiosity-driven optimization , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[71]  H. Kushner A versatile stochastic model of a function of unknown and time varying form , 1962 .

[72]  Aravind Srinivasan,et al.  Innovization: innovating design principles through optimization , 2006, GECCO.

[73]  Peter Stone,et al.  Intrinsically motivated model learning for a developing curious agent , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).