Surrogate Genetic Programming: A semantic aware evolutionary search

Many semantic search based on Genetic Programming (GP) use a trial-and-error scheme to attain semantically diverse offspring in the evolutionary search. This results in significant impediments on the success of semantic-based GP in solving real world problems, due to the additional computational overheads incurred. This paper proposes a surrogate Genetic Programming (or sGP in short) to retain the appeal of semantic-based evolutionary search for handling challenging problems with enhanced efficiency. The proposed sGP divides the population into two parts (µ and λ) then it evolves µ percentage of the population using standard GP search operators, while the remaining λ percentage of the population are evolved with the aid of meta-models (or approximation models) that serve as surrogate to the original objective function evaluation (which is computationally intensive). In contrast to previous works, two forms of meta-models are introduced in this study to make the idea of using surrogate in GP search feasible and successful. The first denotes a "Semantic-model" for prototyping the semantic representation space of the GP trees (genotype/syntactic-space). The second is a "Fitness-model", which maps solutions in the semantic space to the objective or fitness space. By exploiting the two meta-models collectively in serving as a surrogate that replaces the original problem landscape of the GP search process, more cost-effective generation of offspring that guides the search in exploring regions where high quality solutions resides can then be attained. Experimental studies covering three separate GP domains, namely, (1) Symbolic regression, (2) Even n-parity bit, and (3) a real-world Time-series forecasting problem domain involving three datasets, demonstrate that sGP is capable of attaining reliable, high quality, and efficient performance under a limited computational budget. Results also showed that sGP outperformed the standard GP, GP based on random training-set technique, and GP based on conventional data-centric objectives as surrogate.

[1]  Colin G. Johnson,et al.  Semantically driven crossover in genetic programming , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[2]  Fred H. Lesh,et al.  Multi-dimensional least-squares polynomial curve fitting , 1959, CACM.

[3]  Michael O'Neill,et al.  Semantic Aware Crossover for Genetic Programming: The Case for Real-Valued Function Regression , 2009, EuroGP.

[4]  Ahmed Kattan,et al.  Geometric Surrogate Model Based Optimisation for Genetic Programming : Initial Experiments , 2011 .

[5]  Colin G. Johnson,et al.  Semantically driven mutation in genetic programming , 2009, 2009 IEEE Congress on Evolutionary Computation.

[6]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[7]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Nguyen Xuan Hoai,et al.  BASED MUTATION IN GENETIC PROGRAMMING : THE CASE FOR REAL-VALUED SYMBOLIC REGRESSION , 2009 .

[9]  Bernhard Sendhoff,et al.  A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation , 2007, GECCO '07.

[10]  Bu-Sung Lee,et al.  Memetic algorithm using multi-surrogates for computationally expensive optimization problems , 2007, Soft Comput..

[11]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[12]  Krzysztof Krawiec,et al.  Geometric Semantic Genetic Programming , 2012, PPSN.

[13]  Yoel Tenne,et al.  A Model-Assisted Memetic Algorithm for Expensive Optimization Problems , 2009, Nature-Inspired Algorithms for Optimisation.

[14]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[15]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[16]  Kok Wai Wong,et al.  Surrogate-Assisted Evolutionary Optimization Frameworks for High-Fidelity Engineering Design Problems , 2005 .

[17]  Adrian G. Bors,et al.  Introduction of the Radial Basis Function (RBF) Networks , 2000 .

[18]  Michael O'Neill,et al.  Semantic Similarity Based Crossover in GP: The Case for Real-Valued Function Regression , 2009, Artificial Evolution.

[19]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[20]  Riccardo Poli,et al.  Genetic-Programming Based Prediction of Data Compression Saving , 2009, Artificial Evolution.

[21]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[22]  Gregory Piatetsky-Shapiro,et al.  High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality , 2000 .

[23]  Andy J. Keane,et al.  Aircraft wing design using GA-based multi-level strategies , 2000 .

[24]  Andreas Zell,et al.  Evolution strategies assisted by Gaussian processes with improved preselection criterion , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[25]  Kyriakos C. Giannakoglou,et al.  Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence , 2002 .

[26]  Salvador Pintos,et al.  An Optimization Methodology of Alkaline-Surfactant-Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates , 2004 .

[27]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[28]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[29]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[30]  D. Mackay,et al.  Introduction to Gaussian processes , 1998 .

[31]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[32]  Masoud Rais-Rohani,et al.  Ensemble of Metamodels with Optimized Weight Factors , 2008 .

[33]  Alberto Moraglio.  Geometric Unification of Evolutionary Algorithms , 2006 .

[34]  David Jackson,et al.  Promoting Phenotypic Diversity in Genetic Programming , 2010, PPSN.

[35]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[36]  Ahmed Kattan,et al.  Evolving radial basis function networks via GP for estimating fitness values using surrogate models , 2012, 2012 IEEE Congress on Evolutionary Computation.

[37]  Bernhard Sendhoff,et al.  Evolutionary Optimization with Dynamic Fidelity Computational Models , 2008, ICIC.

[38]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[39]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[40]  Mohammed Zaki,et al.  The use of genetic programming for adaptive text compression , 2009, Int. J. Inf. Coding Theory.