Improving the Generalisation Ability of Genetic Programming with Semantic Similarity based Crossover

This paper examines the impact of semantic control on the ability of Genetic Programming (GP) to generalise via a semantic based crossover operator (Semantic Similarity based Crossover - SSC). The use of validation sets is also investigated for both standard crossover and SSC. All GP systems are tested on a number of real-valued symbolic regression problems. The experimental results show that while using validation sets barely improve generalisation ability of GP, by using semantics, the performance of Genetic Programming is enhanced both on training and testing data. Further recorded statistics shows that the size of the evolved solutions by using SSC are often smaller than ones obtained from GP systems that do not use semantics. This can be seen as one of the reasons for the success of SSC in improving the generalisation ability of GP.

[1]  Peter Nordin,et al.  The Effect of Extensive Use of the Mutation Operator on Generalization in Genetic Programming Using Sparse Data Sets , 1996, PPSN.

[2]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[3]  José R. Álvarez,et al.  Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired Approach: First International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2005, Las Palmas, Canary Islands, Spain, June 15-18, 2005, Proceedings, Part II , 2005, IWINAC.

[4]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[5]  Colin G. Johnson,et al.  Semantically driven crossover in genetic programming , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[6]  Jens Gottlieb,et al.  Evolutionary Computation in Combinatorial Optimization , 2006, Lecture Notes in Computer Science.

[7]  Cyril Fonlupt,et al.  Tarpeian Bloat Control and Generalization Accuracy , 2005, EuroGP.

[8]  Nicholas Freitag McPhee,et al.  Semantic Building Blocks in Genetic Programming , 2008, EuroGP.

[9]  Jacques-André Landry,et al.  Relaxed genetic programming , 2006, GECCO.

[10]  Marc Parizeau,et al.  Genetic Programming, Validation Sets, and Parsimony Pressure , 2006, EuroGP.

[11]  Edmund K. Burke,et al.  On improving genetic programming for symbolic regression , 2005, 2005 IEEE Congress on Evolutionary Computation.

[12]  Leonardo Vanneschi,et al.  Using crossover based similarity measure to improve genetic programming generalization ability , 2009, GECCO.

[13]  Conor Ryan,et al.  On Improving Generalisation in Genetic Programming , 2009, EuroGP.

[14]  Colin G. Johnson,et al.  Genetic Programming with Fitness Based on Model Checking , 2007, EuroGP.

[15]  Hans-Paul Schwefel,et al.  Parallel Problem Solving from Nature — PPSN IV , 1996, Lecture Notes in Computer Science.

[16]  S. B. Atienza-Samols,et al.  With Contributions by , 1978 .

[17]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[18]  Gerhard Goos,et al.  Automated Technology for Verification and Analysis , 2004, Lecture Notes in Computer Science.

[19]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[20]  Kwong-Sak Leung,et al.  An induction system that learns programs in different programming languages using genetic programming and logic grammars , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[21]  Doron A. Peled,et al.  Genetic Programming and Model Checking: Synthesizing New Mutual Exclusion Algorithms , 2008, ATVA.

[22]  Matthew P. Evett,et al.  Preventing overfitting in GP with canary functions , 2005, GECCO '05.

[23]  Ibrahim Kushchu,et al.  An Evaluation of EvolutionaryGeneralisation in Genetic Programming , 2002, Artificial Intelligence Review.

[24]  Krzysztof Krawiec,et al.  Approximating geometric crossover in semantic space , 2009, GECCO.

[25]  Alfonso Ortega,et al.  Attribute Grammar Evolution , 2005, IWINAC.

[26]  Colin G. Johnson.  What can automatic programming learn from theoretical computer science , 2002 .

[27]  Michael O'Neill,et al.  An Attribute Grammar Decoder for the 01 MultiConstrained Knapsack Problem , 2005, EvoCOP.

[28]  Michael O'Neill,et al.  Semantic Aware Crossover for Genetic Programming: The Case for Real-Valued Function Regression , 2009, EuroGP.

[29]  Mohamed Nassim Seghir,et al.  A Lightweight Approach for Loop Summarization , 2011, ATVA.

[30]  Doron A. Peled,et al.  Model Checking-Based Genetic Programming with an Application to Mutual Exclusion , 2008, TACAS.

[31]  Michael O'Neill,et al.  Semantic Similarity Based Crossover in GP: The Case for Real-Valued Function Regression , 2009, Artificial Evolution.

[32]  N. Given.  Genetic Programming , Validation Sets , and Parsimony Pressure , 2005 .

[33]  Peter Nordin,et al.  Benchmarking the generalization capabilities of a compiling genetic programming system using sparse data sets , 1996 .

[34]  Colin G. Johnson.  Deriving Genetic Programming Fitness Properties by Static Analysis , 2002, EuroGP.

[35]  Maarten Keijzer,et al.  Improving Symbolic Regression with Interval Arithmetic and Linear Scaling , 2003, EuroGP.

[36]  Riccardo Poli,et al.  Genetic Programming: An Introduction and Tutorial, with a Survey of Techniques and Applications , 2008, Computational Intelligence: A Compendium.