Transfer learning in genetic programming

Transfer learning is a process in which a system can apply knowledge and skills learned in previous tasks to novel tasks. This technique has emerged as a new framework to enhance the performance of learning methods in machine learning. Surprisingly, transfer learning has not deservedly received the attention from the Genetic Programming research community. In this paper, we propose several transfer learning methods for Genetic Programming (GP). These methods were implemented by transferring a number of good individuals or sub-individuals from the source to the target problem. They were tested on two families of symbolic regression problems. The experimental results showed that transfer learning methods help GP to achieve better training errors. Importantly, the performance of GP on unseen data when implemented with transfer learning was also considerably improved. Furthermore, the impact of transfer learning to GP code bloat was examined that showed that limiting the size of transferred individuals helps to reduce the code growth problem in GP.

[1]  Ahmet Arslan,et al.  Genetic transfer learning , 2010, Expert Syst. Appl..

[2]  Qiang Yang,et al.  Transferring Naive Bayes Classifiers for Text Classification , 2007, AAAI.

[3]  Olac Fuentes,et al.  Knowledge Transfer in Deep convolutional Neural Nets , 2007, Int. J. Artif. Intell. Tools.

[4]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[5]  Leonardo Vanneschi,et al.  Operator equalisation for bloat free genetic programming and a survey of bloat control methods , 2011, Genetic Programming and Evolvable Machines.

[6]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[7]  T. Krink,et al.  Dynamic memory model for non-stationary optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[8]  Thomas G. Dietterich,et al.  To transfer or not to transfer , 2005, NIPS 2005.

[9]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[10]  Wojciech Jaskowski,et al.  Better GP benchmarks: community survey results and proposals , 2012, Genetic Programming and Evolvable Machines.

[11]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[12]  Andries Petrus Engelbrecht,et al.  Adaptive Genetic Programming for dynamic classification problems , 2009, 2009 IEEE Congress on Evolutionary Computation.

[13]  Nguyen Xuan Hoai,et al.  Developmental Evaluation in Genetic Programming: The Preliminary Results , 2006, EuroGP.

[14]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[15]  Lawrence Carin,et al.  Logistic regression with an auxiliary data source , 2005, ICML.

[16]  Riccardo Poli,et al.  Developmental plasticity in linear genetic programming , 2009, GECCO.

[17]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[18]  Gisele L. Pappa,et al.  Semi-supervised genetic programming for classification , 2011, GECCO '11.

[19]  Raymond J. Mooney,et al.  Transfer Learning from Minimal Target Data by Mapping across Relational Domains , 2009, IJCAI.

[20]  Eoin Murphy.  Examining grammars and grammatical evolution in dynamic environments , 2011, GECCO '11.

[21]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[22]  Anthony Brabazon,et al.  Foundations in Grammatical Evolution for Dynamic Environments , 2009, Studies in Computational Intelligence.

[23]  Matthew E. Taylor.  Transfer in Reinforcement Learning Domains , 2009, Studies in Computational Intelligence.

[24]  Michael O'Neill,et al.  Semantic-Based Subtree Crossover Applied to Dynamic Problems , 2011, 2011 Third International Conference on Knowledge and Systems Engineering.

[25]  Gang Wang,et al.  Boosting for transfer learning from multiple data sources , 2012, Pattern Recognit. Lett..

[26]  Rich Caruana,et al.  Inductive Transfer for Bayesian Network Structure Learning , 2007, ICML Unsupervised and Transfer Learning.

[27]  Xinshun Xu,et al.  AdaGP-Rank: Applying boosting technique to genetic programming for learning to rank , 2010, 2010 IEEE Youth Conference on Information, Computing and Telecommunications.

[28]  Peter Stone,et al.  Graph-Based Domain Mapping for Transfer Learning in General Games , 2007, ECML.