Instance based Transfer Learning for Genetic Programming for Symbolic Regression

Transfer learning aims to utilise knowledge acquired from the source domain to improve the learning performance in the target domain. It attracts increasing interests and many transfer learning approaches have been proposed. However, studies on transfer learning for genetic programming for symbolic regression are still rare, although clearly desired, due to the difficulty to evolve models with a good cross-domain generalisation ability. This work proposes a new instance weighting framework for transfer learning in genetic programming for symbolic regression. The key idea is to utilise a local weight updating scheme to identify and learn from more useful source domain instances and reduce the effort on the source domain instances, which are more different from the target domain data. The experimental results show that the proposed method notably enhances the learning capacity and the generalisation performance of genetic programming on the target domain and also outperforms some state-of-the-art regression methods.

[1]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[2]  Feng Zhou,et al.  A deeply supervised residual network for HEp-2 cell classification via cross-modal transfer learning , 2018, Pattern Recognit..

[3]  Gang Chen,et al.  Evolutionary Multitask Optimisation for Dynamic Job Shop Scheduling Using Niched Genetic Programming , 2018, Australasian Conference on Artificial Intelligence.

[4]  Peter Stone,et al.  Boosting for Regression Transfer , 2010, ICML.

[5]  Debjani Chakraborty,et al.  Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration. , 2017, Biomedical optics express.

[6]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[7]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[8]  Sean Luke,et al.  Fighting Bloat with Nonparametric Parsimony Pressure , 2002, PPSN.

[9]  D.P. Solomatine,et al.  AdaBoost.RT: a boosting algorithm for regression problems , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[10]  Bing Xue,et al.  Common subtrees in related problems: A novel transfer learning approach for genetic programming , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[11]  Nguyen Quang Uy,et al.  Transfer learning in genetic programming , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[12]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[13]  Grant Dick,et al.  Revisiting interval arithmetic for regression problems in genetic programming , 2017, GECCO.

[14]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[15]  Witold Pedrycz,et al.  Fuzzy Regression Transfer Learning in Takagi–Sugeno Fuzzy Models , 2017, IEEE Transactions on Fuzzy Systems.

[16]  Mengjie Zhang,et al.  Further investigation on genetic programming with transfer learning for symbolic regression , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[17]  Mengjie Zhang,et al.  Structural Risk Minimization-Driven Genetic Programming for Enhancing Generalization in Symbolic Regression , 2019, IEEE Transactions on Evolutionary Computation.

[18]  Tao Xiang,et al.  Joint Semantic and Latent Attribute Modelling for Cross-Class Transfer Learning , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Bing Xue,et al.  Cross-Domain Reuse of Extracted Knowledge in Genetic Programming for Image Classification , 2017, IEEE Transactions on Evolutionary Computation.

[20]  John R. Koza,et al.  Genetic Programming II , 1992 .

[21]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[22]  Masashi Sugiyama,et al.  Direct Density Ratio Estimation for Large-scale Covariate Shift Adaptation , 2008, SDM.

[23]  Leo Breiman,et al.  Discussion: Multivariate Adaptive Regression Splines , 1991 .