论文信息 - Exploring Overfitting in Genetic Programming

Exploring Overfitting in Genetic Programming

The problem of overfitting (focusing closely on examples at the loss of generalization power) is encountered in all supervised machine learning schemes. This study is dedicated to explore some aspects of overfitting in the particular case of genetic programming. After recalling the causes usually invoked to explain overfitting such as hypothesis complexity or noisy learning examples, we test and compare the resistance to overfitting on three variants of genetic programming algorithms (basic GP, sizefair crossover GP and GP with boosting) on two benchmarks, a symbolic regression and a classification problem. We propose guidelines based on these results to help reduce overfitting with genetic programming.

[1] Cyril Fonlupt,et al. Applying Boosting Techniques to Genetic Programming , 2001, Artificial Evolution.

[2] H. Iba. Bagging, Boosting, and bloating in Genetic Programming , 1999 .

[3] S. Luke. When short runs beat long runs , 2001 .

[4] Yoav Freund,et al. A Short Introduction to Boosting , 1999 .

[5] William B. Langdon,et al. Combining Decision Trees and Neural Networks for Drug Discovery , 2002, EuroGP.

[6] Pedro M. Domingos. The Role of Occam's Razor in Knowledge Discovery , 1999, Data Mining and Knowledge Discovery.

[7] Andrew Y. Ng,et al. Preventing "Overfitting" of Cross-Validation Data , 1997, ICML.

[8] John R. Koza,et al. Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[9] Kumar Chellapilla,et al. Data mining using genetic programming: the implications of parsimony on generalization error , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[10] Yishay Mansour,et al. Why averaging classifiers can protect against overfitting , 2001, AISTATS.

[11] Cyril Fonlupt,et al. Genetic Programming with Boosting for Ambiguities in Regression Problems , 2003, EuroGP.

[12] William B. Langdon,et al. Size Fair and Homologous Tree Crossovers for Tree Genetic Programming , 2000, Genetic Programming and Evolvable Machines.