Guidelines for defining benchmark problems in Genetic Programming

The field of Genetic Programming has recently seen a surge of attention to the fact that benchmarking and comparison of approaches is often done in non-standard ways, using poorly designed comparison problems. We raise some issues concerning the design of benchmarks, within the domain of symbolic regression, through experimental evidence. A set of guidelines is provided, aiming towards careful definition and use of artificial functions as symbolic regression benchmarks.

[1]  Leonardo Vanneschi,et al.  Genetic programming needs better benchmarks , 2012, GECCO '12.

[2]  Maarten Keijzer,et al.  Improving Symbolic Regression with Interval Arithmetic and Linear Scaling , 2003, EuroGP.

[3]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[4]  Wojciech Jaskowski,et al.  Better GP benchmarks: community survey results and proposals , 2012, Genetic Programming and Evolvable Machines.

[5]  Miguel Nicolau,et al.  Introducing Grammar Based Extensions for Grammatical Evolution , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[6]  Michael F. Korns.  Accuracy in Symbolic Regression , 2011 .

[7]  Michael D. McKay,et al.  Latin hypercube sampling as a tool in uncertainty analysis of computer models , 1992, WSC '92.

[8]  Dick den Hertog,et al.  Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via Pareto Genetic Programming , 2009, IEEE Transactions on Evolutionary Computation.

[9]  Michael O'Neill,et al.  Grammatical evolution - evolutionary automatic programming in an arbitrary language , 2003, Genetic programming.

[10]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[11]  Riccardo Poli,et al.  A Simple but Theoretically-Motivated Method to Control Bloat in Genetic Programming , 2003, EuroGP.

[12]  Paulien Hogeweg,et al.  Evolutionary Consequences of Coevolving Targets , 1997, Evolutionary Computation.

[13]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[14]  Jason H. Moore,et al.  Genetic Programming Theory and Practice IX , 2011 .

[15]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.