PAC learning and genetic programming

Genetic programming (GP) is a very successful type of learning algorithm that is hard to understand from a theoretical point of view. With this paper we contribute to the computational complexity analysis of genetic programming that has been started recently. We analyze GP in the well-known PAC learning framework and point out how it can observe quality changes in the the evolution of functions by random sampling. This leads to computational complexity bounds for a linear GP algorithm for perfectly learning any member of a simple class of linear pseudo-Boolean functions. Furthermore, we show that the same algorithm on the functions from the same class finds good approximations of the target function in less time.

[1]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[2]  Ibrahim Kushchu,et al.  Genetic programming and evolutionary generalization , 2002, IEEE Trans. Evol. Comput..

[3]  Riccardo Poli,et al.  Theoretical results in genetic programming: the next ten years? , 2010, Genetic Programming and Evolvable Machines.

[4]  Stephen R. Marsland,et al.  Convergence Properties of (μ + λ) Evolutionary Algorithms , 2011, AAAI.

[5]  Frank Neumann,et al.  Bioinspired computation in combinatorial optimization: algorithms and their computational complexity , 2012, GECCO '12.

[6]  Naim Dahnoun,et al.  Studies in Computational Intelligence , 2013 .

[7]  Vitaly Feldman,et al.  Evolvability from learning algorithms , 2008, STOC.

[8]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[9]  Una-May O'Reilly,et al.  Computational complexity analysis of simple genetic programming on two problems modeling isolated program semantics , 2010, FOGA '11.

[10]  Leslie G. Valiant,et al.  Evolvability , 2009, JACM.

[11]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[12]  David E. Goldberg,et al.  Where Does the Good Stuff Go, and Why? How Contextual Semantics Influences Program Structure in Simple Genetic Programming , 1998, EuroGP.

[13]  Cândida Ferreira,et al.  Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence , 2014, Studies in Computational Intelligence.

[14]  Candida Ferreira.  Gene expression programming , 2006 .

[15]  Cândida Ferreira,et al.  Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence (Studies in Computational Intelligence) , 2006 .

[16]  Vitaly Feldman.  A Complete Characterization of Statistical Query Learning with Applications to Evolvability , 2009, FOCS.

[17]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[18]  Heinz Mühlenbein,et al.  How Genetic Algorithms Really Work: Mutation and Hillclimbing , 1992, PPSN.