Evolutionary Optimization Through PAC Learning

Abstract Strategies for evolutionary optimization (EO) typically experience a convergence phenomenon that involves a steady increase in the frequency of particular allelic combinations. When some allele is consistent throughout the population we essentially have a reduction in the dimension of the binary string space that is the objective function domain of feasible solutions. In this paper we consider dimension reduction to be the most salient feature of evolutionary optimization and present the theoretical setting for a novel algorithm that manages this reduction in a very controlled albeit stochastic manner. The ”Rising Tide Algorithm” facilitates dimension reductions through the discovery of bit interdependencies that are expressible as ring-sum-expansions (linear combinations in GF2). When suitable constraints are placed on the objective function these interdependencies are generated by algorithms involving approximations to the discrete Fourier transform (or Walsh Transform). Based on analytic techniques that are now used by researchers in PAC learning, the Rising Tide Algorithm attempts to capitalize on the intrinsic binary nature of the fitness function deriving from it a representation that is highly amenable to a theoretical analysis. Our overall objective is to describe certain algorithms for evolutionary optimization as heuristic techniques that work within the analytically rich environment of PAC learning. We also contend that formulation of these algorithms and empirical demonstrations of their success should give EO practitioners new insights into the current traditional strategies.

[1]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[2]  David E. Goldberg,et al.  SEARCH, Blackbox Optimization, And Sample Complexity , 1996, FOGA.

[3]  Johannes P. Ros.  Learning Boolean Functions with Genetic Algorithms: A PAC Analysis , 1992, FOGA.

[4]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[5]  Michael D. Vose,et al.  Isomorphisms of Genetic Algorithms , 1993, Artif. Intell..

[6]  L. Darrell Whitley,et al.  Predicting Epistasis from Mathematical Models , 1999, Evolutionary Computation.

[7]  John N. Hooker,et al.  Constraint satisfaction methods for generating valid cuts , 1997 .

[8]  Eyal Kushilevitz,et al.  Learning Decision Trees Using the Fourier Spectrum , 1993, SIAM J. Comput..

[9]  Hans Ulrich Simon,et al.  On learning ring-sum-expansions , 1990, COLT '90.

[10]  GUNAR E. LIEPINS,et al.  Representational issues in genetic optimization , 1990, J. Exp. Theor. Artif. Intell..

[11]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[12]  Richard K. Belew,et al.  When Both Individuals and Populations Search: Adding Simple Learning to the Genetic Algorithm , 1989, ICGA.

[13]  Nicholas J. Radcliffe,et al.  Non-Linear Genetic Representations , 1992, PPSN.

[14]  John A. Campbell,et al.  Harmonic Analysis, Epistasis and Genetic Algorithms , 1992, PPSN.

[15]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[16]  Jim Smith,et al.  Recombination strategy adaptation via evolution of gene linkage , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[17]  Michèle Sebag,et al.  Controlling Crossover through Inductive Learning , 1994, PPSN.

[18]  Melanie Mitchell,et al.  What makes a problem hard for a genetic algorithm? Some anomalous results and their explanation , 1993, Machine Learning.

[19]  Eric B. Baum,et al.  Neural net algorithms that learn in polynomial time from examples and queries , 1991, IEEE Trans. Neural Networks.

[20]  Lawrence Davis,et al.  Adapting Operator Probabilities in Genetic Algorithms , 1989, ICGA.

[21]  Robert J. Lechner.  HARMONIC ANALYSIS OF SWITCHING FUNCTIONS , 1971 .

[22]  Larry J. Eshelman,et al.  Biases in the Crossover Landscape , 1989, ICGA.

[23]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1993, JACM.

[24]  Nader H. Bshouty,et al.  More efficient PAC-learning of DNF with membership queries under the uniform distribution , 1999, COLT '99.

[25]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.