On genetic algorithms

We analyze the performance of a Genetic Type Algorithm we call Culling and a variety of other algorithms on a problem we refer to as ASP. Culling is near optimal for this problem, highly noise tolerant, and the best known a~~roach . . in some regimes. We show that the problem of learning the Ising perception is reducible to noisy ASP. These results provide an example of a rigorous analysis of GA’s and give insight into when and how C,A’s can beat competing methods. To analyze the genetic algorithm, we view it as a special type of submartingale. We prove some new large deviation bounds on this submartingale w~ich enable us to determine the running time of the algorithm.

[1]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[2]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[4]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[5]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[6]  Vasek Chvátal,et al.  Mastermind , 1983, Comb..

[7]  John H. Holland,et al.  Properties of the bucket brigade algorithm , 1985 .

[8]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[11]  Santosh S. Venkatesh,et al.  On learning binary weights for majority functions , 1991, COLT '91.

[12]  Yuh-Dauh Lyuu,et al.  The Transition to Perfect Generalization in Perceptrons , 1991, Neural Computation.

[13]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[14]  Avi Wigderson,et al.  Quadratic dynamical systems , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[15]  Santosh S. Venkatesh,et al.  On the average tractability of binary integer programming and the curious transition to perfect generalization in learning majority functions , 1993, COLT '93.

[16]  Melanie Mitchell,et al.  What makes a problem hard for a genetic algorithm? Some anomalous results and their explanation , 1993, Machine Learning.

[17]  John H. Holland,et al.  When will a Genetic Algorithm Outperform Hill Climbing , 1993, NIPS.

[18]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[19]  D. Haussler,et al.  Rigorous learning curve bounds from statistical mechanics , 1994, COLT '94.