Towards billion-bit optimization via a parallel estimation of distribution algorithm

This paper presents a highly efficient, fully parallelized implementation of the compact genetic algorithm (cGA) to solve very large scale problems with millions to billions of variables. The paper presents principled results demonstrating the scalable solution of a difficult test function on instances over a billion variables using a parallel implementation of cGA. The problem addressed is a noisy, blind problem over a vector of binary decision variables. Noise is added equaling up to a tenth of the deterministic objective function variance of the problem, thereby making it difficult for simple hillclimbers to find the optimal solution. The compact GA, on the other hand, is able to find the optimum in the presence of noise quickly, reliably, and accurately, and the solution scalability follows known convergence theories. These results on noisy problem together with other results on problems involving varying modularity, hierarchy, and overlap foreshadow routine solution of billion-variable problems across the landscape of search problems.

[1]  Stavros A. Zenios,et al.  A Scalable Parallel Interior Point Algorithm for Stochastic Linear Programming and Robust Optimization , 1997, Comput. Optim. Appl..

[2]  M. Schoenauer,et al.  An efficient memetic, permutation-based evolutionary algorithm for real-world train timetabling , 2005, 2005 IEEE Congress on Evolutionary Computation.

[3]  Seung-wook Shin,et al.  A parallel algorithm for large-scale linear programs with a special structure , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[4]  Kihong Park,et al.  Scalability problems of genetic search , 1994, Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

[5]  Kalyanmoy Deb,et al.  Efficiently Solving: A Large-Scale Integer Linear Program Using a Customized Genetic Algorithm , 2004, GECCO.

[6]  David E. Goldberg,et al.  Let's Get Ready to Rumble: Crossover Versus Mutation Head to Head , 2004, GECCO.

[7]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[8]  Stavros A. Zenios,et al.  Scalable Parallel Benders Decomposition for Stochastic Linear Programming , 1997, Parallel Comput..

[9]  Lois C. McInnes,et al.  A case study in the performance and scalability of optimization algorithms , 2001, TOMS.

[10]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[11]  E. Cantu-Paz,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1997, Evolutionary Computation.

[12]  Franz Rothlauf,et al.  Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms , 2004 .

[13]  Heinz Mühlenbein,et al.  How Genetic Algorithms Really Work: Mutation and Hillclimbing , 1992, PPSN.

[14]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[15]  Jacek Gondzio,et al.  Direct Solution of Linear Systems of Size 109 Arising in Optimization with Interior Point Methods , 2005, PPAM.

[16]  Michael C. Fu,et al.  Guest editorial , 2003, TOMC.

[17]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[18]  David E. Goldberg,et al.  Fluctuating Crosstalk as a Source of Deterministic Noise and Its Effects on GA Scalability , 2006, EvoWorkshops.

[19]  L. Biegler,et al.  Large-scale nonlinear programming with a cape-open compliant interface , 2005 .

[20]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[21]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[22]  D. Goldberg,et al.  A matrix approach for finding extrema: problems with modularity, hierarchy, and overlap , 2006 .

[23]  K. Deb,et al.  Optimal Scheduling of Casting Sequence Using Genetic Algorithms , 2003 .

[24]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[25]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[26]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[27]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.