Minimal walsh structure and ordinal linkage of monotonicity-invariant function classes on bit strings

Problem structure, or linkage, refers to the interaction between variables in a black-box fitness function. Discovering structure is a feature of a range of algorithms, including estimation of distribution algorithms (EDAs) and perturbation methods (PMs). The complexity of structure has traditionally been used as a broad measure of problem difficulty, as the computational complexity relates directly to the complexity of structure. The EDA literature describes necessary and unnecessary interactions in terms of the relationship between problem structure and the structure of probabilistic graphical models discovered by the EDA. In this paper we introduce a classification of problems based on monotonicity invariance. We observe that the minimal problem structures for these classes often reveal that significant proportions of detected structures are unnecessary. We perform a complete classification of all functions on 3 bits. We consider nonmonotonicity linkage discovery using perturbation methods and derive a concept of directed ordinal linkage associated to optimization schedules. The resulting refined classification factored out by relabeling, shows a hierarchy of nine directed ordinal linkage classes for all 3-bit functions. We show that this classification allows precise analysis of computational complexity and parallelizability and conclude with a number of suggestions for future work.

[1]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[2]  Martin Pelikan,et al.  Spurious dependencies and EDA scalability , 2010, GECCO '10.

[3]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[4]  Gilbert Owusu,et al.  A fully multivariate DEUM algorithm , 2009, 2009 IEEE Congress on Evolutionary Computation.

[5]  Dirk Thierens,et al.  Optimal mixing evolutionary algorithms , 2011, GECCO '11.

[6]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[7]  Chung-Yao Chuang,et al.  Likage identification by perturbation and decision tree induction , 2007, 2007 IEEE Congress on Evolutionary Computation.

[8]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[9]  Matthew J. Streeter Upper Bounds on the Time and Space Complexity of Optimizing Additively Separable Functions , 2004, GECCO.

[10]  V. Ralph Algazi,et al.  Unified Matrix Treatment of the Fast Walsh-Hadamard Transform , 1976, IEEE Transactions on Computers.

[11]  Martin Pelikan,et al.  Bayesian Optimization Algorithm , 2005 .

[12]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[13]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[14]  Masaharu Munetomo,et al.  Metropolitan Area Network Design Using GA Based on Hierarchical Linkage Identification , 2003, GECCO.

[15]  R. Fua,et al.  The analysis of the (100)surface of GaAs for NEA photocathode with XPS , 2004, IVESC 2004. The 5th International Vacuum Electron Sources Conference Proceedings (IEEE Cat. No.04EX839).

[16]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[17]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[18]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part I, A Gentle Introduction , 1989, Complex Syst..

[19]  Masaharu Munetomo,et al.  Identifying Linkage Groups by Nonlinearity/Non-monotonicity Detection , 1999 .

[20]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.