Finding Building Blocks through Eigenstructure Adaptation

A fundamental aspect of many evolutionary approaches to synthesis of complex systems is the need to compose atomic elements into useful higher-level building blocks. However, the ability of genetic algorithms to promote useful building blocks is based critically on genetic linkage - the assumption that functionally related alleles are also arranged compactly on the genome. In many practical problems, linkage is not known a priori or may change dynamically. Here we propose that a problem's Hessian matrix reveals this linkage, and that an eigenstructure analysis of the Hessian provides a transformation of the problem to a space where first-order genetic linkage is optimal. Genetic algorithms that dynamically transforms the problem space can operate much more efficiently. We demonstrate the proposed approach on a real-valued adaptation of Kaufmann's NK landscapes and discuss methods for extending it to higher-order linkage.

[1]  D. Shanno Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[2]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[3]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[4]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[5]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[6]  John R. Koza,et al.  Hierarchical Genetic Algorithms Operating on Populations of Computer Programs , 1989, IJCAI.

[7]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[8]  Stuart A. Kauffman,et al.  ORIGINS OF ORDER IN EVOLUTION: SELF-ORGANIZATION AND SELECTION , 1992 .

[9]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[10]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[11]  Arantxa Etxeverria The Origins of Order , 1993 .

[12]  David E. Goldberg,et al.  Learning Linkage , 1996, FOGA.

[13]  Hillol Kargupta,et al.  The performance of the gene expression messy genetic algorithm on real test functions , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[14]  Hillol Kargupta,et al.  The Gene Expression Messy Genetic Algorithm , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[15]  Samir W. Mahfoud Niching methods for genetic algorithms , 1996 .

[16]  Hava T. Siegelmann,et al.  High Order Eigentensors as Symbolic Rules in Competitive Learning , 1998, Hybrid Neural Systems.

[17]  Stefan Wermter,et al.  A Novel Modular Neural Architecture for Rule-Based and Similarity-Based Reasoning , 1998, Hybrid Neural Systems.

[18]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[19]  David E. Goldberg,et al.  Introducing Start Expression Genes to the Linkage Learning Genetic Algorithm , 2002, PPSN.

[20]  J. Pollack,et al.  A computational model of symbiotic composition in evolutionary transitions. , 2003, Bio Systems.

[21]  Kenneth O. Stanley,et al.  Achieving High-Level Functionality through Complexification , 2003 .