An introduction to population approaches for optimization and hierarchical objective functions: A discussion on the role of tabu search

Population approaches suitable for global combinatorial optimization are discussed in this paper. They are composed of a number of distinguishable individuals called "agents", each one using a particular optimization strategy. Periods of independent search follow phases on which the population is restarted from new configurations. Due to its intrinsic parallelism and the asynchronicity of the method, it is particularly suitable for parallel computers. Results on two test problems are presented in this paper. The individual search optimization strategies for each agent have been chosen having the basic characteristics of tabu search. This has been done in order to avoid mixing the hypothesized properties of these population approaches with those of more elaborate tabu search strategies, but remarking on its main characteristics. A set of four test problem "landscapes" is discussed and their use to improve and benchmark the results by using tabu search as the individual optimization strategy within a population heuristic is suggested and explored. The application of tabu search to new problem areas, like molecular biology, is also investigated.

[1]  Néstor Parga,et al.  Ultrametricity in the Kauffman model: a numerical test , 1988 .

[2]  M. Kimura.  The Neutral Theory of Molecular Evolution: Introduction , 1983 .

[3]  J. Beardwood,et al.  The shortest path through many points , 1959, Mathematical Proceedings of the Cambridge Philosophical Society.

[4]  Brian W. Kernighan,et al.  An Effective Heuristic Algorithm for the Traveling-Salesman Problem , 1973, Oper. Res..

[5]  F. Parak,et al.  Dynamics of metmyoglobin crystals investigated by nuclear gamma resonance absorption. , 1981, Journal of molecular biology.

[6]  Jean-Luc Chatelain,et al.  A fractal approach to the clustering of earthquakes: Applications to the seismicity of the New Hebrides , 1987 .

[7]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[8]  G. W. Snedecor.  Statistical Methods , 1964 .

[9]  Schuster,et al.  Physical aspects of evolutionary optimization and adaptation. , 1989, Physical review. A, General physics.

[10]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[11]  M. G. Shnirman,et al.  Hierarchical model of defect development and seismicity , 1990 .

[12]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[13]  Bernardo A. Huberman,et al.  The ecology of computation , 1988, Digest of Papers. COMPCON Spring 89. Thirty-Fourth IEEE Computer Society International Conference: Intellectual Leverage.

[14]  D. Whitefield,et al.  A review of: “Practical Nonpararnetric Statistics. By W. J. CONOVER. (New York: Wiley, 1971.) [Pl" x+462.] £5·25. , 1972 .

[15]  Eugene L. Lawler,et al.  The Traveling Salesman Problem: A Guided Tour of Combinatorial Optimization , 1985 .

[16]  E. Bonomi,et al.  The N-City Travelling Salesman Problem: Statistical Mechanics and the Metropolis Algorithm , 1984 .

[17]  Hans Frauenfelder,et al.  Temperature-dependent X-ray diffraction as a probe of protein structural dynamics , 1979, Nature.

[18]  Joel E. Cohen,et al.  Threshold phenomena in random structures , 1988, Discret. Appl. Math..

[19]  F J Ayala,et al.  Is a New Evolutionary Synthesis Necessary? , 1981, Science.

[20]  Geoffrey C. Fox,et al.  A deterministic annealing approach to clustering , 1990, Pattern Recognit. Lett..

[21]  S. Kirkpatrick,et al.  Configuration space analysis of travelling salesman problems , 1985 .

[22]  Jean-Luc Lutton,et al.  The asymptotic behaviour of quadratic sum assignment problems: A statistical mechanics approach , 1986 .

[23]  M R Chance,et al.  Linkage of functional and structural heterogeneity in proteins: dynamic hole burning in carboxymyoglobin. , 1987, Science.

[24]  Edoardo Amaldi,et al.  Stability-Capacity Diagram of a Neural Network with Ising Bonds , 1989 .

[25]  R. Palmer,et al.  Models of hierarchically constrained dynamics for glassy relaxation , 1984 .

[26]  H Frauenfelder,et al.  Dynamics of ligand binding to myoglobin. , 1975, Biochemistry.

[27]  Richard M. Karp,et al.  Probabilistic Analysis of Partitioning Algorithms for the Traveling-Salesman Problem in the Plane , 1977, Math. Oper. Res..

[28]  J. Edmonds.  Paths, Trees, and Flowers , 1965, Canadian Journal of Mathematics.

[29]  Claude-Nicolas Fiechter,et al.  A Parallel Tabu Search Algorithm for Large Traveling Salesman Problems , 1994, Discret. Appl. Math..

[30]  Takayuki Hirata,et al.  A correlation between the b value and the fractal dimension of earthquakes , 1989 .

[31]  Steve R. White,et al.  Configuration Space Analysis for Optimization Problems , 1986 .

[32]  Hogg,et al.  Dynamics of computational ecosystems. , 1989, Physical review. A, General physics.

[33]  José F. Fontanari,et al.  Landscape statistics of the binary perceptron , 1990 .

[34]  B A Huberman,et al.  Ultradiffusion: the relaxation of hierarchical systems , 1985 .

[35]  K. Wüthrich.  Protein structure determination in solution by nuclear magnetic resonance spectroscopy. , 1989, Science.

[36]  David Sankoff,et al.  Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison , 1983 .

[37]  Wille Lt.  Intractable computations without local minima. , 1987 .

[38]  J. Fontanari,et al.  Stochastic versus deterministic update in simulated annealing , 1990 .

[39]  N Go,et al.  Structural basis of hierarchical multiple substates of a protein. I: Introduction , 1989, Proteins.

[40]  J. Stephen Judd,et al.  Neural network design and the complexity of learning , 1990, Neural network modeling and connectionism.

[41]  Bernard Derrida,et al.  Multivalley structure in Kauffman's model: analogy with spin glasses , 1986 .

[42]  Bernard Derrida,et al.  Valleys and overlaps in Kauffman's model , 1987 .

[43]  Dana S. Richards,et al.  Punctuated Equilibria: A Parallel Genetic Algorithm , 1987, ICGA.

[44]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[45]  V. F. Pisarenko,et al.  Multiscale models of failure and percolation , 1990 .

[46]  E. D. Weinberger,et al.  The NK model of rugged fitness landscapes and its application to maturation of the immune response. , 1989, Journal of theoretical biology.

[47]  Bernardo A. Huberman,et al.  The performance of cooperative processes , 1990 .

[48]  H. Frauenfelder,et al.  Function and Dynamics of Myoglobin a , 1987, Annals of the New York Academy of Sciences.

[49]  D. Werra,et al.  Tabu search: a tutorial and an application to neural networks , 1989 .

[50]  Richard M. Karp,et al.  A Patching Algorithm for the Nonsymmetric Traveling-Salesman Problem , 1979, SIAM J. Comput..

[51]  Michael O. Ball,et al.  The design and analysis of heuristics , 1981, Networks.

[52]  Vladimir Keilis-Borok,et al.  Introduction: Non-linear systems in the problem of earthquake prediction , 1990 .

[53]  Jadranka Skorin-Kapov,et al.  Tabu Search Applied to the Quadratic Assignment Problem , 1990, INFORMS J. Comput..

[54]  Champion,et al.  Spectral broadening in biomolecules. , 1986, Physical review letters.

[55]  Gary S. Grest,et al.  Monte Carlo and mean field slow cooling simulations for spin glasses: relation to NP-completeness , 1987 .

[56]  H. Frauenfelder,et al.  Conformational substates in proteins. , 1988, Annual review of biophysics and biophysical chemistry.

[57]  Bernard Derrida,et al.  Barrier heights in the Kauffman model , 1989 .

[58]  Gérard Toulouse.  How ‘Frustration’ Set In , 1989 .

[59]  Néstor Parga.  Overlap distributions and taxonomy analysis of spin glass states with equal weights , 1987 .

[60]  N Agmon,et al.  Reactive line-shape narrowing in low-temperature inhomogeneous geminate recombination of CO to myoglobin. , 1988, Biochemistry.

[61]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[62]  F. Stillinger,et al.  Packing Structures and Transitions in Liquids and Solids , 1984, Science.

[63]  M. Padberg,et al.  Addendum: Optimization of a 532-city symmetric traveling salesman problem by branch and cut , 1990 .

[64]  Gary S. Grest,et al.  Irreversibility and metastability in spin-glasses. I. Ising model , 1983 .

[65]  Gerard Toulouse,et al.  Theory of the frustration effect in spin glasses: I , 1986 .

[66]  T. Gingeras,et al.  Steps toward computer analysis of nucleotide sequences. , 1980, Science.

[67]  S. Kauffman,et al.  Towards a general theory of adaptive walks on rugged landscapes. , 1987, Journal of theoretical biology.

[68]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[69]  Tad Hogg,et al.  Phase Transitions in Artificial Intelligence Systems , 1987, Artif. Intell..

[70]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[71]  Peter G. Wolynes,et al.  The aperiodic crystal picture and free energy barriers in glasses , 1987 .

[72]  G A Petsko,et al.  Fluctuations in protein structure from X-ray diffraction. , 1984, Annual review of biophysics and bioengineering.

[73]  J. P. Secrétan,et al.  Der Saccus endolymphaticus bei Entzündungsprozessen , 1944 .

[74]  Gol'danskiĭ Vi,et al.  [Tunneling between quasi-degenerate conformational states and the low-temperature thermal capacity of biopolymers. A glass-like protein model]. , 1983 .

[75]  David J. Lockhart,et al.  Nonphotochemical holeburning in a protein matrix: Chlorophyllide in apomyoglobin , 1987 .

[76]  M. Karplus,et al.  Multiple conformational states of proteins: a molecular dynamics analysis of myoglobin. , 1987, Science.

[77]  Søren Brunak,et al.  A Travelling Salesman Approach to Protein Conformation , 1989, Complex Syst..

[78]  Bernard Manderick,et al.  The Genetic Algorithm and the Structure of the Fitness Landscape , 1991, ICGA.

[79]  Michael S. Waterman,et al.  General methods of sequence comparison , 1984 .

[80]  H. Keller,et al.  Evidence for Conformational and Diffusional Mean Square Displacements in Frozen Aqueous Solution of Oxymyoglobin , 1980 .

[81]  M. Kimura,et al.  The neutral theory of molecular evolution. , 1983, Scientific American.

[82]  Frauenfelder,et al.  Glassy behavior of a protein. , 1989, Physical review letters.

[83]  Pál Ormos,et al.  Proteins and pressure , 1990 .

[84]  L. Darrell Whitley,et al.  Scheduling Problems and Traveling Salesmen: The Genetic Edge Recombination Operator , 1989, International Conference on Genetic Algorithms.

[85]  Tad Hogg,et al.  Complexity and adaptation , 1986 .

[86]  Friedrich,et al.  Conformational barriers in low-temperature proteins and glasses. , 1988, Physical review. A, General physics.

[87]  Ron Elber,et al.  A method for determining reaction paths in large molecules: application to myoglobin , 1987 .