Dynamical System Approaches to Combinatorial Optimization

This article describes and compares several dynamical system approaches to combinatorial optimization problems. These include penalty methods, the approach of Hopfield and Tank, self-organizing maps, i.e., Kohonen networks, coupled selection equations, and hybrid methods. Many of them are investigated analytically and the costs of the solutions are compared numerically with those of solutions obtained by simulated annealing and the costs of a global optimal solution. In order to get reproducible simulation results, a pseudo-random number generator with integer arithmetic is used to produce the data sets.

[1]  Kaleem Siddiqi,et al.  Matching Hierarchical Structures Using Association Graphs , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Mordecai Avriel,et al.  Nonlinear programming , 1976 .

[3]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[4]  Hermann Haken Decision Making and Optimization in Regional Planning , 1998 .

[5]  P. Pardalos,et al.  Handbook of Combinatorial Optimization , 1998 .

[6]  W. Ebeling,et al.  Self-Organization, Valuation and Optimization , 1994 .

[7]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[8]  Toshio Fukuda,et al.  Cellular Robotics and Micro Robotic Systems , 1994, World Scientific Series in Robotics and Intelligent Systems.

[9]  Vladimir I. Arnold,et al.  Dynamical Systems I , 1988 .

[10]  Rainer E. Burkard,et al.  Methoden der ganzzahligen Optimierung , 1972 .

[11]  S. Kirkpatrick,et al.  Configuration space analysis of travelling salesman problems , 1985 .

[12]  Toshio Fukuda,et al.  Approach to the dynamically reconfigurable robotic system , 1988, J. Intell. Robotic Syst..

[13]  Kamgar-Parsi On problem solving with Hopfield neural networks , 1989 .

[14]  David E. van den Bout,et al.  A traveling salesman objective function that works , 1988, IEEE 1988 International Conference on Neural Networks.

[15]  Jacek Klinowski,et al.  Taboo Search: An Approach to the Multiple Minima Problem , 1995, Science.

[16]  R. Brockett,et al.  Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[17]  Michael Schanz,et al.  Self-Organized Behaviour of Distributed Autonomous Mobile Robotic Systems by Pattern Formation Principles , 1998, DARS.

[18]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[19]  D. Psaltis,et al.  Holography in artificial neural networks , 1990, Nature.

[20]  W. Banzhaf,et al.  A new dynamical approach to the travelling salesman problem , 1989 .

[21]  Kazuo Tsuchiya,et al.  A deterministic annealing algorithm for a combinatorial optimization problem by the use of replicator equations , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[22]  James A. Anderson,et al.  Neurocomputing (vol. 2): directions for research , 1990 .

[23]  Shun-ichi Amari,et al.  Mathematical foundations of neurocomputing , 1990, Proc. IEEE.

[24]  S. Wiggins Introduction to Applied Nonlinear Dynamical Systems and Chaos , 1989 .

[25]  H. Haken,et al.  Associative memory of a dynamical system: the example of the convection instability , 1991 .

[26]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[27]  R. Mishra,et al.  Self-Organization , 2021, Encyclopedic Dictionary of Archaeology.

[28]  M. Eigen,et al.  The hypercycle. A principle of natural self-organization. Part A: Emergence of the hypercycle. , 1977, Die Naturwissenschaften.

[29]  Hermann Haken Pattern Formation and Pattern Recognition — An Attempt at a Synthesis , 1979 .

[30]  Stephen D. Collins Neurocomputing 2 , 1992, Neurology.

[31]  C. Robinson Dynamical Systems: Stability, Symbolic Dynamics, and Chaos , 1994 .

[32]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[33]  Fred W. Glover,et al.  A user's guide to tabu search , 1993, Ann. Oper. Res..

[34]  Nostrand Reinhold,et al.  the utility of using the genetic algorithm approach on the problem of Davis, L. (1991), Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York. , 1991 .

[35]  Behrooz Kamgar-Parsi,et al.  On problem solving with Hopfield neural networks , 1990, International 1989 Joint Conference on Neural Networks.

[36]  Professor Dr. Dr. h.c. Hermann Haken,et al.  Synergetic Computers and Cognition , 1991, Springer Series in Synergetics.

[37]  G. Conte,et al.  New Trends in Systems Theory , 1991 .

[38]  Peter Spellucci,et al.  Numerische Verfahren der nichtlinearen Optimierung , 1993 .

[39]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Wing Shing Wong,et al.  Matrix representation and gradient flows for NP-hard problems , 1995 .

[41]  V. Arnold,et al.  Ordinary Differential Equations , 1973 .

[42]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[43]  M. Hirsch,et al.  Differential Equations, Dynamical Systems, and Linear Algebra , 1974 .

[44]  Oppo,et al.  Pattern formation in a liquid-crystal light valve with feedback, including polarization, saturation, and internal threshold effects. , 1995, Physical review. A, Atomic, molecular, and optical physics.

[45]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[46]  J. Henry,et al.  System Modelling and Optimization: Proceedings of the 16th IFIP-TC7 Conference, Compiègne, France, July 5-9, 1993 , 1994, System Modelling and Optimization.

[47]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[48]  Johannes Terno,et al.  Numerik der Optimierung , 1993 .

[49]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[50]  R. Brockett,et al.  A Gradient Flow for the Assignment Problem , 1991 .

[51]  Carsten Peterson,et al.  Neural optimization , 1998 .

[52]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[53]  Josef Hofbauer,et al.  The theory of evolution and dynamical systems , 1988 .

[54]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[55]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[56]  E. Polak Introduction to linear and nonlinear programming , 1973 .

[57]  Bill Baird,et al.  Bifurcation and category learning in network models of oscilating cortex , 1990 .

[58]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[59]  Jens Starke Cost oriented competing processes — a new handling of assignment problems , 1996 .

[60]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[61]  Scott Kirkpatrick,et al.  Optimization by simulated annealing: Quantitative studies , 1984 .

[62]  Roland Thord,et al.  Knowledge and Networks in a Dynamic Economy , 1998 .

[63]  H. Haken Pattern Formation by Dynamic Systems and Pattern Recognition , 1979, Springer Series in Synergetics.

[64]  R. Graham,et al.  Handbook of Combinatorics , 1995 .

[65]  Frank Schweitzer,et al.  Self-Organization of Complex Structures: From Individual to Collective Dynamics - Some Introductory , 1997 .

[66]  Marcello Pelillo,et al.  Replicator Equations, Maximal Cliques, and Graph Isomorphism , 1998, Neural Computation.

[67]  Satoshi Matsuda Theoretical considerations on the capabilities of crossbar switching by Hopfield networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[68]  K. Binder,et al.  Spin glasses: Experimental facts, theoretical concepts, and open questions , 1986 .

[69]  Demetri Psaltis,et al.  Optical Neural Computers , 1987, Topical Meeting on Optical Computing.

[70]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[71]  Bernard Angéniol,et al.  Self-organizing feature maps and the travelling salesman problem , 1988, Neural Networks.

[72]  Andrew Kusiak,et al.  Flexible Manufacturing Systems: A Structural Approach , 1985 .

[73]  Gene A. Tagliarini,et al.  Optimization Using Neural Networks , 1991, IEEE Trans. Computers.

[74]  Yu. Nesterov,et al.  Interior-point methods: An old and new approach to nonlinear programming , 1997, Math. Program..

[75]  Werner Ebeling,et al.  Physik der Evolutionsprozesse , 1990 .

[76]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[77]  Kate A. Smith,et al.  Neural Networks for Combinatorial Optimization: a Review of More Than a Decade of Research , 1999 .

[78]  Panos M. Pardalos,et al.  The maximum clique problem , 1994, J. Glob. Optim..

[79]  H. Haken,et al.  Treatment of combinatorial optimization problems using selection equations with cost terms. Part II. NP -hard three-dimensional assignment problems , 1999 .

[80]  Kiichi Urahama Analog circuit for solving assignment problems , 1994 .

[81]  Immanuel M. Bomze,et al.  Evolution towards the Maximum Clique , 1997, J. Glob. Optim..

[82]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[83]  Angel P. del Pobil,et al.  Practical Motion Planning in Robotics: Current Approaches and Future Directions , 1998 .

[84]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[85]  Francis E. H. Tay Contingency management in flexible manufacturing systems using modal state logic , 1999 .

[86]  R A Brooks,et al.  New Approaches to Robotics , 1991, Science.

[87]  H. P. Schwefel,et al.  Numerische Optimierung von Computermodellen mittels der Evo-lutionsstrategie , 1977 .

[88]  James A. Anderson,et al.  Neurocomputing: Foundations of Research , 1988 .

[89]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[90]  Wolfgang Kinzel,et al.  Spin Glasses and Memory , 1987 .

[91]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[92]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[93]  Konstantinos Paparrizos,et al.  A Dual Forest Algorithm for the Assignment Problem , 1990, Applied Geometry And Discrete Mathematics.

[94]  S. Kawasaki,et al.  Springer Verlag, Berlin, Heidelberg, New York (1995) , 1996 .

[95]  M W Hirsch,et al.  Computing with dynamic attractors in neural networks. , 1995, Bio Systems.

[96]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[97]  I ScottKirkpatrick Optimization by Simulated Annealing: Quantitative Studies , 1984 .

[98]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[99]  Andrew H. Gee,et al.  An analytical framework for optimizing neural networks , 1993, Neural Networks.

[100]  Alan L. Yuille,et al.  Constrained optimization and the elastic net , 1998 .

[101]  Berndt Müller,et al.  Neural networks: an introduction , 1990 .

[102]  Satoshi Matsuda Stability of solutions in hopfield neural network , 1995, Systems and Computers in Japan.

[103]  M. Grötschel,et al.  Combinatorial optimization , 1996 .

[104]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[105]  Yoshinori Uesaka MATHEMATICAL ASPECTS OF NEURO-DYNAMICS FOR COMBINATORIAL OPTIMIZATION , 1991 .