Convergence Properties of the Softassign Quadratic Assignment Algorithm

The softassign quadratic assignment algorithm is a discrete-time, continuous-state, synchronous updating optimizing neural network. While its effectiveness has been shown in the traveling salesman problem, graph matching, and graph partitioning in thousands of simulations, its convergence properties have not been studied. Here, we construct discrete-time Lyapunov functions for the cases of exact and approximate doubly stochastic constraint satisfaction, which show convergence to a fixed point. The combination of good convergence properties and experimental success makes the softassign algorithm an excellent choice for neural quadratic assignment optimization.

[1]  Richard Sinkhorn.  A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[2]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[3]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Westervelt,et al.  Dynamics of iterated-map neural networks. , 1989, Physical review. A, General physics.

[5]  Carsten Peterson,et al.  A New Method for Mapping Optimization Problems Onto Neural Networks , 1989, Int. J. Neural Syst..

[6]  Eric Goles Ch.,et al.  Energy Functions in Neural Networks with Continuous Local Functions , 1989, Complex Systems.

[7]  John S. Bridle,et al.  Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters , 1989, NIPS.

[8]  David E. van den Bout,et al.  Graph partitioning using annealed neural networks , 1990, International 1989 Joint Conference on Neural Networks.

[9]  Christoph von der Malsburg,et al.  Network self-organization , 1990 .

[10]  Eric Mjolsness,et al.  Algebraic transformations of objective functions , 1990, Neural Networks.

[11]  B. Beavis,et al.  Optimization and Stability Theory for Economic Analysis: DYNAMICS AND STABILITY , 1990 .

[12]  Petar D. Simic,et al.  Statistical mechanics as the underlying theory of ‘elastic’ and ‘neural’ optimisations , 1990 .

[13]  Petar D. Simic.  Constrained Nets for Graph Matching and Other Quadratic Assignment Problems , 1991, Neural Comput..

[14]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[15]  Xin Wang,et al.  Stability of fixed points and periodic orbits and bifurcations in analog neural networks , 1992, Neural Networks.

[16]  S. Barnett,et al.  Analog neural networks with local competition. II: Application to associative memory , 1993 .

[17]  Waugh,et al.  Analog neural networks with local competition. I. Dynamics and stability. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[18]  Andrew H. Gee,et al.  An analytical framework for optimizing neural networks , 1993, Neural Networks.

[19]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[20]  Alan L. Yuille,et al.  The invisible hand algorithm: Solving the assignment problem with statistical physics , 1994, Neural Networks.

[21]  Andrew H. Gee,et al.  Polyhedral Combinatorics and Neural Networks , 1994, Neural Computation.

[22]  Pascal Koiran.  Dynamics of Discrete Time, Continuous State Hopfield Networks , 1994, Neural Computation.

[23]  W. Wolfe,et al.  Hopfield-style neural networks and the TSP , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[24]  J. J. Kosowsky,et al.  Statistical Physics Algorithms That Converge , 1994, Neural Computation.

[25]  K. Urahama.  Mathematical programming formulations for neural combinatorial optimization algorithms , 1995 .

[26]  Manfred K. Warmuth,et al.  Additive versus exponentiated gradient updates for linear prediction , 1995, STOC '95.

[27]  Eric Mjolsness,et al.  A Novel Optimizing Network Architecture with Applications , 1996, Neural Computation.

[28]  James S. Duncan,et al.  A Robust Point Matching Algorithm for Autoradiograph Alignment , 1996, VBC.

[29]  Steven Gold,et al.  Softmax to Softassign: neural network algorithms for combinatorial optimization , 1996 .

[30]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Alan L. Yuille,et al.  A Convergence Proof for the Softassign Quadratic Assignment Algorithm , 1996, NIPS.

[32]  Manfred K. Warmuth,et al.  Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[33]  Xin Wang,et al.  Absence of Cycles in Symmetric Neural Networks , 1998, Neural Computation.

[34]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .