Self Annealing: Unifying Deterministic Annealing and Relaxation Labelling

Deterministic annealing and relaxation labeling algorithms for classification and matching are presented and discussed. A new approach —self annealing—is introduced to bring deterministic annealing and relaxation labeling into accord. Self annealing results in an emergent linear schedule for winner-take-all and assignment problems. Also, the relaxation labeling algorithm can be seen as an approximation to the self annealing algorithm for matching and labeling problems.

[1]  Marcello Pelillo,et al.  Learning Compatibility Coefficients for Relaxation Labeling Processes , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  John S. Bridle,et al.  Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters , 1989, NIPS.

[3]  Larry S. Davis,et al.  Shape Matching Using Relaxation Techniques , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shmuel Peleg,et al.  A New Probabilistic Relaxation Scheme , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Carsten Peterson,et al.  A New Method for Mapping Optimization Problems Onto Neural Networks , 1989, Int. J. Neural Syst..

[6]  Eric Mjolsness,et al.  A Lagrangian relaxation network for graph matching , 1996, IEEE Trans. Neural Networks.

[7]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[8]  Keith E. Price,et al.  Relaxation Matching Techniques-A Comparison , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Alan L. Yuille,et al.  The invisible hand algorithm: Solving the assignment problem with statistical physics , 1994, Neural Networks.

[10]  Eric Mjolsness,et al.  A Novel Optimizing Network Architecture with Applications , 1996, Neural Computation.

[11]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[13]  Kiichi Urahama,et al.  Gradient Projection Network: Analog Solver for Linearly Constrained Nonlinear Programming , 1996, Neural Computation.

[14]  James S. Duncan,et al.  A Robust Point Matching Algorithm for Autoradiograph Alignment , 1996, VBC.

[15]  Petar D. Simic,et al.  Statistical mechanics as the underlying theory of ‘elastic’ and ‘neural’ optimisations , 1990 .

[16]  Joachim M. Buhmann,et al.  Central and Pairwise Data Clustering by Competitive Neural Networks , 1993, NIPS.

[17]  Marcello Pelillo,et al.  On the dynamics of relaxation labeling processes , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[18]  Petar D. Simic.  Constrained Nets for Graph Matching and Other Quadratic Assignment Problems , 1991, Neural Comput..

[19]  Richard Sinkhorn.  A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[20]  G. Parisi,et al.  Statistical Field Theory , 1988 .

[21]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[22]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[23]  B. Beavis,et al.  Optimization and Stability Theory for Economic Analysis: DYNAMICS AND STABILITY , 1990 .

[24]  K. Urahama.  Mathematical programming formulations for neural combinatorial optimization algorithms , 1995 .

[25]  Azriel Rosenfeld,et al.  Scene Labeling by Relaxation Operations , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[26]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Andrew H. Gee,et al.  Polyhedral Combinatorics and Neural Networks , 1994, Neural Computation.

[28]  Manfred K. Warmuth,et al.  Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[29]  Waugh,et al.  Analog neural networks with local competition. I. Dynamics and stability. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[30]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[31]  Alan L. Yuille,et al.  A Convergence Proof for the Softassign Quadratic Assignment Algorithm , 1996, NIPS.

[32]  J. J. Kosowsky,et al.  Statistical Physics Algorithms That Converge , 1994, Neural Computation.

[33]  Josef Kittler,et al.  Discrete relaxation , 1990, Pattern Recognit..

[34]  Geoffrey C. Fox,et al.  Constrained Clustering as an Optimization Method , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  William J. Christmas,et al.  Structural Matching in Computer Vision Using Probabilistic Relaxation , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Olivier D. Faugeras,et al.  Improving Consistency and Reducing Ambiguity in Stochastic Labeling: An Optimization Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  L. Kitchen.  Discrete Relaxation for Matching Relational Structures , 1978 .

[39]  Azriel Rosenfeld,et al.  Point pattern matching by relaxation , 1980, Pattern Recognit..

[40]  W. Wolfe,et al.  Hopfield-style neural networks and the TSP , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[41]  Kamgar-Parsi.  On problem solving with Hopfield neural networks , 1989 .

[42]  David E. van den Bout,et al.  Graph partitioning using annealed neural networks , 1990, International 1989 Joint Conference on Neural Networks.

[43]  Alan L. Yuille,et al.  Generalized Deformable Models, Statistical Physics, and Matching Problems , 1990, Neural Computation.

[44]  Alan L. Yuille,et al.  Statistical Physics, Mixtures of Distributions, and the EM Algorithm , 1994, Neural Computation.

[45]  Marc Berthod,et al.  Scene labeling: an optimization approach , 1980, Pattern Recognit..