Hopfield Network as Static Optimizer: Learning the Weights and Eliminating the Guesswork

This article presents a simulation study for validation of an adaptation methodology for learning weights of a Hopfield neural network configured as a static optimizer. The quadratic Liapunov function associated with the Hopfield network dynamics is leveraged to map the set of constraints associated with a static optimization problem. This approach leads to a set of constraint-specific penalty or weighting coefficients whose values need to be defined. The methodology leverages a learning-based approach to define values of constraint weighting coefficients through adaptation. These values are in turn used to compute values of network weights, effectively eliminating the guesswork in defining weight values for a given static optimization problem, which has been a long-standing challenge in artificial neural networks. The simulation study is performed using the Traveling Salesman problem from the domain of combinatorial optimization. Simulation results indicate that the adaptation procedure is able to guide the Hopfield network towards solutions of the problem starting with random values for weights and constraint weighting coefficients. At the conclusion of the adaptation phase, the Hopfield network acquires weight values which readily position the network to search for local minimum solutions. The demonstrated successful application of the adaptation procedure eliminates the need to guess or predetermine the values for weights of the Hopfield network.

[1]  Marcus Gallagher,et al.  Multi-layer Perceptron Error Surfaces: Visualization, Structure and Modelling , 2000 .

[2]  Gursel Serpen Managing spatio-temporal complexity in Hopfield neural , 2003 .

[3]  Geoffrey E. Hinton,et al.  Keeping Neural Networks Simple , 1993 .

[4]  Jirí Síma Energy-Based Computation with Symmetric Hopfield Nets , 2007 .

[5]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[6]  Gürsel Serpen Adaptive Hopfield Network , 2003, ICANN.

[7]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[8]  Kate Smith-Miles,et al.  Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research , 1999, INFORMS J. Comput..

[9]  Kate A. Smith,et al.  Neural Networks for Combinatorial Optimization: a Review of More Than a Decade of Research , 1999 .

[10]  Gürsel Serpen,et al.  Determination of weights for relaxation recurrent neural networks , 2000, Neurocomputing.

[11]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[12]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[13]  Pekka Orponen,et al.  Continuous-Time Symmetric Hopfield Nets Are Computationally Universal , 2003, Neural Computation.

[14]  Olivier Bournez,et al.  A Survey on Continuous Time Computations , 2009, ArXiv.

[15]  Gürsel Serpen,et al.  On the performance of Hopfield network for graph search problem , 1997, Neurocomputing.

[16]  Gonzalo Joya,et al.  Dynamical analysis of continuous higher-order hopfield networks for combinatorial optimization. , 2005, Neural computation.

[17]  S. Abe Global convergence and suppression of spurious states of the Hopfield neural networks , 1993 .

[18]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[19]  S. Abe Theories on the Hopfield neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[20]  G. Serpen,et al.  A Heuristic and Its Mathematical Analogue within Artificial Neural Network Adaptation Context , 2005 .

[21]  Lars Kai Hansen,et al.  Massive Weight Sharing: A Cure For Extremely Ill-Posed Problems , 1994 .

[22]  David L. Livingston,et al.  AN ADAPTIVE CONSTRAINT SATISFACTION NETWORK , 1990, 1990 Conference Record Twenty-Fourth Asilomar Conference on Signals, Systems and Computers, 1990..

[23]  Pekka Orponen,et al.  General-Purpose Computation with Neural Networks: A Survey of Complexity Theoretic Results , 2003, Neural Computation.

[24]  Sergey Ablameyko,et al.  Limitations and Future Trends in Neural Computation , 2003 .

[25]  Francisco Sandoval Hernández,et al.  Dynamical Analysis of Continuous Higher-Order Hopfield Networks for Combinatorial Optimization , 2005, Neural Computation.

[26]  Kate A. Smith,et al.  Neural network models for combinatorial optimization: a survey of deterministic, stochastic and chaotic approaches , 2002 .

[27]  Shigeo Abe,et al.  Convergence acceleration of the Hopfield neural network by optimizing integration step sizes , 1996, IEEE Trans. Syst. Man Cybern. Part B.