On the Computational Complexity of Binary and Analog Symmetric Hopfield Nets

We investigate the computational properties of finite binary- and analog-state discrete-time symmetric Hopfield nets. For binary networks, we obtain a simulation of convergent asymmetric networks by symmetric networks with only a linear increase in network size and computation time. Then we analyze the convergence time of Hopfield nets in terms of the length of their bit representations. Here we construct an analog symmetric network whose convergence time exceeds the convergence time of any binary Hopfield net with the same representation length. Further, we prove that the MIN ENERGY problem for analog Hopfield nets is NP-hard and provide a polynomial time approximation algorithm for this problem in the case of binary nets. Finally, we show that symmetric analog nets with an external clock are computationally Turing universal.

[1]  John S. Denker,et al.  Neural Networks for Computing , 1998 .

[2]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[3]  Svatopluk Poljak,et al.  On periodical behaviour in societies with symmetric influences , 1983, Comb..

[4]  Peter Kwong-Shun Tam,et al.  A Neural Network Methodology of Quadratic Optimization , 1999, Int. J. Neural Syst..

[5]  Pekka Orponen,et al.  Some Afterthoughts on Hopfield Networks , 1999, SOFSEM.

[6]  Don R. Hush,et al.  Bounds on the complexity of recurrent neural network implementations of finite state machines , 1993, Neural Networks.

[7]  Mike Casey,et al.  The Dynamics of Discrete-Time Computation, with Application to Recurrent Neural Networks and Finite State Machine Extraction , 1996, Neural Computation.

[8]  Ramesh Hariharan,et al.  Derandomizing semidefinite programming based approximation algorithms , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[9]  Bing J. Sheu,et al.  Hardware annealing for fast retrieval of optimal solutions in Hopfield neural networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[10]  János Komlós,et al.  Convergence results in an associative memory model , 1988, Neural Networks.

[11]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[12]  Hava T. Siegelmann,et al.  Analog computation via neural networks , 1993, [1993] The 2nd Israel Symposium on Theory and Computing Systems.

[13]  Eric Goles Ch.,et al.  Decreasing energy functions as a tool for studying threshold networks , 1985, Discret. Appl. Math..

[14]  Jirí Wiedermann,et al.  Complexity Issues in Discrete Neurocomputing , 1990, IMYCS.

[15]  Jacek Mandziuk.  Optimization with the Hopfield network based on correlated noises: Experimental approach , 2000, Neurocomputing.

[16]  S. N. Srihari,et al.  Neural network models and their application to handwritten digit recognition , 1988, IEEE 1988 International Conference on Neural Networks.

[17]  T. Kailath,et al.  Discrete Neural Computation: A Theoretical Foundation , 1995 .

[18]  Jirí Síma.  Hopfield Languages , 1995, SOFSEM.

[19]  Pekka Orponen,et al.  Complexity Issues in Discrete Hopfield Networks , 1994 .

[20]  David S. Johnson,et al.  Some Simplified NP-Complete Graph Problems , 1976, Theor. Comput. Sci..

[21]  Ian Parberry,et al.  Circuit complexity and neural networks , 1994 .

[22]  Piotr Indyk.  Optimal Simulation of Automata by Neural Nets , 1995, STACS.

[23]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Pekka Orponen,et al.  The Computational Power of Discrete Hopfield Nets with Hidden Units , 1996, Neural Computation.

[25]  Pascal Koiran.  Dynamics of Discrete Time, Continuous State Hopfield Networks , 1994, Neural Computation.

[26]  E. Goles,et al.  Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating , 1989, Complex Syst..

[27]  Hava T. Siegelmann,et al.  Computational power of neural networks: a characterization in terms of Kolmogorov complexity , 1997, IEEE Trans. Inf. Theory.

[28]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[29]  Eduardo Sontag,et al.  Computational power of neural networks , 1995 .

[30]  J. R,et al.  Analog Stable Simulation of Discrete Neural Networks , 1997 .

[31]  Paola Campadelli,et al.  Approximability of the Ground State Problem for Certain Ising Spin Glasses , 1997, J. Complex..

[32]  Daihee Park,et al.  On the Design of BSB Neural Associative Memories Using Semidefinite Programming , 1999, Neural Computation.

[33]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[34]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[35]  Ingo Wegener,et al.  The complexity of Boolean functions , 1987 .

[36]  Hava T. Siegelmann,et al.  Neural networks and analog computation - beyond the Turing limit , 1999, Progress in theoretical computer science.

[37]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[38]  Paola Campadelli,et al.  On the approximability of the energy function of Ising spin glasses , 1994 .

[39]  Martín Matamala,et al.  Symmetric Discrete Universal Neural Networks , 1996, Theor. Comput. Sci..

[40]  Eric Goles Ch.,et al.  Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating , 1989, Complex Syst..

[41]  Pekka Orponen,et al.  On the Effect of Analog Noise in Discrete-Time Analog Computations , 1996, Neural Computation.

[42]  Patrik Floréen.  Worst-case convergence times for Hopfield memories , 1991, IEEE Trans. Neural Networks.

[43]  Tzi-Dar Chiueh,et al.  A Neural Network Classifier Based on Coding Theory , 1987, NIPS.

[44]  Pekka Orponen,et al.  A Continuous-Time Hopfield Net Simulation of Discrete Neural Networks , 2000 .

[45]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.

[46]  Emile H. L. Aarts,et al.  Simulated annealing and Boltzmann machines - a stochastic approach to combinatorial optimization and neural computing , 1990, Wiley-Interscience series in discrete mathematics and optimization.

[47]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[48]  Jirí Wiedermann,et al.  Theory of neuromata , 1998, JACM.

[49]  José L. Balcázar,et al.  Structural Complexity I , 1995, Texts in Theoretical Computer Science An EATCS Series.

[50]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[51]  Bernard Gold.  Hopfield model applied to vowel and consonant discrimination , 1986 .

[52]  L. Bieche,et al.  On the ground states of the frustration model of a spin glass by a matching method of graph theory , 1980 .