The Computational Power of Discrete Hopfield Nets with Hidden Units

We prove that polynomial size discrete Hopfield networks with hidden units compute exactly the class of Boolean functions PSPACE/poly, i.e., the same functions as are computed by polynomial space-bounded nonuniform Turing machines. As a corollary to the construction, we observe also that networks with polynomially bounded interconnection weights compute exactly the class of functions P/poly, i.e., the class computed by polynomial time-bounded nonuniform Turing machines.

[1]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[2]  Eric Goles Ch.,et al.  Decreasing energy functions as a tool for studying threshold networks , 1985, Discret. Appl. Math..

[3]  James A. Anderson,et al.  Neurocomputing: Foundations of Research , 1988 .

[4]  Martin Hasler,et al.  Recursive neural networks for associative memory , 1990, Wiley-interscience series in systems and optimization.

[5]  Mihalis Yannakakis,et al.  Simple Local Search Problems That are Hard to Solve , 1991, SIAM J. Comput..

[6]  Ian Parberry,et al.  Circuit complexity and neural networks , 1994 .

[7]  José L. Balcázar,et al.  Structural complexity 1 , 1988 .

[8]  Ingo Wegener,et al.  The complexity of Boolean functions , 1987 .

[9]  S C Kleene,et al.  Representation of Events in Nerve Nets and Finite Automata , 1951 .

[10]  Hava T. Siegelmann,et al.  Analog computation via neural networks , 1993, [1993] The 2nd Israel Symposium on Theory and Computing Systems.

[11]  Don R. Hush,et al.  Bounds on the complexity of recurrent neural network implementations of finite state machines , 1993, Neural Networks.

[12]  Maurice Tchuente.  Sequential Simulation of Parallel Iterations and Applications , 1986, Theor. Comput. Sci..

[13]  Eric Goles Ch.,et al.  The Convergence of Symmetric Threshold Automata , 1981, Inf. Control..

[14]  Automata Networks , 1986, Lecture Notes in Computer Science.

[15]  José L. Balcázar,et al.  Structural Complexity I , 1995, Texts in Theoretical Computer Science An EATCS Series.

[16]  Yves Robert,et al.  Automata networks in computer science : theory and applications , 1987 .

[17]  Eric Goles,et al.  Neural and automata networks , 1990 .

[18]  Eric Goles Ch.,et al.  Transient length in sequential iteration of threshold functions , 1983, Discret. Appl. Math..

[19]  Eric Goles Ch.,et al.  Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating , 1989, Complex Syst..

[20]  E. Goles,et al.  Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating , 1989, Complex Syst..

[21]  Pekka Orponen,et al.  Computing with Truly Asynchronous Threshold Logic Networks , 1997, Theor. Comput. Sci..

[22]  Piotr Indyk.  Optimal Simulation of Automata by Neural Nets , 1995, STACS.

[23]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[24]  E. Goles.  Fixed Point Behavior of Threshold Functions on a Finite Set , 1982 .

[25]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[26]  Pekka Orponen,et al.  Complexity Issues in Discrete Hopfield Networks , 1994 .

[27]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Michael Luby,et al.  Steepest Descent Can Take Exponential Time for Symmetric Connection Networks , 1988, Complex Syst..

[29]  Joseph W. Goodman,et al.  A generalized convergence theorem for neural networks , 1988, IEEE Trans. Inf. Theory.

[30]  Svatopluk Poljak,et al.  On periodical behaviour in societies with symmetric influences , 1983, Comb..

[31]  Hava T. Siegelmann,et al.  Computational power of neural networks: a characterization in terms of Kolmogorov complexity , 1997, IEEE Trans. Inf. Theory.

[32]  Marvin Minsky,et al.  Computation : finite and infinite machines , 2016 .

[33]  José L. Balcázar,et al.  On Characterizations of the Class PSPACE/POLY , 1987, Theor. Comput. Sci..