Complexity Issues in Discrete Neurocomputing

An overview of the basic results in complexity theory of discrete neural computations is presented. Especially, the computational power and efficiency of single neurons, neural circuits, symmetric neural networks (Hopfield model), and of Boltzmann machines is investigated and characterized. Corresponding intractability results are mentioned as well. The evidence is presented why discrete neural networks (inclusively Boltzmann machines) are not to be expected to solve intractable problems more efficiently than other conventional models of computing.

[1]  Peter van Emde Boas,et al.  Machine Models and Simulation , 1990, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[2]  Marvin Minsky,et al.  Computation : finite and infinite machines , 2016 .

[3]  Georg Schnitger,et al.  Parallel Computation with Threshold Functions , 1988, J. Comput. Syst. Sci..

[4]  Emile H. L. Aarts,et al.  Combinatorial Optimization on a Boltzmann Machine , 1989, J. Parallel Distributed Comput..

[5]  Rainer Schrader,et al.  On the Convergence of Stationary Distributions in Simulated Annealing Algorithms , 1988, Inf. Process. Lett..

[6]  J. Reif,et al.  On Threshold Circuits and Polynomial Computation , 1992, SIAM J. Comput..

[7]  P. Boas.  Machine models and simulations , 1991 .

[8]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[9]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[11]  Jerome A. Feldman.  Energy and the Behavior of Connectionist Models. , 1985 .

[12]  James L. Johnson.  A Neural Network Approach to the 3-Satisfiability Problem , 1989, J. Parallel Distributed Comput..

[13]  Saburo Muroga,et al.  Enumeration of Threshold Functions of Eight Variables , 1970, IEEE Transactions on Computers.

[14]  Saburo Muroga,et al.  Threshold logic and its applications , 1971 .

[15]  Terence R. Smith,et al.  Computable functions and complexity in neural networks , 1987 .

[16]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[17]  Uzi Vishkin,et al.  Constant Depth Reducibility , 1984, SIAM J. Comput..

[18]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[19]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[20]  Jirí Wiedermann,et al.  On the Computational Efficiency of Symmetric Neural Networks , 1991, Theor. Comput. Sci..

[21]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[22]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[23]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[24]  Georg Schnitger,et al.  Relating Boltzmann machines to conventional models of computation , 1987, Neural Networks.

[25]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[26]  Leonard M. Adleman,et al.  Two theorems on random polynomial time , 1978, 19th Annual Symposium on Foundations of Computer Science (sfcs 1978).

[27]  John J. Hopfield,et al.  The Effectiveness of Neural Computing , 1989, IFIP Congress.

[28]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.