Statistical Mechanics of Neural Computation

Neural computation is a style of computation that draws inspiration from the way the brain computes. It is an in trinsically collective paradigm characterized by high con nectivity among a very large number of simple pro cessors running in parallel, possibly asynchronously. Methods developed in the theory of many-particle systems can be brought to bear on important conceptual questions about the operation and programming of such computational assemblies. This paper reviews several basic problems that arise in this area: the mathematical formulation of the collective computation done by such a network and of algorithms for programming ("teaching") them. The importance of phase transitions for understanding the generic behavior of such systems and algorithms is emphasized.

[1]  J J Hopfield,et al.  Learning algorithms and probability distributions in feed-forward and feed-back networks. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[3]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[4]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[5]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[6]  R. Glauber.  Time‐Dependent Statistics of the Ising Model , 1963 .

[7]  E. Oja.  On the convergence of an associative learning algorithm in the presence of noise , 1980 .

[8]  E. Gardner,et al.  Maximum Storage Capacity in Neural Networks , 1987 .

[9]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[10]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[12]  J.A. Anderson.  Two models for memory organization using interacting traces , 1970 .

[13]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[14]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[15]  I. Guyon,et al.  Information storage and retrieval in spin-glass like neural networks , 1985 .

[16]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[17]  Haim Sompolinsky,et al.  The Theory of Neural Networks: The Hebb Rule and Beyond , 1987 .

[18]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[19]  J. A. Hertz,et al.  Irreversible spin glasses and neural networks , 1987 .

[20]  T. Sejnowski,et al.  Predicting the secondary structure of globular proteins using neural network models. , 1988, Journal of molecular biology.

[21]  Baaquie.  Path-integral derivation of the heat kernel on SU(N) space. , 1985, Physical review. D, Particles and fields.

[22]  I. Morgenstern,et al.  Heidelberg Colloquium on Glassy Dynamics , 1987 .