Phase transitions of neural networks

Abstract The cooperative behaviour of interacting neurons and synapses is studied using models and methods from statistical physics. The competition between training error and entropy may lead to discontinuous properties of the neural network. This is demonstrated for a few examples: perceptron, associative memory, learning from examples, generalization, multilayer networks, structure recognition, Bayesian estimate, on-line training, noise estimation and time series generation.

[1]  Sompolinsky,et al.  Statistical mechanics of the maximum-likelihood density estimation. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[2]  Kanter,et al.  Statistical mechanics of a multilayered neural network. , 1990, Physical review letters.

[3]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[4]  Michael Biehl,et al.  Learning by on-line gradient descent , 1995 .

[5]  J. Nadal,et al.  Optimal unsupervised learning , 1994 .

[6]  Michael Biehl,et al.  Transient dynamics of on-line learning in two-layered neural networks , 1996 .

[7]  Michael Biehl,et al.  Statistical Mechanics of Unsupervised Learning , 1993 .

[8]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[9]  Kanter,et al.  Generation and prediction of time series by a neural network. , 1995, Physical review letters.

[10]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[11]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[12]  Michael Biehl,et al.  Noise robustness in multilayer neural networks , 1997 .

[13]  Saad,et al.  Exact solution for on-line learning in multilayer neural networks. , 1995, Physical review letters.

[14]  L. Schläfli.  Theorie der vielfachen Kontinuität , 1901 .

[15]  Opper,et al.  Generalization in a two-layer neural network. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[16]  Opper,et al.  Learning and generalization in a two-layer neural network: The role of the Vapnik-Chervonvenkis dimension. , 1994, Physical review letters.

[17]  Kanter,et al.  Analytical study of time series generation by feed-forward networks. , 1995, Physical review letters.

[18]  J. Yeomans,et al.  Statistical mechanics of phase transitions , 1992 .

[19]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[21]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[22]  E. Gardner.  The space of interactions in neural network models , 1988 .

[23]  O. Kinouchi,et al.  Optimal generalization in perceptions , 1992 .