Phase transitions of neural networks

Abstract The cooperative behaviour of interacting neurons and synapses is studied using models and methods from statistical physics. The competition between training error and entropy may lead to discontinuous properties of the neural network. This is demonstrated for a few examples: perceptron, associative memory, learning from examples, generalization, multilayer networks, structure recognition, Bayesian estimate, on-line training, noise estimation and time series generation.

[1]  Michael Biehl,et al.  Noise robustness in multilayer neural networks , 1997 .

[2]  Michael Biehl,et al.  Transient dynamics of on-line learning in two-layered neural networks , 1996 .

[3]  Kanter,et al.  Analytical study of time series generation by feed-forward networks. , 1995, Physical review letters.

[4]  Saad,et al.  Exact solution for on-line learning in multilayer neural networks. , 1995, Physical review letters.

[5]  Michael Biehl,et al.  Learning by on-line gradient descent , 1995 .

[6]  Kanter,et al.  Generation and prediction of time series by a neural network. , 1995, Physical review letters.

[7]  Sompolinsky,et al.  Statistical mechanics of the maximum-likelihood density estimation. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[8]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[9]  Opper,et al.  Learning and generalization in a two-layer neural network: The role of the Vapnik-Chervonvenkis dimension. , 1994, Physical review letters.

[10]  J. Nadal,et al.  Optimal unsupervised learning , 1994 .

[11]  Michael Biehl,et al.  Statistical Mechanics of Unsupervised Learning , 1993 .

[12]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[13]  O. Kinouchi,et al.  Optimal generalization in perceptions , 1992 .

[14]  Opper,et al.  Generalization in a two-layer neural network. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[15]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[16]  Kanter,et al.  Statistical mechanics of a multilayered neural network. , 1990, Physical review letters.

[17]  E. Gardner.  The space of interactions in neural network models , 1988 .

[18]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[19]  C. Thompson.  The Statistical Mechanics of Phase Transitions , 1978 .

[20]  O. Bagasra,et al.  Proceedings of the National Academy of Sciences , 1914, Science.

[21]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[22]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[23]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[24]  L. Schläfli.  Theorie der vielfachen Kontinuität , 1901 .

[25]  J. Steinberger.  Annals of Physics , 2022 .