Dynamics of learning

Supervised leaning in attractor networks which perform as an associative memory is investigated. Two learning algorithms, the PERCEPTRON of optimal stability and the ADALINE, are derived from optimization problems and exact results for their dynamics are obtained. The ADALINE is extended to networks with binary synapses and is studied numerically. The basins of attraction during the learning process are calculated using a Gaussian approximation. Finally analytical results for forgetting in the ADALINE network are presented.

[1]  Ron Meir,et al.  Storing and Retrieving Information in a Layered Spin System , 1986 .

[2]  B. A. Huberman,et al.  Adaptation and self-repair in parallel computing structures , 1984 .

[4]  Michel Kerszberg,et al.  Synchronization in Neural Assemblies , 1990 .

[5]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[6]  B. Forrest.  Content-addressability and learning in neural networks , 1988 .

[7]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  M. Opper.  Learning in Neural Networks: Solvable Dynamics , 1989 .

[10]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation columns. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[11]  U. Krey,et al.  Dynamical Learning Process for Recognition of Correlated Patterns in Symmetric Spin Glass Models , 1987 .

[12]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[13]  Marc Mézard,et al.  Basins of Attraction in a Perception-like Neural Network , 1988, Complex Syst..

[14]  S Dehaene,et al.  Neural networks that learn temporal sequences by selection. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Tamás Geszti,et al.  Learning within bounds and dream sleep , 1987 .

[16]  Anthony C. C. Coolen,et al.  Delays in Neural Networks , 1988 .

[17]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[18]  W. Kinzel,et al.  Layered neural networks , 1989 .

[19]  Joachim M. Buhmann,et al.  Noise-driven temporal association in neural networks , 1987 .

[20]  Opper.  Learning times of neural networks: Exact solution for a PERCEPTRON algorithm. , 1988, Physical review. A, General physics.

[21]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[22]  P. Carnevali,et al.  Exhaustive Thermodynamical Analysis of Boolean Learning Networks , 1987 .

[23]  S Dehaene,et al.  Spin glass model of learning by selection. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Opper,et al.  Replicators with random interactions: A solvable model. , 1989, Physical review. A, General physics.

[25]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[26]  J. Hertz,et al.  Phase transitions in simple learning , 1989 .

[27]  Werner Krauth,et al.  Critical storage capacity of the J = ± 1 neural network , 1989 .

[28]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[29]  Spingläser und Hirngespinste: Physikalische Modelle des Lernens und Erkennens , 1988 .

[30]  P. Peretto,et al.  Collective Properties of Neural Networks , 1986 .

[31]  Françoise Fogelman-Soulié,et al.  Disordered Systems and Biological Organization , 1986, NATO ASI Series.

[32]  K. Binder,et al.  Spin glasses: Experimental facts, theoretical concepts, and open questions , 1986 .

[33]  Meir,et al.  Layered feed-forward neural network with exactly soluble dynamics. , 1988, Physical review. A, General physics.

[34]  Meir,et al.  Iterated learning in a layered feed-forward neural network. , 1988, Physical review. A, General physics.

[35]  J. Hemmen,et al.  The Hebb rule: storing static and dynamic objects in an associative neural network , 1988 .

[36]  Stanislas Dehaene,et al.  Networks of Formal Neurons and Memory Palimpsests , 1986 .

[37]  J. Hemmen.  Nonlinear neural networks near saturation. , 1987 .

[38]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[39]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[40]  L. Personnaz,et al.  Collective computational properties of neural networks: New learning mechanisms. , 1986, Physical review. A, General physics.

[41]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[42]  M. Opper,et al.  Distribution of Internal Fields and Dynamics of Neural Networks , 1990 .

[43]  G. Parisi.  A memory which forgets , 1986 .

[44]  E. Gardner.  The space of interactions in neural network models , 1988 .

[45]  Michael Biehl,et al.  The AdaTron: An Adaptive Perceptron Algorithm , 1989 .

[46]  J. Nadal,et al.  Learning and forgetting on asymmetric, diluted neural networks , 1987 .

[47]  Wolfgang Kinzel,et al.  Learning algorithm for a neural network with binary synapses , 1990 .

[48]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[49]  Sompolinsky,et al.  Neural networks with nonlinear synapses and a static noise. , 1986, Physical review. A, General physics.

[50]  Thomas B. Kepler,et al.  Domains of attraction in neural networks , 1988 .

[51]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[52]  Wolfgang Kinzel,et al.  Basins of attraction near the critical storage capacity for neural networks with constant stabilities , 1989 .