Layered neural networks

Exact solutions for the dynamics of layered feedforward neural networks are presented. These networks are expected to respond to an input by going through a sequence of preassigned states on the various layers. The family of networks considered has a variety of interlayer couplings: linear and non-linear Hebbian, Hebbian with Gaussian synaptic noise and with various kinds of dilution. In addition, the authors also solve the problem of layered networks with the pseudoinverse (projector) matrix of couplings. In all cases the solutions take the form of layer-to-layer recursions for the mean overlap with a (random) key pattern and for the width of the embedding field distribution. The dynamics is governed by the fixed points of these recursions. For all cases, non-trivial domains of attraction of the memory states are found and graphically displayed.

[1]  Professor Moshe Abeles,et al.  Local Cortical Circuits , 1982, Studies of Brain Function.

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Ron Meir,et al.  Storing and Retrieving Information in a Layered Spin System , 1986 .

[4]  J. L. Hemmen,et al.  Nonlinear neural networks. , 1986, Physical review letters.

[5]  L. Personnaz,et al.  Collective computational properties of neural networks: New learning mechanisms. , 1986, Physical review. A, General physics.

[6]  Sompolinsky,et al.  Neural networks with nonlinear synapses and a static noise. , 1986, Physical review. A, General physics.

[7]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[8]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[9]  J. Hemmen.  Nonlinear neural networks near saturation. , 1987 .

[10]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[11]  Meir,et al.  Exact solution of a layered neural network model. , 1987, Physical review letters.

[12]  Haim Sompolinsky,et al.  The Theory of Neural Networks: The Hebb Rule and Beyond , 1987 .

[13]  Eytan Domany,et al.  Stochastic dynamics of a layered neural network. Exact solution , 1987 .

[14]  I. Morgenstern,et al.  Heidelberg Colloquium on Glassy Dynamics , 1987 .

[15]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[16]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[17]  Marc Mézard,et al.  The roles of stability and symmetry in the dynamics of neural networks , 1988 .

[18]  Meir,et al.  Chaotic behavior of a layered neural network. , 1988, Physical review. A, General physics.

[19]  Ron Meir.  Extensions of a solvable feed forward neural network , 1988 .

[20]  Eytan Domany.  Neural networks: A biased overview , 1988 .

[21]  Meir,et al.  Layered feed-forward neural network with exactly soluble dynamics. , 1988, Physical review. A, General physics.

[22]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[23]  Marc Mézard,et al.  Basins of Attraction in a Perception-like Neural Network , 1988, Complex Syst..

[24]  Eytan Domany,et al.  Learning by Choice of Internal Representations , 1988, Complex Syst..

[25]  J. A. Hertz,et al.  Statistical Mechanics of Neural Computation , 1988 .

[26]  Wolfgang Kinzel,et al.  Basins of attraction near the critical storage capacity for neural networks with constant stabilities , 1989 .

[27]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.