Storing patterns in a spin-glass model of neural networks nears saturation

A Gaussian approximation for the synaptic noise and the n to 0 replica method are used to study spin-glass models of neural networks near saturation, i.e. when the number p of stored patterns increases with the size of the network N and p= alpha N. Qualitative features are predicted surprisingly well. For instance, at T=0 the linear Hopfield network provides effective associative memory with errors not exceeding 0.05% for alpha <or= alpha c approximately=0.15. In a network with clipped synapses, the number of patterns which can be stored with some given error tolerance is reduced by a factor of 2/pi as compared with the linear Hopfield model. A simple learning within bounds algorithm is found to continuously interpolate between the linear Hopfield model and the network with clipped synapses.

[1]  R. Palmer,et al.  The replica method and solvable spin glass model , 1979 .

[2]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[3]  Marc Mézard,et al.  Solvable models of working memories , 1986 .

[4]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[5]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[6]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Lev B. Ioffe,et al.  The statistical properties of the hopfield model of memory , 1986 .

[8]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[9]  G. Parisi.  A memory which forgets , 1986 .

[10]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[11]  S Dehaene,et al.  Spin glass model of learning by selection. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Sompolinsky,et al.  Neural networks with nonlinear synapses and a static noise. , 1986, Physical review. A, General physics.

[13]  J. L. Hemmen,et al.  Nonlinear neural networks. , 1986, Physical review letters.

[14]  Stanislas Dehaene,et al.  Networks of Formal Neurons and Memory Palimpsests , 1986 .

[15]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .