Optimal storage properties of neural network models

The authors calculate the number, p= alpha N of random N-bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter K. For each value of alpha and K, there is a minimum fraction fmin of wrong bits. They find a critical line, alpha c(K) with alpha c(0)=2. The minimum fraction of wrong bits vanishes for alpha alpha c(K). The calculations are done using a saddle-point method and the order parameters at the saddle point are assumed to be replica symmetric. This solution is locally stable in a finite region of the K, alpha plane including the line, alpha c(K) but there is a line above which the solution becomes unstable and replica symmetry must be broken.

[1]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[2]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .

[3]  S. Edwards,et al.  Theory of spin glasses , 1975 .

[4]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[5]  Henri Orland,et al.  White and weighted averages over solutions of Thouless Anderson Palmer equations for the Sherrington Kirkpatrick spin glass , 1980 .

[6]  A. Bray,et al.  Metastable states in spin glasses , 1980 .

[7]  A. Bray,et al.  Metastable states in spin glasses with short-ranged interactions , 1981 .

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  I. Guyon,et al.  Information storage and retrieval in spin-glass like neural networks , 1985 .

[10]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[11]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[12]  Rammile Ettelaie,et al.  Residual entropy and simulated annealing , 1985 .

[13]  Marc Mézard,et al.  Solvable models of working memories , 1986 .

[14]  G. Parisi.  A memory which forgets , 1986 .

[15]  S Dehaene,et al.  Spin glass model of learning by selection. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[16]  E. Gardner,et al.  Metastable states of a spin glass chain at 0 temperature , 1986 .

[17]  E. Gardner.  Structure of metastable states in the Hopfield model , 1986 .

[18]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[19]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[20]  E. Gardner,et al.  Maximum Storage Capacity in Neural Networks , 1987 .

[21]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[22]  S. Venkatesh.  Epsilon capacity of neural networks , 1987 .

[23]  P. Leath,et al.  The failure distribution in percolation models of breakdown , 1987 .

[24]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[25]  E. Gardner.  The space of interactions in neural network models , 1988 .

[26]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.