Capacity of strong attractor patterns to model behavioural and cognitive prototypes

We solve the mean field equations for a stochastic Hopfield network with temperature (noise) in the presence of strong, i.e., multiply stored, patterns, and use this solution to obtain the storage capacity of such a network. Our result provides for the first time a rigorous solution of the mean filed equations for the standard Hopfield model and is in contrast to the mathematically unjustifiable replica technique that has been used hitherto for this derivation. We show that the critical temperature for stability of a strong pattern is equal to its degree or multiplicity, when the sum of the squares of degrees of the patterns is negligible compared to the network size. In the case of a single strong pattern, when the ratio of the number of all stored pattens and the network size is a positive constant, we obtain the distribution of the overlaps of the patterns with the mean field and deduce that the storage capacity for retrieving a strong pattern exceeds that for retrieving a simple pattern by a multiplicative factor equal to the square of the degree of the strong pattern. This square law property provides justification for using strong patterns to model attachment types and behavioural prototypes in psychology and psychotherapy.

[1]  P. Peretto,et al.  On learning rules and memory storage abilities of asymmetrical neural networks , 1988 .

[2]  V. Dotsenko.  An Introduction to the Theory of Spin Glasses and Neural Networks , 1995 .

[3]  T. Lewis,et al.  A General Theory of Love , 2000 .

[4]  Gutfreund.  Neural networks with hierarchically correlated patterns. , 1988, Physical review. A, General physics.

[5]  E. Bolthausen.  Random Media and Spin Glasses: An Introduction into Some Mathematical Results and Problems , 2007 .

[6]  Sompolinsky,et al.  Information storage in neural networks with low levels of activity. , 1987, Physical review. A, General physics.

[7]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[8]  Patrick Billingsley,et al.  Probability and Measure. , 1986 .

[9]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[10]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[11]  M. Tsodyks,et al.  Capacity of networks with correlated attractors , 1994 .

[12]  鈴木 増雄.  Time-Dependent Statistics of the Ising Model , 1965 .

[13]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[14]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[15]  Klaus Minde,et al.  Affect Dysregulation and Disorders of the Self , 2006 .

[16]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[17]  A. Crisanti,et al.  Saturation Level of the Hopfield Model for Neural Network , 1986 .

[18]  J. Norris.  Appendix: probability and measure , 1997 .

[19]  Francis Crick,et al.  The function of dream sleep , 1983, Nature.

[20]  A. Bovier,et al.  Hopfield Models as Generalized Random Mean Field Models , 1996, cond-mat/9607103.

[21]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[22]  Matthias Löwe.  On the storage capacity of Hopfield models with correlated patterns , 1998 .

[23]  A V Lukashin,et al.  [Physical models of neural networks]. , 1987, Biofizika.

[24]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[25]  L. Baum,et al.  Convergence rates in the law of large numbers , 1963 .

[26]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[27]  R. Hoffman.  Computer simulations of neural information processing and the schizophrenia-mania dichotomy. , 1987, Archives of general psychiatry.

[28]  Abbas Edalat,et al.  Strong attractors of Hopfield neural networks to model attachment types and behavioural patterns , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).