Martingale approach to neural networks with hierarchically structured information

A modified version of a model with hierarchically structured information, originally proposed by Parga and Virasoro, is studied in the context of martingale theory. The number of generations is allowed to be arbitrary but fixed. In the calculation of the storage capacity both the patterns and all their ancestors have to be taken into account as retrieval states. Due to the martingale property, the free energy greatly simplifies and for the retrieval states is shown to reduce to the one of the Hopfield model. So the storage capacity and retrieval quality of both models are identical. Numerical stimulations confirm the theoretical predictions.

[1]  J. L. van Hemmen,et al.  Storing patterns in a spin-glass model of neural networks nears saturation , 1987 .

[2]  R. Palmer,et al.  The replica method and solvable spin glass model , 1979 .

[3]  J. Doob.  What is a Martingale , 1971 .

[4]  J. Lamperti.  Stochastic Processes , 2022, The SAGE Encyclopedia of Research Design.

[5]  J. Lamperti.  Stochastic processes : a survey of the mathematical theory , 1979 .

[6]  J. Hemmen.  Nonlinear neural networks near saturation. , 1987 .

[7]  Jan Leonard van Hemmen,et al.  Dynamics and ergodicity of the infinite harmonic crystal , 1980 .

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Corinna Cortes,et al.  Hierarchical associative networks , 1987 .

[10]  W. Kinzel.  Learning and pattern recognition in spin glass models , 1985 .

[11]  V S Dotsdernko,et al.  'Ordered' spin glass: a hierarchical memory machine , 1985 .

[12]  J. L. van Hemmen,et al.  Storing extensively many weighted patterns in a saturated neural network , 1987 .

[13]  Néstor Parga,et al.  The ultrametric organization of memories in a neural network , 1986 .

[14]  Gérard Weisbuch,et al.  Scaling laws for the attractors of Hopfield networks , 1985 .

[15]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[16]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[17]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .