Partially connected models of neural networks

A partially connected Hopfield neural network model is studied under the restriction that w, the ratio of connections per site to the size of the system, remains finite as the size N to infinity with the connection structure at each site being the same. The replica symmetric mean field theory equations for the order parameters are derived. The zero-temperature forms of these equations are then solved numerically for a few different 'local' connectivity architectures showing phase transitions at different critical storage ratios, alpha c, where the states which the authors are trying to store in the network become discontinuously unstable. They show that the information capacity per connection improves for partially connected systems.

[1]  E. Gardner.  The space of interactions in neural network models , 1988 .

[2]  A. Crisanti,et al.  Saturation Level of the Hopfield Model for Neural Network , 1986 .

[3]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[4]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .

[5]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[6]  D. J. Wallace,et al.  Learning and memory properties in fully connected networks , 1987 .

[7]  B. Forrest.  Content-addressability and learning in neural networks , 1988 .

[8]  Kanter,et al.  Mean-field theory of spin-glasses with finite coordination number. , 1987, Physical review letters.

[9]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[10]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[11]  A. R. Gardner-Medwin.  The recall of events through the learning of associations between their parts , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[12]  S. Edwards,et al.  Theory of spin glasses , 1975 .

[13]  D. J. Wallace,et al.  Dynamics and statistical mechanics of the Hopfield model , 1987 .

[14]  Sompolinsky,et al.  Neural networks with nonlinear synapses and a static noise. , 1986, Physical review. A, General physics.

[15]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[16]  Sompolinsky,et al.  Information storage in neural networks with low levels of activity. , 1987, Physical review. A, General physics.

[17]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.