Neural networks with low levels of activity: Ising vs. McCulloch-Pitts neurons

The performance of neural networks used as associative memory for uncorrelated patterns with prescribed mean activity is analyzed within the replica symmetric mean field theory. The optimal representation of the possible states of the neutrons, active or inactive, is found to depend on the mean activity. For activity equal one half Ising neurons and for low activities McCulloch-Pitts neurons are optimal. In this optimal representation the noise due to noncondensed patterns is reduced.

[1]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Néstor Parga,et al.  The ultrametric organization of memories in a neural network , 1986 .

[4]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[5]  U. Krey,et al.  Dynamical Learning Process for Recognition of Correlated Patterns in Symmetric Spin Glass Models , 1987 .

[6]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[7]  Sompolinsky,et al.  Information storage in neural networks with low levels of activity. , 1987, Physical review. A, General physics.

[8]  J. Zittartz,et al.  Glauber dynamics of the Little-Hopfield model , 1988 .

[9]  J. L. van Hemmen,et al.  Martingale approach to neural networks with hierarchically structured information , 1988 .

[10]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[11]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[12]  Gutfreund.  Neural networks with hierarchically correlated patterns. , 1988, Physical review. A, General physics.

[13]  Spingläser und Hirngespinste: Physikalische Modelle des Lernens und Erkennens , 1988 .