Information storage in high-order neural networks with unequal neural activity

Abstract Neural networks with high-order interactions only have been shown to be sufficient to provide satisfactory attractivity to the stored patterns and error corrections. Such interactions increase the storage capacity of the networks and allow one to solve a class of problems which are intractable with standard networks. In this paper we analyse the capacity of these higher-order networks by the statistical method and show why the probability of the states of neurons being active and passive can always be chosen equal, i.e. with a probability of 0.5 each.

[1]  Baldi,et al.  Number of stable points for spin-glasses and neural networks of higher orders. , 1987, Physical review letters.

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[4]  Demetri Psaltis,et al.  Higher order associative memories and their optical implementations , 1988, Neural Networks.

[5]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[6]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[7]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[8]  C. L. Giles,et al.  High order correlation model for associative memory , 1987 .

[9]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[10]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[11]  Heng-Ming Tai,et al.  Neural networks with higher-order nonlinearity , 1988 .

[12]  Heng-Ming Tai,et al.  Hopfield model with complementary binary representations , 1988 .

[13]  D Psaltis,et al.  Optical information processing based on an associative-memory model of neural nets with thresholding and feedback. , 1985, Optics letters.

[14]  Peter Grant,et al.  A comparison of neural network and matched filter processing for detecting lines in images , 1987 .