On the capacity of associative memories with linear threshold functions

Some important features of various known constructions of associative memories based on linear threshold functions are analyzed. Two important features are dealt with: (a) the ability to select an arbitrary set of desired memory vectors and design a network for this set; (b) the sizes and shapes of the domains of attraction of the desired memory vectors and their relation to various design parameters. The static capacity for randomly chosen desired memories is also analyzed. Two extremal examples of sets of desired memories are then analyzed in detail. For spectral schemes with randomly chosen O(N/ln N) memories, it is shown that almost all of the Hamming sphere around each memory is directly attracted. >

[1]  J. Riordan,et al.  The distribution of random determinants , 1954 .

[2]  T. Greville,et al.  Some Applications of the Pseudoinverse of a Matrix , 1960 .

[3]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[4]  A. Prékopa.  ON RANDOM DETERMINANTS I , 1967 .

[5]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[6]  I. Guyon,et al.  Information storage and retrieval in spin-glass like neural networks , 1985 .

[7]  Yaser S. Abu-Mostafa,et al.  Information capacity of the Hopfield model , 1985, IEEE Trans. Inf. Theory.

[8]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[9]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[10]  J. W. Silverstein.  The Smallest Eigenvalue of a Large Dimensional Wishart Matrix , 1985 .

[11]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[12]  B V Kumar,et al.  Evaluation of the use of the Hopfield neural network model as a nearest-neighbor algorithm. , 1986, Applied optics.

[13]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[14]  J S Denker,et al.  Neural network models of learning and adaptation , 1986 .

[15]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[16]  S. Venkatesh.  Epsilon capacity of neural networks , 1987 .

[17]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[18]  Jehoshua Bruck,et al.  A study on neural networks , 1988, Int. J. Intell. Syst..

[19]  Pierre Baldi,et al.  Neural networks, orientations of the hypercube, and algebraic threshold functions , 1988, IEEE Trans. Inf. Theory.

[20]  Demetri Psaltis,et al.  Linear and logarithmic capacities in associative neural networks , 1989, IEEE Trans. Inf. Theory.

[21]  A. Dembo.  On random determinants , 1989 .