Stability and statistical properties of second-order bidirectional associative memory

In this paper, a bidirectional associative memory (BAM) model with second-order connections, namely second-order bidirectional associative memory (SOBAM), is first reviewed. The stability and statistical properties of the SOBAM are then examined. We use an example to illustrate that the stability of the SOBAM is not guaranteed. For this result, we cannot use the conventional energy approach to estimate its memory capacity. Thus, we develop the statistical dynamics of the SOBAM. Given that a small number of errors appear in the initial input, the dynamics shows how the number of errors varies during recall. We use the dynamics to estimate the memory capacity, the attraction basin, and the number of errors in the retrieved items. Extension of the results to higher-order bidirectional associative memories is also discussed.

[1]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[2]  Santosh S. Venkatesh,et al.  Programmed interactions in higher-order neural networks: The outer-product algorithm , 1991, J. Complex..

[3]  Demetri Psaltis,et al.  Higher order associative memories and their optical implementations , 1988, Neural Networks.

[4]  Chi-Sing Leung.  The performance of dummy augmentation encoding , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[5]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[6]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[7]  János Komlós,et al.  Convergence results in an associative memory model , 1988, Neural Networks.

[8]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[9]  J. Austin.  Associative memory , 1987 .

[10]  Gregory Allen Kohring.  On the Q-state neuron problem in attractor neural networks , 1993, Neural Networks.

[11]  C. S. Leung.  Encoding method for bidirectional associative memory using projection on convex sets , 1993, IEEE Trans. Neural Networks.

[12]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[13]  Charles M. Newman,et al.  Memory capacity in neural network models: Rigorous lower bounds , 1988, Neural Networks.

[14]  Robert Hecht-Nielsen,et al.  A BAM with increased information storage capacity , 1988, IEEE 1988 International Conference on Neural Networks.

[15]  Jose B. Cruz,et al.  Two coding strategies for bidirectional associative memory , 1990, IEEE Trans. Neural Networks.

[16]  Chi-Sing Leung.  Optimum learning for bidirectional associative memory in the sense of capacity , 1994 .

[17]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[18]  J Shamir,et al.  Design for a massive all-optical bidirectional associative memory: the big BAM. , 1988, Applied optics.

[19]  P.K. Simpson,et al.  Higher-ordered and intraconnected bidirectional associative memories , 1990, IEEE Trans. Syst. Man Cybern..

[20]  Shun-ichi Amari,et al.  Mathematical foundations of neurocomputing , 1990, Proc. IEEE.