Higher Order Neurodynamics of Associative Memory for Sequential Patterns

This paper describes higher order neurodynamics of associative memory for sequential patterns using a statistical method. First, the statistical analysis of direct correlations between the cross talk noise terms for higher order neural networks is made. Further, it is shown that storage capacities for k = 1, 2 and 3 dimensional cases are 0.263n , $0.207\binom{n}{2}$ and $0.180\binom{n}{3}$, respectively, where n is the number of neurons and $\binom{n}{k}$ means the combination of k from n . The result for the one dimensional case is in fairly general agreement with Meir's result, 0.269n , obtained by the replica theory.

[1]  Meir,et al.  Exact solution of a layered neural network model. , 1987, Physical review letters.

[2]  Shun-ichi Amari,et al.  Mathematical foundations of neurocomputing , 1990, Proc. IEEE.

[3]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[4]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[5]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[6]  M. Okada,et al.  Transient dynamics for sequence processing neural networks , 2002, cond-mat/0202431.

[7]  Masato Okada,et al.  Notions of Associative Memory and Sparse Coding , 1996, Neural Networks.

[8]  S.-i. Amari.  Statistical neurodynamics of various versions of correlation associative memory , 1988, IEEE 1988 International Conference on Neural Networks.

[9]  Madan M. Gupta,et al.  Static and Dynamic Neural Networks: From Fundamentals to Advanced Theory , 2003 .

[10]  Hiromi Miyajima,et al.  Statistical dynamics of associative memory for higher order neural networks , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[11]  Abbott,et al.  Storage capacity of generalized networks. , 1987, Physical review. A, General physics.

[12]  H. Nishimori,et al.  Retrieval dynamics of associative memory of the Hopfield type , 1993 .

[13]  S. Yatsuki,et al.  Associative ability of higher order neural networks , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).