Pseudo-Relaxation Learning Algorithm for Complex-Valued Associative Memory

HAM (Hopfield Associative Memory) and BAM (Bidirectinal Associative Memory) are representative associative memories by neural networks. The storage capacity by the Hebb rule, which is often used, is extremely low. In order to improve it, some learning methods, for example, pseudo-inverse matrix learning and gradient descent learning, have been introduced. Oh introduced pseudo-relaxation learning algorithm to HAM and BAM. In order to accelerate it, Hattori proposed quick learning. Noest proposed CAM (Complex-valued Associative Memory), which is complex-valued HAM. The storage capacity of CAM by the Hebb rule is also extremely low. Pseudo-inverse matrix learning and gradient descent learning have already been generalized to CAM. In this paper, we apply pseudo-relaxation learning algorithm to CAM in order to improve the capacity.

[1]  S. Agmon.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[2]  Heekuck Oh,et al.  A pseudo-relaxation learning algorithm for bidirectional associative memory , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[3]  H. Aoki.  Image Association Using a Complex-Valued Associative Memory Model , 2000 .

[4]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[5]  Aoyagi,et al.  Network of Neural Oscillators for Retrieving Phase Information. , 1994, Physical review letters.

[6]  Yukio Kosugi,et al.  Characteristics of the complex‐valued associative memory model having penalty term , 2000 .

[7]  Teuvo Kohonen,et al.  An Adaptive Associative Memory Principle , 1974, IEEE Transactions on Computers.

[8]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons , 2000 .

[9]  B Kosko,et al.  Adaptive bidirectional associative memories. , 1987, Applied optics.

[10]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[11]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[12]  Richard S. Zemel,et al.  Directional-Unit Boltzmann Machines , 1992, NIPS.

[13]  André J. Noest,et al.  Phasor Neural Networks , 1987, NIPS.

[14]  Toshio Aoyagi,et al.  OSCILLATOR NEURAL NETWORK RETRIEVING SPARSELY CODED PHASE PATTERNS , 1999 .

[15]  K. Hasegawa,et al.  Improved Pseudo-Relaxation Learning Algorithm for Robust Bidirectional Associative Memory , 1999, ICANNGA.

[16]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications , 2012 .

[17]  J Cook,et al.  The mean-field theory of a Q-state neural network model , 1989 .

[18]  Jacek M. Zurada,et al.  A new design method for the complex-valued multistate Hopfield associative memory , 2003, IEEE Trans. Neural Networks.

[19]  Richard S. Zemel,et al.  Lending direction to neural networks , 1995, Neural Networks.

[20]  Naum N. Aizenberg,et al.  CNN based on multi-valued neuron as a model of associative memory for grey scale images , 1992, CNNA '92 Proceedings Second International Workshop on Cellular Neural Networks and Their Applications.

[21]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[22]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[24]  Masao Nakagawa,et al.  Quick learning for bidirectional associative memory , 1994 .

[25]  Jacek M. Zurada,et al.  Complex-valued multistate neural associative memory , 1996, IEEE Trans. Neural Networks.

[26]  Heekuck Oh,et al.  A new learning approach to enhance the storage capacity of the Hopfield model , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[27]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[28]  Donq-Liang Lee.  Improving the capacity of complex-valued neural networks with a modified gradient descent learning rule , 2001, IEEE Trans. Neural Networks.

[29]  Noest.  Discrete-state phasor neural networks. , 1988, Physical review. A, General physics.

[30]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[31]  Masao Nakagawa,et al.  New results of Quick Learning for Bidirectional Associative Memory having high capacity , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[32]  Heekuck Oh,et al.  Adaptation of the relaxation method for learning in bidirectional associative memory , 1994, IEEE Trans. Neural Networks.