Improving the capacity of complex-valued neural networks with a modified gradient descent learning rule

Jankowski et al. proposed (1996) a complex-valued neural network (CVNN) which is capable of storing and recalling gray-scale images. The convergence property of the CVNN has also been proven by means of the energy function approach. However, the memory capacity of the CVNN is very low because they use a generalized Hebb rule to construct the connection matrix. In this letter, a modified gradient descent learning rule (MGDR) is proposed to enhance the capacity of the CVNN. The proposed technique is derived by applying gradient search over a complex error surface. Simulation shows that the capacity of CVNN with MGDR is greatly improved.

[1]  Dong-Liang Lee.  New stability conditions for Hopfield networks in partial simultaneous update mode , 1999, IEEE Trans. Neural Networks.

[2]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[3]  Donq-Liang Lee,et al.  Relaxation of the stability condition of the complex-valued neural networks , 2001, IEEE Trans. Neural Networks.

[4]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[5]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[6]  Tzi-Dar Chiueh,et al.  Multivalued associative memories based on recurrent networks , 1993, IEEE Trans. Neural Networks.

[7]  Chua-Chin Wang,et al.  Capacity analysis of the asymptotically stable multi-valued exponential bidirectional associative memory , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[8]  R. Perfetti.  A neural network to design neural networks , 1991 .

[9]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[10]  Jennie Si,et al.  Analysis and synthesis of a class of discrete-time neural networks with multilevel threshold neurons , 1995, IEEE Trans. Neural Networks.

[11]  Seppo J. Ovaska,et al.  Fuzzy neural network with general parameter adaptation for modeling of nonlinear time-series , 2001, IEEE Trans. Neural Networks.

[12]  Jacek M. Zurada,et al.  Complex-valued multistate neural associative memory , 1996, IEEE Trans. Neural Networks.

[13]  Yusuf Öztürk,et al.  A New Family of Multivalued Networks , 1996, Neural Networks.

[14]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Jacek M. Zurada,et al.  Generalized Hopfield networks for associative memories with multi-valued stable states , 1996, Neurocomputing.

[16]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .