Generalization Characteristics of Complex-Valued Feedforward Neural Networks in Relation to Signal Coherence

Applications of complex-valued neural networks (CVNNs) have expanded widely in recent years-in particular in radar and coherent imaging systems. In general, the most important merit of neural networks lies in their generalization ability. This paper compares the generalization characteristics of complex-valued and real-valued feedforward neural networks in terms of the coherence of the signals to be dealt with. We assume a task of function approximation such as interpolation of temporal signals. Simulation and real-world experiments demonstrate that CVNNs with amplitude-phase-type activation function show smaller generalization error than real-valued networks, such as bivariate and dual-univariate real-valued neural networks. Based on the results, we discuss how the generalization characteristics are influenced by the coherence of the signals depending on the degree of freedom in the learning and on the circularity in neural dynamics.

[1]  Donald L. Birx,et al.  A complex mapping network for phase sensitive classification , 1993, IEEE Trans. Neural Networks.

[2]  Shigeo Sato,et al.  Quantum Neural Network Composed of Kane's Qubits , 2006 .

[3]  Jacek M. Zurada,et al.  Blur Identification by Multilayer Neural Network Based on Multivalued Neurons , 2008, IEEE Transactions on Neural Networks.

[4]  Akira Hirose,et al.  Applications of complex-valued neural networks to coherent optical computing using phase-sensitive detection scheme , 1994 .

[5]  A. Hirose Nature of complex number and complex-valued neural networks , 2011 .

[6]  Hiroshi Sawada,et al.  Polar coordinate based nonlinear function for frequency-domain blind source separation , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Akira Hirose,et al.  Beamforming of ultra-wideband pulses by a complex-valued spatio-temporal multilayer neural network , 2005, Int. J. Neural Syst..

[8]  Yuanliang Ma,et al.  CGHA for principal component extraction in the complex domain , 1997, IEEE Trans. Neural Networks.

[9]  Cris Koutsougeras,et al.  Complex domain backpropagation , 1992 .

[10]  Akira Hirose,et al.  Plastic mine detecting radar system using complex-valued self-organizing map that deals with multiple-frequency interferometric images , 2004, Neural Networks.

[11]  Akira Hirose,et al.  Walled LTSA Array for Rapid, High Spatial Resolution, and Phase-Sensitive Imaging to Visualize Plastic Landmines , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Danilo P. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters , 2009 .

[13]  Pascal Chevalier,et al.  Widely linear estimation with complex data , 1995, IEEE Trans. Signal Process..

[14]  Danilo P. Mandic,et al.  Duality between widely linear and dual channel adaptive filtering , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  Danilo P. Mandic,et al.  Nonlinear adaptive prediction of complex-valued signals by complex-valued PRNN , 2005, IEEE Transactions on Signal Processing.

[16]  Tohru Nitta,et al.  An Extension of the Back-Propagation Algorithm to Complex Numbers , 1997, Neural Networks.

[17]  Danilo P. Mandic,et al.  An Augmented Echo State Network for Nonlinear Adaptive Filtering of Complex Noncircular Signals , 2011, IEEE Transactions on Neural Networks.

[18]  Akira Hirose,et al.  Developmental Learning With Behavioral Mode Tuning by Carrier-Frequency Modulation in Coherent Neural Networks , 2006, IEEE Transactions on Neural Networks.

[19]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[20]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[21]  Akira Hirose,et al.  Continuous complex-valued back-propagation learning , 1992 .

[22]  Akira Hirose,et al.  Coherent optical neural network that learns desirable phase values in the frequency domain by use of multiple optical-path differences. , 2003, Optics letters.

[23]  Gouhei Tanaka,et al.  Complex-valued multistate associative memory with nonlinear multilevel functions for gray-level image reconstruction , 2009, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[24]  Akira Hirose,et al.  Efficient generation of holographic movies with frame interpolation using a coherent neural network , 2006, IEICE Electron. Express.

[25]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[26]  Akira Hirose,et al.  Behavior control of coherent-type neural networks by carrier-frequency modulation , 1996, IEEE Trans. Neural Networks.