Complex-Valued Neural Networks: Learning Algorithms and Applications

Complex-valued data arise in various applications, such as radar and array signal processing, magnetic resonance imaging, communication systems, and processing data in the frequency domain. To deal with such data properly, neural networks are extended to the complex domain, referred to as complex-valued neural networks (CVNNs), allowing the network parameters to be complex numbers and the computations to follow the complex algebraic rules. Unlike the real-valued case, the nonlinear functions in the CVNNs do not have standard complex derivatives as the Cauchy-Riemann equations do not hold for them. Consequently, the traditional approach for deriving learning algorithms reformulates the problem in the real domain which is often tedious. In this thesis, we first develop a systematic and simpler approach using Wirtinger calculus to derive the learning algorithms in the CVNNs. It is shown that adopting three steps: (i) computing a pair of derivatives in the conjugate coordinate system, (ii) using coordinate transformation between real and conjugate coordinates, and (iii) organizing derivative computations through functional dependency graph greatly simplify the derivations. To illustrate, a gradient descent and LevenbergMarquardt algorithms are considered. Although a single-layered network, referred to as functional link network (FLN), has been widely used in the real domain because of its simplicity and faster processing, no such study exists in the complex domain. In the FLN, the nonlinearity is endowed in the input layer by constructing linearly independent basis functions in addition to the original variables. We design a parsimonious complex-valued FLN (CFLN) using orthogonal least squares (OLS) method, where the basis functions are multivariate polynomial terms. It is observed that the OLS based CFLN yields simple structure with favorable performance comparing to the multilayer CVNNs in several applications. It is well known and interesting that a complex-valued neuron can solve several nonlinearly separable problems, including the XOR, parity-n, and symmetry detection problems, which a real-valued neuron cannot. With this motivation, we perform an empirical study of classification performance of single-layered CVNNs on several real-world benchmark classification problems with two new activation functions. The experimental results exhibit that the classification performances of single-layered CVNNs are comparable to those of multilayer real-valued neural networks. Further enhancement of discrimination ability has been obtained using the ensemble approach. I dedicate this dissertation to my parents