A Complex-Valued Projection Neural Network for Constrained Optimization of Real Functions in Complex Variables

In this paper, we present a complex-valued projection neural network for solving constrained convex optimization problems of real functions with complex variables, as an extension of real-valued projection neural networks. Theoretically, by developing results on complex-valued optimization techniques, we prove that the complex-valued projection neural network is globally stable and convergent to the optimal solution. Obtained results are completely established in the complex domain and thus significantly generalize existing results of the real-valued projection neural networks. Numerical simulations are presented to confirm the obtained results and effectiveness of the proposed complex-valued projection neural network.

[1]  Danilo P. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters , 2009 .

[2]  B. A. D. H. Brandwood.  A complex gradient operator and its applica-tion in adaptive array theory , 1983 .

[3]  Jitao Sun,et al.  Stability of Complex-Valued Recurrent Neural Networks With Time-Delays , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[4]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[5]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[6]  Jun Wang,et al.  Global Stability of Complex-Valued Recurrent Neural Networks With Time-Delays , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Bin Liu,et al.  A synthesis method based on stability analysis for complex-valued Hopfield neural network , 2009, 2009 7th Asian Control Conference.

[8]  Masao Fukushima,et al.  A Sequential Quadratically Constrained Quadratic Programming Method for Differentiable Convex Minimization , 2002, SIAM J. Optim..

[9]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[10]  Mauro Forti,et al.  Convergence of Neural Networks for Programming Problems via a Nonsmooth Łojasiewicz Inequality , 2006, IEEE Transactions on Neural Networks.

[11]  A. Bos.  Complex gradient and Hessian , 1994 .

[12]  Yuanliang Ma,et al.  CGHA for principal component extraction in the complex domain , 1997, IEEE Trans. Neural Networks.

[13]  Huaguang Zhang,et al.  Robust Stability Analysis for Interval Cohen–Grossberg Neural Networks With Unknown Time-Varying Delays , 2008, IEEE Transactions on Neural Networks.

[14]  Akira Hirose,et al.  Complex-Valued Neural Networks: Theories and Applications , 2003 .

[15]  Q. Song,et al.  Boundedness and Stability for Discrete-Time Delayed Neural Network with Complex-Valued Linear Threshold Neurons , 2010 .

[16]  Zidong Wang,et al.  Global Synchronization for Discrete-Time Stochastic Complex Networks With Randomly Occurred Nonlinearities and Mixed Time Delays , 2010, IEEE Transactions on Neural Networks.

[17]  Bing Chen,et al.  Global Stability Criterion for Delayed Complex-Valued Recurrent Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Kazuyuki Aihara,et al.  Complex-valued forecasting of wind profile , 2006 .

[19]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[20]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[21]  Wen Xu,et al.  Fast estimation of sparse doubly spread acoustic channels. , 2012, The Journal of the Acoustical Society of America.

[22]  Marcos Eduardo Valle,et al.  Complex-Valued Recurrent Correlation Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Saleem A. Kassam,et al.  Channel Equalization Using Adaptive Complex Radial Basis Function Networks , 1995, IEEE J. Sel. Areas Commun..

[24]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Danilo P. Mandic,et al.  An Augmented Extended Kalman Filter Algorithm for Complex-Valued Recurrent Neural Networks , 2006, Neural Computation.

[26]  Huaguang Zhang,et al.  Global Asymptotic Stability of Recurrent Neural Networks With Multiple Time-Varying Delays , 2008, IEEE Transactions on Neural Networks.

[27]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[28]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[29]  Benedikt Loesch,et al.  Cramér-Rao Bound for Circular and Noncircular Complex Independent Component Analysis , 2013, IEEE Transactions on Signal Processing.

[30]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[31]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[32]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[33]  Igor N. Aizenberg,et al.  Solving the XOR and parity N problems using a single universal binary neuron , 2007, Soft Comput..

[34]  Hualiang Li,et al.  Optimization in the Complex Domain for Nonlinear Adaptive Filtering , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[35]  M. Bohner,et al.  Global Stability of Complex-Valued Neural Networks on Time Scales , 2011 .

[36]  Zhigang Zeng,et al.  Multistability of Two Kinds of Recurrent Neural Networks With Activation Functions Symmetrical About the Origin on the Phase Plane , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[38]  Huaguang Zhang,et al.  Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Ken Kreutz-Delgado,et al.  The Complex Gradient Operator and the CR-Calculus ECE275A - Lecture Supplement - Fall 2005 , 2009, 0906.4835.

[40]  Lieven De Lathauwer,et al.  Unconstrained Optimization of Real Functions in Complex Variables , 2012, SIAM J. Optim..

[41]  Yong Tan,et al.  Global asymptotical stability of continuous-time delayed neural networks without global Lipschitz activation functions , 2009 .

[42]  Masaki Kobayashi,et al.  Exceptional Reducibility of Complex-Valued Neural Networks , 2010, IEEE Transactions on Neural Networks.

[43]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[44]  D.-L. Lee,et al.  Improvements of Complex-Valued Hopfield Associative Memory by Using Generalized Projection Rules , 2006, IEEE Transactions on Neural Networks.

[45]  N. N. Vakhania,et al.  Random Vectors with Values in Complex Hilbert Spaces , 1997 .

[46]  Wei Xing Zheng,et al.  A complex-valued neural dynamical optimization approach and its stability analysis , 2015, Neural Networks.

[47]  R. Remmert,et al.  Theory of Complex Functions , 1990 .

[48]  Jacek M. Zurada,et al.  Discrete-Time Recurrent Neural Networks With Complex-Valued Linear Threshold Neurons , 2009, IEEE Transactions on Circuits and Systems II: Express Briefs.

[49]  Are Hjørungnes,et al.  Complex-Valued Matrix Differentiation: Techniques and Key Results , 2007, IEEE Transactions on Signal Processing.

[50]  Gang Feng,et al.  A neural network for robust LCMP beamforming , 2006, Signal Process..

[51]  Jonas Schreiber.  Complex Valued Neural Networks Utilizing High Dimensional Parameters , 2016 .