A One-Layer Recurrent Neural Network for Constrained Complex-Variable Convex Optimization

In this paper, based on $\mathbb {CR}$ calculus and penalty method, a one-layer recurrent neural network is proposed for solving constrained complex-variable convex optimization. It is proved that for any initial point from a given domain, the state of the proposed neural network reaches the feasible region in finite time and converges to an optimal solution of the constrained complex-variable convex optimization finally. In contrast to existing neural networks for complex-variable convex optimization, the proposed neural network has a lower model complexity and better convergence. Some numerical examples and application are presented to substantiate the effectiveness of the proposed neural network.

[1]  Jun Wang,et al.  A one-layer recurrent neural network for support vector machine learning , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Jun Wang,et al.  A One-Layer Recurrent Neural Network for Constrained Nonsmooth Optimization , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[4]  Lijun Zhao,et al.  Neural network for constrained nonsmooth optimization using Tikhonov regularization , 2015, Neural Networks.

[5]  Wei Bian,et al.  Neural Network for Solving Constrained Convex Optimization Problems With Global Attractivity , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[6]  Predrag S. Stanimirovic,et al.  Neural network for computing pseudoinverses and outer inverses of complex-valued matrices , 2016, Appl. Math. Comput..

[7]  Jun Wang,et al.  A recurrent neural network with exponential convergence for solving convex quadratic program and related linear piecewise equations , 2004, Neural Networks.

[8]  Yuanliang Ma,et al.  CGHA for principal component extraction in the complex domain , 1997, IEEE Trans. Neural Networks.

[9]  Danilo Rastovic,et al.  From Non-Markovian Processes to Stochastic Real Time Control for Tokamak Plasma Turbulence Via Artificial Intelligence Techniques , 2015 .

[10]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[11]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[12]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[13]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[14]  Jun Wang,et al.  A Complex-Valued Projection Neural Network for Constrained Optimization of Real Functions in Complex Variables , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Long Cheng,et al.  Recurrent Neural Network for Non-Smooth Convex Optimization Problems With Application to the Identification of Genetic Regulatory Networks , 2011, IEEE Transactions on Neural Networks.

[16]  Wei Bian,et al.  Subgradient-Based Neural Networks for Nonsmooth Nonconvex Optimization Problems , 2009, IEEE Transactions on Neural Networks.

[17]  Benedikt Loesch,et al.  Cramér-Rao Bound for Circular and Noncircular Complex Independent Component Analysis , 2013, IEEE Transactions on Signal Processing.

[18]  Sitian Qin,et al.  A Two-Layer Recurrent Neural Network for Nonsmooth Convex Optimization Problems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Jun Wang.  Analysis and design of a recurrent neural network for linear programming , 1993 .

[20]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network with a Discontinuous Activation Function for Linear Programming , 2008, Neural Computation.

[21]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[22]  Danilo Rastovic,et al.  Targeting and synchronization at tokamak with recurrent artificial neural networks , 2012, Neural Computing and Applications.

[23]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[24]  Lieven De Lathauwer,et al.  Unconstrained Optimization of Real Functions in Complex Variables , 2012, SIAM J. Optim..

[25]  Igor N. Aizenberg,et al.  Solving the XOR and parity N problems using a single universal binary neuron , 2007, Soft Comput..

[26]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[27]  Masao Fukushima,et al.  A Sequential Quadratically Constrained Quadratic Programming Method for Differentiable Convex Minimization , 2002, SIAM J. Optim..

[28]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .

[29]  Wei Bian,et al.  Subgradient-Based Neural Networks for Nonsmooth Convex Optimization Problems , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[30]  Ken Kreutz-Delgado,et al.  The Complex Gradient Operator and the CR-Calculus ECE275A - Lecture Supplement - Fall 2005 , 2009, 0906.4835.

[31]  Gang Feng,et al.  A neural network for robust LCMP beamforming , 2006, Signal Process..

[32]  Danilo Rastovic.  TOKAMAK DESIGN AS ONE SUSTAINABLE SYSTEM , 2011 .

[33]  N. N. Vakhania,et al.  Random Vectors with Values in Complex Hilbert Spaces , 1997 .

[34]  Wei Xing Zheng,et al.  A complex-valued neural dynamical optimization approach and its stability analysis , 2015, Neural Networks.

[35]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[36]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .