A New Recurrent Neural Network for Solving Convex Quadratic Programming Problems With an Application to the $k$-Winners-Take-All Problem

In this paper, a new recurrent neural network is proposed for solving convex quadratic programming (QP) problems. Compared with existing neural networks, the proposed one features global convergence property under weak conditions, low structural complexity, and no calculation of matrix inverse. It serves as a competitive alternative in the neural network family for solving linear or quadratic programming problems. In addition, it is found that by some variable substitution, the proposed network turns out to be an existing model for solving minimax problems. In this sense, it can be also viewed as a special case of the minimax neural network. Based on this scheme, a k-winners-take-all (k-WTA) network with O(n) complexity is designed, which is characterized by simple structure, global convergence, and capability to deal with some ill cases. Numerical simulations are provided to validate the theoretical results obtained. More importantly, the network design method proposed in this paper has great potential to inspire other competitive inventions along the same line.

[1]  Xiaolin Hu,et al.  Solving the k-Winners-Take-All Problem and the Oligopoly Cournot-Nash Equilibrium Problem Using the General Projection Neural Networks , 2007, ICONIP.

[2]  Jinde Cao,et al.  Globally Projected Dynamical System and Its Applications , 2005 .

[3]  L. Liao,et al.  A neural network for monotone variational inequalities with linear constraints , 2003 .

[4]  Wei Bian,et al.  A project neural network for solving degenerate convex quadratic program , 2007, Neurocomputing.

[5]  Elisa Ricci,et al.  Analog neural network for support vector machine learning , 2006, IEEE Transactions on Neural Networks.

[6]  Kiichi Urahama,et al.  K-winners-take-all circuit with O(N) complexity , 1995, IEEE Trans. Neural Networks.

[7]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[8]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[9]  Corneliu A. Marinov,et al.  Stable computational dynamics for a class of circuits with O(N) interconnections capable of KWTA and rank extractions , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[10]  Jun Wang Analysis and design of a recurrent neural network for linear programming , 1993 .

[11]  Jun Wang,et al.  A recurrent neural network with exponential convergence for solving convex quadratic program and related linear piecewise equations , 2004, Neural Networks.

[12]  Xiaolin Hu,et al.  A Recurrent Neural Network for Solving a Class of General Variational Inequalities , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Jinde Cao,et al.  Solving Quadratic Programming Problems by Delayed Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[14]  Gang Feng,et al.  On Convergence Conditions of an Extended Projection Neural Network , 2005, Neural Computation.

[15]  Corneliu A. Marinov,et al.  Another K-winners-take-all analog neural network , 2000, IEEE Trans. Neural Networks Learn. Syst..

[16]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[18]  Jinde Cao,et al.  A Delayed Neural Network Method for Solving Convex Optimization Problems , 2006, Int. J. Neural Syst..

[19]  Xiaolin Hu,et al.  An Improved Dual Neural Network for Solving a Class of Quadratic Programming Problems and Its $k$-Winners-Take-All Application , 2008, IEEE Transactions on Neural Networks.

[20]  Xing-Bao Gao,et al.  A novel neural network for nonlinear convex programming , 2004, IEEE Trans. Neural Networks.

[21]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[22]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[23]  Liqun Qi,et al.  A novel neural network for variational inequalities with linear and nonlinear constraints , 2005, IEEE Transactions on Neural Networks.

[24]  Xiaolin Hu,et al.  Solving Generally Constrained Generalized Linear Variational Inequalities Using the General Projection Neural Networks , 2007, IEEE Transactions on Neural Networks.

[25]  W K Chen,et al.  A high-performance neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[26]  Nezam Mahdavi-Amiri,et al.  An efficient simplified neural network for solving linear and quadratic programming problems , 2006, Appl. Math. Comput..

[27]  Youshen Xia,et al.  Neural network for solving linear programming problems with bounded variables , 1995, IEEE Trans. Neural Networks.

[28]  Jinde Cao,et al.  A delayed neural network for solving linear projection equations and its analysis , 2005, IEEE Transactions on Neural Networks.

[29]  Insley B. Pyne,et al.  Linear programming on an electronic analogue computer , 1956, Transactions of the American Institute of Electrical Engineers, Part I: Communication and Electronics.

[30]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[31]  Corneliu A. Marinov,et al.  Performance analysis for a K-winners-take-all analog neural network: basic theory , 2003, IEEE Trans. Neural Networks.

[32]  B. Sekerkiran,et al.  A CMOS K-winners-take-all circuit with O(n) complexity , 1999 .

[33]  Li-Zhi Liao,et al.  A Novel Neural Network for a Class of Convex Quadratic Minimax Problems , 2006, Neural Computation.

[34]  S Barbaros A CMOS K-Winners-Take-All Circuit With Complexity , 1999 .

[35]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[36]  Youshen Xia,et al.  A recurrent neural network for nonlinear convex optimization subject to nonlinear inequality constraints , 2004, IEEE Trans. Circuits Syst. I Regul. Pap..

[37]  Edgar Sanchez-Sinencio,et al.  Nonlinear switched capacitor 'neural' networks for optimization problems , 1990 .

[38]  Xiaolin Hu,et al.  Design of General Projection Neural Networks for Solving Monotone Linear Variational Inequalities and Linear and Quadratic Optimization Problems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[40]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[41]  Youshen Xia A new neural network for solving linear programming problems and its application , 1996, IEEE Trans. Neural Networks.

[42]  Boon Poh Tan,et al.  Semiparallel rank order filtering in analog VLSI , 2001 .

[43]  Jinde Cao,et al.  A simple and high performance neural network for quadratic programming problems , 2001, Appl. Math. Comput..

[44]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.