An Improved Dual Neural Network for Solving a Class of Quadratic Programming Problems and Its $k$-Winners-Take-All Application

<para> This paper presents a novel recurrent neural network for solving a class of convex quadratic programming (QP) problems, in which the quadratic term in the objective function is the square of the Euclidean norm of the variable. This special structure leads to a set of simple optimality conditions for the problem, based on which the neural network model is formulated. Compared with existing neural networks for general convex QP, the new model is simpler in structure and easier to implement. The new model can be regarded as an improved version of the dual neural network in the literature. Based on the new model, a simple neural network capable of solving the <formula formulatype="inline"> <tex Notation="TeX">$k$</tex></formula>-winners-take-all (<formula formulatype="inline"> <tex Notation="TeX">$k$</tex></formula>-WTA) problem is formulated. The stability and global convergence of the proposed neural network is proved rigorously and substantiated by simulation results. </para>

[1]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[2]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[3]  Yaser S. Abu-Mostafa,et al.  On the K-Winners-Take-All Network , 1988, NIPS.

[4]  Corneliu A. Marinov,et al.  Stable computational dynamics for a class of circuits with O(N) interconnections capable of KWTA and rank extractions , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Jun Wang,et al.  A dual neural network for kinematic control of redundant robot manipulators , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Xiaolin Hu,et al.  A Recurrent Neural Network for Solving a Class of General Variational Inequalities , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Xiaolin Hu,et al.  Solving Generally Constrained Generalized Linear Variational Inequalities Using the General Projection Neural Networks , 2007, IEEE Transactions on Neural Networks.

[8]  Jun Wang,et al.  A deterministic annealing neural network for convex programming , 1994, Neural Networks.

[9]  Yunong Zhang,et al.  A dual neural network for convex quadratic programming subject to linear equality and inequality constraints , 2002 .

[10]  A. Nagurney,et al.  Projected Dynamical Systems and Variational Inequalities with Applications , 1995 .

[11]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[12]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[13]  Avinash C. Kak,et al.  Vision for Mobile Robot Navigation: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Ivica Kostanic,et al.  Principles of Neurocomputing for Science and Engineering , 2000 .

[15]  A. Auslender.  Optimisation : méthodes numériques , 1976 .

[16]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[17]  Qingshan Liu,et al.  Two k-winners-take-all networks with discontinuous activation functions , 2008, Neural Networks.

[18]  Patrick T. Harker,et al.  Finite-dimensional variational inequality and nonlinear complementarity problems: A survey of theory, algorithms and applications , 1990, Math. Program..

[19]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[20]  Y. Xia,et al.  Further Results on Global Convergence and Stability of Globally Projected Dynamical Systems , 2004 .

[21]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[22]  Xue-Bin Liang,et al.  Global exponential stability of neural networks with globally Lipschitz continuous activations and its application to linear variational inequality problem , 2001, IEEE Trans. Neural Networks.

[23]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[24]  Xing-Bao Gao,et al.  A novel neural network for nonlinear convex programming , 2004, IEEE Trans. Neural Networks.

[25]  G. L. Dempsey,et al.  Circuit implementation of a peak detector neural network , 1993 .

[26]  Xiaolin Hu,et al.  Design of General Projection Neural Networks for Solving Monotone Linear Variational Inequalities and Linear and Quadratic Optimization Problems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[28]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[29]  A. Tesi,et al.  New conditions for global stability of neural networks with application to linear and quadratic programming problems , 1995 .

[30]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[31]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[32]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[33]  Andrew Chi-Sing Leung,et al.  Analysis for a class of winner-take-all model , 1999, IEEE Trans. Neural Networks.

[34]  Jun Wang.  Analogue winner-take-all neural networks for determining maximum and minimum signals , 1994 .

[35]  Jinde Cao,et al.  Solving Quadratic Programming Problems by Delayed Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[36]  Corneliu A. Marinov,et al.  Another K-winners-take-all analog neural network , 2000, IEEE Trans. Neural Networks Learn. Syst..