Solving convex optimization problems using recurrent neural networks in finite time

A recurrent neural network is proposed to deal with the convex optimization problem. By employing a specific nonlinear unit, the proposed neural network is proved to be convergent to the optimal solution in finite time, which increases the computation efficiency dramatically. Compared with most of existing stability conditions, i.e., asymptotical stability and exponential stability, the obtained finite-time stability result is more attractive, and therefore could be considered as a useful supplement to the current literature. In addition, a switching structure is suggested to further speed up the neural network convergence. Moreover, by using the penalty function method, the proposed neural network can be extended straightforwardly to solving the constrained optimization problem. Finally, the satisfactory performance of the proposed approach is illustrated by two simulation examples.

[1]  Min Tan,et al.  Using a nonlinear mechanism to speed up the neural optimization processes , 2002, Proceedings of the IEEE Internatinal Symposium on Intelligent Control.

[2]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[3]  Liqun Qi,et al.  A novel neural network for variational inequalities with linear and nonlinear constraints , 2005, IEEE Transactions on Neural Networks.

[4]  Michael A. Shanblatt,et al.  Linear and quadratic programming neural network analysis , 1992, IEEE Trans. Neural Networks.

[5]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[6]  Elisa Ricci,et al.  Analog neural network for support vector machine learning , 2006, IEEE Transactions on Neural Networks.

[7]  Long Cheng,et al.  A simplified recurrent neural network for solving nonlinear variational inequalities , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[9]  Sanqing Hu,et al.  A Recurrent Neural Network for Non-smooth Nonlinear Programming Problems , 2007, 2007 International Joint Conference on Neural Networks.

[10]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .

[11]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[12]  Long Wang,et al.  Fast information sharing in networks of autonomous agents , 2008, 2008 American Control Conference.

[13]  Long Cheng,et al.  A Neutral-Type Delayed Projection Neural Network for Solving Nonlinear Variational Inequalities , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[14]  Jun Wang,et al.  A general methodology for designing globally convergent optimization neural networks , 1998, IEEE Trans. Neural Networks.

[15]  Long Cheng,et al.  Constrained multi-variable generalized predictive control using a dual neural network , 2007, Neural Computing and Applications.

[16]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[17]  Dennis S. Bernstein,et al.  Finite-Time Stability of Continuous Autonomous Systems , 2000, SIAM J. Control. Optim..

[18]  Long Cheng,et al.  A Recurrent Neural Network for Hierarchical Control of Interconnected Dynamic Systems , 2007, IEEE Transactions on Neural Networks.

[19]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[20]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[21]  Yunong Zhang,et al.  A dual neural network for convex quadratic programming subject to linear equality and inequality constraints , 2002 .

[22]  Youshen Xia.  A new neural network for solving linear programming problems and its application , 1996, IEEE Trans. Neural Networks.

[23]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[24]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.