Finite time dual neural networks with a tunable activation function for solving quadratic programming problems and its application

In this paper, finite time dual neural networks with a new activation function are presented to solve quadratic programming problems. The activation function has two tunable parameters, which give more flexibility to design the neural networks. By Lyapunov theorem, finite-time stability can be derived for the proposed neural networks, and the actual optimal solutions of the quadratic programming problems can be obtained in finite time interval. Different from the existing recurrent neural networks for solving the quadratic programming problems, the neural networks of this paper have a faster convergent speed, at the same time, they can reduce oscillation when delay appears, and have less sensitivity to additive noise with careful selection of the parameters. Simulations are presented to evaluate the performance of the neural networks with the tunable activation function. In addition, the proposed neural networks are applied to estimate parameters for an energy model of belt conveyors. The effectiveness of our methods are validated by theoretical analysis and numerical simulations.

[1]  Kate Smith-Miles,et al.  Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research , 1999, INFORMS J. Comput..

[2]  Suk-Geun Hwang,et al.  Cauchy's Interlace Theorem for Eigenvalues of Hermitian Matrices , 2004, Am. Math. Mon..

[3]  Jinde Cao,et al.  A delayed neural network for solving linear projection equations and its analysis , 2005, IEEE Transactions on Neural Networks.

[4]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[5]  Dennis S. Bernstein,et al.  Finite-Time Stability of Continuous Autonomous Systems , 2000, SIAM J. Control. Optim..

[6]  Yuehua Huang,et al.  Global finite-time stabilisation for a class of nonlinear systems , 2012, Int. J. Syst. Sci..

[7]  Long Cheng,et al.  A Neutral-Type Delayed Projection Neural Network for Solving Nonlinear Variational Inequalities , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[8]  X. Xia,et al.  Semi-global finite-time observers for nonlinear systems , 2008, Autom..

[9]  Shuai Li,et al.  Selective Positive–Negative Feedback Produces the Winner-Take-All Competition in Recurrent Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Jinde Cao,et al.  Solving Quadratic Programming Problems by Delayed Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[11]  Qingshan Liu,et al.  A One-Layer Projection Neural Network for Nonsmooth Optimization Subject to Linear Equalities and Bound Constraints , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Mahesan Niranjan,et al.  The use of recurrent neural networks for classification , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[13]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[14]  M. Vidyasagar.  An Elementary Derivation of the Large Deviation Rate Function for Finite State Markov Chains , 2014 .

[15]  W K Chen,et al.  A high-performance neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[16]  Xiaohua Xia,et al.  Adaptive Parameter Estimation for an Energy Model of Belt Conveyor with DC Motor , 2014 .

[17]  Long Cheng,et al.  A Delayed Projection Neural Network for Solving Linear Variational Inequalities , 2009, IEEE Transactions on Neural Networks.

[18]  Qingshan Liu,et al.  Two k-winners-take-all networks with discontinuous activation functions , 2008, Neural Networks.

[19]  Long Cheng,et al.  Recurrent Neural Network for Non-Smooth Convex Optimization Problems With Application to the Identification of Genetic Regulatory Networks , 2011, IEEE Transactions on Neural Networks.

[20]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[21]  Xiaohua Xia,et al.  A new energy calculation model of belt conveyor , 2009, AFRICON 2009.

[22]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[23]  Jun Wang,et al.  A dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits , 2003, IEEE Trans. Neural Networks.

[24]  J. Hale,et al.  Ordinary Differential Equations , 2019, Fundamentals of Numerical Mathematics for Physicists and Engineers.

[25]  Long Cheng,et al.  Solving convex optimization problems using recurrent neural networks in finite time , 2009, 2009 International Joint Conference on Neural Networks.

[26]  Yunong Zhang,et al.  A dual neural network for convex quadratic programming subject to linear equality and inequality constraints , 2002 .

[27]  Jun Wang,et al.  Analysis and Design of a $k$ -Winners-Take-All Model With a Single State Variable and the Heaviside Step Activation Function , 2010, IEEE Transactions on Neural Networks.

[28]  Jun Wang,et al.  A One-Layer Recurrent Neural Network for Constrained Nonsmooth Optimization , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[30]  Yunong Zhang,et al.  Performance analysis of gradient neural network exploited for online time-varying quadratic minimization and equality-constrained quadratic programming , 2011, Neurocomputing.

[31]  Jun Wang,et al.  A recurrent neural network with exponential convergence for solving convex quadratic program and related linear piecewise equations , 2004, Neural Networks.

[32]  Jinde Cao,et al.  Global exponential stability of discrete-time recurrent neural network for solving quadratic programming problems subject to linear constraints , 2011, Neurocomputing.

[33]  Xiaojun Chen,et al.  Smoothing Neural Network for Constrained Non-Lipschitz Optimization With Applications , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Shuai Li,et al.  Model-free control of Lorenz chaos using an approximate optimal control strategy , 2012 .

[35]  Ying Tan,et al.  Solving for a quadratic programming with a quadratic constraint based on a neural network frame , 2000, Neurocomputing.

[36]  Xiaolin Hu,et al.  A New Recurrent Neural Network for Solving Convex Quadratic Programming Problems With an Application to the $k$-Winners-Take-All Problem , 2009, IEEE Transactions on Neural Networks.

[37]  Shuai Li,et al.  Bluetooth aided mobile phone localization , 2014, ACM Trans. Embed. Comput. Syst..

[38]  Dennis S. Bernstein,et al.  Geometric homogeneity with applications to finite-time stability , 2005, Math. Control. Signals Syst..

[39]  Xiaohua Xia,et al.  Modeling and energy efficiency optimization of belt conveyors , 2011 .

[40]  Xiaolin Hu,et al.  Applications of the general projection neural network in solving extended linear-quadratic programming problems with linear constraints , 2009, Neurocomputing.

[41]  Zhanshan Wang,et al.  A projection neural network with mixed delays for solving linear variational inequality , 2014, Neurocomputing.

[42]  Shuai Li,et al.  Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks , 2012, Neurocomputing.

[43]  Changyin Sun,et al.  A novel neural dynamical approach to convex quadratic program and its efficient applications , 2009, Neural Networks.

[44]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[45]  Edgar Sanchez-Sinencio,et al.  Nonlinear switched capacitor 'neural' networks for optimization problems , 1990 .

[46]  Weibing Gao,et al.  Variable structure control of nonlinear systems: a new approach , 1993, IEEE Trans. Ind. Electron..

[47]  John G. Harris,et al.  Noise-Robust Automatic Speech Recognition Using a Predictive Echo State Network , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[48]  Qingshan Liu,et al.  Finite-Time Convergent Recurrent Neural Network With a Hard-Limiting Activation Function for Constrained Optimization With Piecewise-Linear Objective Functions , 2011, IEEE Transactions on Neural Networks.

[49]  Yangming Li,et al.  A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application , 2013, Neural Networks.

[50]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .