Performance analysis of gradient neural network exploited for online time-varying quadratic minimization and equality-constrained quadratic programming

In this paper, the performance of a gradient neural network (GNN), which was designed intrinsically for solving static problems, is investigated, analyzed and simulated in the situation of time-varying coefficients. It is theoretically proved that the gradient neural network for online solution of time-varying quadratic minimization (QM) and quadratic programming (QP) problems could only approximately approach the time-varying theoretical solution, instead of converging exactly. That is, the steady-state error between the GNN solution and the theoretical solution can not decrease to zero. In order to understand the situation better, the upper bound of such an error is estimated firstly, and then the global exponential convergence rate is investigated for such a GNN when approaching an error bound. Computer-simulation results, including those based on a six-link robot manipulator, further substantiate the performance analysis of the GNN exploited to solve online time-varying QM and QP problems.

[1]  Yunong Zhang,et al.  Zhang neural network for online solution of time-varying convex quadratic program subject to time-varying linear-equality constraints , 2009 .

[2]  Leon O. Chua,et al.  Nonlinear programming without computation , 1984 .

[3]  Jun Wang,et al.  A recurrent neural network for solving Sylvester equation with time-varying coefficients , 2002, IEEE Trans. Neural Networks.

[4]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Jiashu Zhang,et al.  Nonlinear dynamic system identification using pipelined functional link artificial recurrent neural network , 2009, Neurocomputing.

[6]  Shuzhong Zhang,et al.  New Results on Quadratic Minimization , 2003, SIAM J. Optim..

[7]  Ekkehard W. Sachs,et al.  Inexact SQP Interior Point Methods and Large Scale Optimal Control Problems , 1999, SIAM J. Control. Optim..

[8]  A. Cichocki,et al.  Neural networks for solving systems of linear equations and related problems , 1992 .

[9]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[10]  F. Cao,et al.  Projection type neural network and its convergence analysis , 2006 .

[11]  Wu Li.  Error Bounds for Piecewise Convex Quadratic Programs and Applications , 1995 .

[12]  Wei Bian,et al.  Subgradient-Based Neural Networks for Nonsmooth Nonconvex Optimization Problems , 2009, IEEE Transactions on Neural Networks.

[13]  Yunong Zhang,et al.  Bi-criteria velocity minimization of robot manipulators using LVI-based primal-dual neural network and illustrated via PUMA560 robot arm , 2010, Robotica.

[14]  Jun Wang,et al.  Electronic realisation of recurrent neural network for solving simultaneous linear equations , 1992 .

[15]  Tor Arne Johansen,et al.  Constrained nonlinear control allocation with singularity avoidance using sequential quadratic programming , 2004, IEEE Transactions on Control Systems Technology.

[16]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[17]  Pierre Apkarian,et al.  Robust Control via Sequential Semidefinite Programming , 2002, SIAM J. Control. Optim..

[18]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[19]  Yunong Zhang,et al.  Revisit the Analog Computer and Gradient-Based Neural System for Matrix Inversion , 2005, Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005..

[20]  Giovanni Costantini,et al.  Quasi-Lagrangian Neural Network for Convex Quadratic Optimization , 2008, IEEE Transactions on Neural Networks.

[21]  Jun Wang,et al.  RECURRENT NEURAL NETWORKS FOR REAL-TIME COMPUTATION OF INVERSE KINEMATICS OF REDUNDANT MANIPULATORS , 2004 .

[22]  Wu Li,et al.  A New Algorithm for Solving Strictly Convex Quadratic Programs , 1997, SIAM J. Optim..

[23]  N. Grudinin.  Reactive power optimization using successive quadratic programming method , 1998 .

[24]  Yunong Zhang,et al.  O(N 2)-Operation Approximation of Covariance Matrix Inverse in Gaussian Process Regression Based on Quasi-Newton BFGS Method , 2007, Commun. Stat. Simul. Comput..

[25]  Ke Chen,et al.  MATLAB Simulink modeling and simulation of LVI-based primal-dual neural network for solving linear and quadratic programs , 2009, Neurocomputing.

[26]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[27]  Yunong Zhang,et al.  Global exponential convergence and stability of Wang neural network for solving online linear equations , 2008 .

[28]  Mauro Forti,et al.  Lyapunov Method and Convergence of the Full-Range Model of CNNs , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[29]  Shuzhi Sam Ge,et al.  Design and analysis of a general recurrent neural network model for time-varying matrix inversion , 2005, IEEE Transactions on Neural Networks.

[30]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[31]  Ke Chen,et al.  Author's Personal Copy Robotics and Autonomous Systems Repetitive Motion of Redundant Robots Planned by Three Kinds of Recurrent Neural Networks and Illustrated with a Four-link Planar Manipulator's Straight-line Example , 2022 .

[32]  Fernando Morgado Dias,et al.  A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function , 2007, Neurocomputing.

[33]  Hyun Myung,et al.  Time-varying two-phase optimization and its application to neural-network learning , 1997, IEEE Trans. Neural Networks.

[34]  Paul Tseng,et al.  Approximation Bounds for Quadratic Optimization with Homogeneous Quadratic Constraints , 2007, SIAM J. Optim..

[35]  Michael A. Shanblatt,et al.  Linear and quadratic programming neural network analysis , 1992, IEEE Trans. Neural Networks.

[36]  Peter Sincak,et al.  Machine Intelligence: Quo Vadis? , 2004 .

[37]  Yunong Zhang,et al.  A set of nonlinear equations and inequalities arising in robotics and its online solution via a primal neural network , 2006, Neurocomputing.