Continuous and discrete Zhang dynamics for real-time varying nonlinear optimization

Online solution of time-varying nonlinear optimization problems is considered an important issue in the fields of scientific and engineering research. In this study, the continuous-time derivative (CTD) model and two gradient dynamics (GD) models are developed for real-time varying nonlinear optimization (RTVNO). A continuous-time Zhang dynamics (CTZD) model is then generalized and investigated for RTVNO to remedy the weaknesses of CTD and GD models. For possible digital hardware realization, a discrete-time Zhang dynamics (DTZD) model, which can be further reduced to Newton-Raphson iteration (NRI), is also proposed and developed. Theoretical analyses indicate that the residual error of the CTZD model has an exponential convergence, and that the maximum steady-state residual error (MSSRE) of the DTZD model has an O(τ2) pattern with τ denoting the sampling gap. Simulation and numerical results further illustrate the efficacy and advantages of the proposed CTZD and DTZD models for RTVNO.

[1]  Marco Gaviano,et al.  Properties and numerical testing of a parallel global optimization algorithm , 2012, Numerical Algorithms.

[2]  John H. Mathews,et al.  Using MATLAB as a programming language for numerical analysis , 1994 .

[3]  Yunong Zhang,et al.  Zhang Neural Network Versus Gradient Neural Network for Solving Time-Varying Linear Inequalities , 2011, IEEE Transactions on Neural Networks.

[4]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[5]  Neculai Andrei.  An accelerated subspace minimization three-term conjugate gradient algorithm for unconstrained optimization , 2013, Numerical Algorithms.

[6]  Fusheng Wang,et al.  A model-hybrid approach for unconstrained optimization problems , 2013, Numerical Algorithms.

[7]  Eugenius Kaszkurewicz,et al.  A Control-Theoretic Approach to the Design of Zero Finding Numerical Methods , 2007, IEEE Transactions on Automatic Control.

[8]  José Mario Martínez,et al.  Handling infeasibility in a large-scale nonlinear optimization algorithm , 2012, Numerical Algorithms.

[9]  Yunong Zhang,et al.  Continuous and discrete time Zhang dynamics for time-varying 4th root finding , 2010, Numerical Algorithms.

[10]  A. Latif,et al.  Variational Analysis, Optimization, and Fixed Point Theory , 2013 .

[11]  Harvey Lipkin,et al.  A dynamic quasi-Newton method for uncalibrated visual servoing , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[12]  Masoud Ahookhosh,et al.  An inexact line search approach using modified nonmonotone strategy for unconstrained optimization , 2013, Numerical Algorithms.

[13]  Liqun Qi,et al.  Neurodynamical Optimization , 2004, J. Glob. Optim..

[14]  George Lindfield,et al.  Numerical Methods Using MATLAB , 1998 .

[15]  Guoqiang Wang,et al.  Complexity analysis and numerical implementation of primal-dual interior-point methods for convex quadratic optimization based on a finite barrier , 2012, Numerical Algorithms.

[16]  Eugenius Kaszkurewicz,et al.  Control Perspectives on Numerical Algorithms And Matrix Problems (Advances in Design and Control) (Advances in Design and Control 10) , 2006 .

[17]  Changyin Sun,et al.  A novel neural dynamical approach to convex quadratic program and its efficient applications , 2009, Neural Networks.

[18]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[19]  Yunong Zhang,et al.  Zhang Neural Networks and Neural-Dynamic Method , 2011 .

[20]  Martin J. Gander,et al.  Optimization of Schwarz waveform relaxation over short time windows , 2012, Numerical Algorithms.

[21]  Abhishek K Gupta,et al.  Numerical Methods using MATLAB , 2014, Apress.

[22]  Shuzhi Sam Ge,et al.  A unified quadratic-programming-based dynamical system approach to joint torque optimization of physically constrained redundant manipulators , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Paul Tseng.  Control perspectives on numerical algorithms and matrix problems , 2008, Math. Comput..

[24]  Shuzhi Sam Ge,et al.  Design and analysis of a general recurrent neural network model for time-varying matrix inversion , 2005, IEEE Transactions on Neural Networks.

[25]  Nicholas G. Maratos,et al.  A Nonfeasible Gradient Projection Recurrent Neural Network for Equality-Constrained Optimization Problems , 2008, IEEE Transactions on Neural Networks.

[26]  Hiroshi Yabe,et al.  Conjugate gradient methods based on secant conditions that generate descent search directions for unconstrained optimization , 2012, J. Comput. Appl. Math..

[27]  Ke Chen,et al.  Performance Analysis of Gradient Neural Network Exploited for Online Time-Varying Matrix Inversion , 2009, IEEE Transactions on Automatic Control.

[28]  Yunong Zhang,et al.  Different Complex ZFs Leading to Different Complex ZNN Models for Time-Varying Complex Generalized Inverse Matrices , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Masoud Ahookhosh,et al.  An efficient nonmonotone trust-region method for unconstrained optimization , 2011, Numerical Algorithms.

[30]  Jinde Cao,et al.  Neurodynamic System Theory and Applications , 2013 .

[31]  Eugenius Kaszkurewicz,et al.  Iterative methods as dynamical systems with feedback control , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[32]  Zhen Li,et al.  Discrete-time ZD, GD and NI for solving nonlinear time-varying equations , 2012, Numerical Algorithms.

[33]  Lin Xiao,et al.  Finite-time solution to nonlinear equation using recurrent neural dynamics with a specially-constructed activation function , 2015, Neurocomputing.