Discrete-Time Zhang Neural Networks for Time-Varying Nonlinear Optimization

As a special kind of recurrent neural networks, Zhang neural network (ZNN) has been successfully applied to various time-variant problems solving. In this paper, we present three Zhang et al. discretization (ZeaD) formulas, including a special two-step ZeaD formula, a general two-step ZeaD formula, and a general five-step ZeaD formula, and prove that the special and general two-step ZeaD formulas are convergent while the general five-step ZeaD formula is not zero-stable and thus is divergent. Then, to solve the time-varying nonlinear optimization (TVNO) in real time, based on the Taylor series expansion and the above two convergent two-step ZeaD formulas, we discrete the continuous-time ZNN (CTZNN) model of TVNO and thus get a special two-step discrete-time ZNN (DTZNN) model and a general two-step DTZNN model. Theoretical analyses indicate that the sequence generated by the first DTZNN model is divergent, while the sequence generated by the second DTZNN model is convergent. Furthermore, for the step-size of the second DTZNN model, its tight upper bound and the optimal step-size are also discussed. Finally, some numerical results and comparisons are provided and analyzed to substantiate the efficacy of the proposed DTZNN models.

[1]  Ke Chen,et al.  Zhang Neural Network Versus Gradient Neural Network for Online Time-Varying Quadratic Function Minimization , 2008, ICIC.

[2]  Yang Shi,et al.  General four-step discrete-time zeroing and derivative dynamics applied to time-varying nonlinear optimization , 2019, J. Comput. Appl. Math..

[3]  Long Jin,et al.  Discrete-time Zhang neural network of O(τ3) pattern for time-varying matrix pseudoinversion with application to manipulator motion generation , 2014, Neurocomputing.

[4]  Min Sun,et al.  A new descent memory gradient method and its global convergence , 2011, J. Syst. Sci. Complex..

[5]  Jun Wang,et al.  A Bi-Projection Neural Network for Solving Constrained Quadratic Optimization Problems , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[6]  J. Lambert.  Numerical Methods for Ordinary Differential Equations , 1991 .

[7]  Long Jin,et al.  Taylor-type 1-step-ahead numerical differentiation rule for first-order derivative approximation and ZNN discretization , 2015, J. Comput. Appl. Math..

[8]  Gene H. Golub,et al.  SOR-like Methods for Augmented Systems , 2001 .

[10]  Jun Wang,et al.  A recurrent neural network for solving a class of generalized convex optimization problems , 2013, Neural Networks.

[11]  Dongsheng Guo,et al.  Design and analysis of two discrete-time ZD algorithms for time-varying nonlinear minimization , 2017, Numerical Algorithms.

[12]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[13]  Yunong Zhang,et al.  ZFD formula 4IgSFD_Y applied to future minimization , 2017 .

[14]  Min Sun,et al.  New hybrid conjugate gradient projection method for the convex constrained equations , 2016 .

[15]  Yunong Zhang,et al.  Three-step general discrete-time Zhang neural network design and application to time-variant matrix inversion , 2018, Neurocomputing.

[16]  Yunong Zhang,et al.  Any ZeaD Formula of Six Instants Having No Quartic or Higher Precision with Proof , 2018, 2018 5th International Conference on Systems and Informatics (ICSAI).

[17]  Huai-Ning Wu,et al.  Adaptive output synchronization of complex delayed dynamical networks with output coupling , 2014, Neurocomputing.

[18]  Long Jin,et al.  Discrete-Time Zhang Neural Network for Online Time-Varying Nonlinear Optimization With Application to Manipulator Motion Generation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Sitian Qin,et al.  Neural network for nonsmooth pseudoconvex optimization with general convex constraints , 2018, Neural Networks.

[20]  Yiju Wang,et al.  Generalized Peaceman-Rachford splitting method for multiple-block separable convex programming with applications to robust PCA , 2017 .

[21]  Desmond J. Higham,et al.  Numerical Methods for Ordinary Differential Equations - Initial Value Problems , 2010, Springer undergraduate mathematics series.