New Discrete-Time ZNN Models for Least-Squares Solution of Dynamic Linear Equation System With Time-Varying Rank-Deficient Coefficient

In this brief, a new one-step-ahead numerical differentiation rule called six-instant <inline-formula> <tex-math notation="LaTeX">$g$ </tex-math></inline-formula>-cube finite difference (6I<inline-formula> <tex-math notation="LaTeX">$g$ </tex-math></inline-formula>CFD) formula is proposed for the first-order derivative approximation with higher precision than existing finite difference formulas (i.e., Euler and Taylor types). Subsequently, by exploiting the proposed 6I<inline-formula> <tex-math notation="LaTeX">$g$ </tex-math></inline-formula>CFD formula to discretize the continuous-time Zhang neural network model, two new-type discrete-time ZNN (DTZNN) models, namely, new-type DTZNNK and DTZNNU models, are designed and generalized to compute the least-squares solution of dynamic linear equation system with time-varying rank-deficient coefficient in real time, which is quite different from the existing ZNN-related studies on solving continuous-time and discrete-time (dynamic or static) linear equation systems in the context of full-rank coefficients. Specifically, the corresponding dynamic normal equation system, of which the solution exactly corresponds to the least-squares solution of dynamic linear equation system, is elegantly introduced to solve such a rank-deficient least-squares problem efficiently and accurately. Theoretical analyses show that the maximal steady-state residual errors of the two new-type DTZNN models have an <inline-formula> <tex-math notation="LaTeX">$O(g^{4})$ </tex-math></inline-formula> pattern, where <inline-formula> <tex-math notation="LaTeX">$g$ </tex-math></inline-formula> denotes the sampling gap. Comparative numerical experimental results further substantiate the superior computational performance of the new-type DTZNN models to solve the rank-deficient least-squares problem of dynamic linear equation systems.

[1]  Long Jin,et al.  Taylor $O(h^{3})$ Discretization of ZNN Models for Dynamic Equality-Constrained Quadratic Programming With Application to Manipulators , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Yunong Zhang,et al.  New discrete-time ZNN models and numerical algorithms derived from a new Zhang function for time-varying linear equations solving , 2013, 2013 Ninth International Conference on Natural Computation (ICNC).

[3]  Jun Wang,et al.  Electronic realisation of recurrent neural network for solving simultaneous linear equations , 1992 .

[4]  Ke Chen.  Implicit dynamic system for online simultaneous linear equations solving , 2013 .

[5]  Haibo He,et al.  A New Discrete-Continuous Algorithm for Radial Basis Function Networks Construction , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[6]  B. Siciliano,et al.  Second-order kinematic control of robot manipulators with Jacobian damped least-squares inverse: theory and experiments , 1997 .

[7]  István Kollár,et al.  Generalization of a total least squares problem in frequency-domain system identification , 2002, IEEE Trans. Instrum. Meas..

[8]  Shuai Li,et al.  A Novel Recurrent Neural Network for Manipulator Control With Improved Noise Tolerance , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[9]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[10]  Long Jin,et al.  Discrete-time Zhang neural network of O(τ3) pattern for time-varying matrix pseudoinversion with application to manipulator motion generation , 2014, Neurocomputing.

[11]  Hongjiong Tian.  Accelerate overrelaxation methods for rank deficient linear systems , 2003, Appl. Math. Comput..

[12]  Yunong Zhang,et al.  Zhang Neural Networks and Neural-Dynamic Method , 2011 .

[13]  Mei Liu,et al.  Performance analyses of recurrent neural network models exploited for online time-varying nonlinear optimization , 2016, Comput. Sci. Inf. Syst..

[14]  Le-ping Sun,et al.  On an iterative method for solving the least squares problem of rank-deficient systems , 2015, Int. J. Comput. Math..

[15]  Yunong Zhang,et al.  Convergence analysis of Zhang neural networks solving time-varying linear equations but without using time-derivative information , 2010, IEEE ICCA 2010.

[16]  Ke Chen.  Robustness analysis of Wang neural network for online linear equation solving , 2012 .

[17]  Long Cheng,et al.  Recurrent Neural Network for Non-Smooth Convex Optimization Problems With Application to the Identification of Genetic Regulatory Networks , 2011, IEEE Transactions on Neural Networks.

[18]  Dongsheng Guo,et al.  Common nature of learning between BP-type and Hopfield-type neural networks , 2015, Neurocomputing.

[19]  Lin Xiao,et al.  A convergence-accelerated Zhang neural network and its solution application to Lyapunov equation , 2016, Neurocomputing.

[20]  Lin Xiao,et al.  A nonlinearly activated neural dynamics and its finite-time solution to time-varying nonlinear equation , 2016, Neurocomputing.

[21]  Ian D. Walker,et al.  Overview of damped least-squares methods for inverse kinematics of robot manipulators , 1995, J. Intell. Robotic Syst..

[22]  Ying Wang,et al.  Different ZFs Leading to Various ZNN Models Illustrated via Online Solution of Time-Varying Underdetermined Systems of Linear Equations with Robotic Application , 2013, ISNN.

[23]  Dongsheng Guo,et al.  Common Nature of Learning Between Back-Propagation and Hopfield-Type Neural Networks for Generalized Matrix Inversion With Simplified Models , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Shuai Li,et al.  Selective Positive–Negative Feedback Produces the Winner-Take-All Competition in Recurrent Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[26]  Long Jin,et al.  Discrete-Time Zhang Neural Network for Online Time-Varying Nonlinear Optimization With Application to Manipulator Motion Generation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[27]  H. Anton.  Elementary Linear Algebra , 1970 .

[28]  Dongsheng Guo,et al.  Design and analysis of two discrete-time ZD algorithms for time-varying nonlinear minimization , 2017, Numerical Algorithms.

[29]  Ke Chen,et al.  Robustness analysis of a hybrid of recursive neural dynamics for online matrix inversion , 2016, Appl. Math. Comput..

[30]  Shuzhi Sam Ge,et al.  Design and analysis of a general recurrent neural network model for time-varying matrix inversion , 2005, IEEE Transactions on Neural Networks.

[31]  Dongsheng Guo,et al.  Novel Recurrent Neural Network for Time-Varying Problems Solving [Research Frontier] , 2012, IEEE Computational Intelligence Magazine.

[32]  Long Jin,et al.  Neural network-based discrete-time Z-type model of high accuracy in noisy environments for solving dynamic system of linear equations , 2016, Neural Computing and Applications.

[33]  Xiaofeng Song,et al.  Sequential local least squares imputation estimating missing value of microarray data , 2008, Comput. Biol. Medicine.

[34]  Nicholas G. Maratos,et al.  A Nonfeasible Gradient Projection Recurrent Neural Network for Equality-Constrained Optimization Problems , 2008, IEEE Transactions on Neural Networks.

[35]  Chenfu Yi,et al.  Linear Simultaneous Equations' Neural Solution and Its Application to Convex Quadratic Programming with Equality-Constraint , 2013, J. Appl. Math..

[36]  Chenfu Yi,et al.  Comparison on neural solvers for the Lyapunov matrix equation with stationary & nonstationary coefficients , 2013 .

[37]  Desmond J. Higham,et al.  Numerical Methods for Ordinary Differential Equations - Initial Value Problems , 2010, Springer undergraduate mathematics series.

[38]  Long Jin,et al.  Infinitely many Zhang functions resulting in various ZNN models for time-varying matrix inversion with link to Drazin inverse , 2015, Inf. Process. Lett..

[39]  Leslie Foster,et al.  Algorithm 853: An efficient algorithm for solving rank-deficient least squares problems , 2006, TOMS.

[40]  Haizhou Li,et al.  A Discrete-Time Neural Network for Optimization Problems With Hybrid Constraints , 2010, IEEE Transactions on Neural Networks.

[41]  Zekeriya Uykan,et al.  Fast-Convergent Double-Sigmoid Hopfield Neural Network as Applied to Optimization Problems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Yunong Zhang,et al.  Analogue recurrent neural network for linear algebraic equation solving , 2008 .

[43]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .