Time-varying square roots finding via Zhang dynamics versus gradient dynamics and the former's link and new explanation to Newton-Raphson iteration

Different from conventional gradient-based neural dynamics, a special class of neural dynamics have been proposed by Zhang et al. since 12 March 2001 for online solution of time-varying and static (or termed, time-invariant) problems (e.g., nonlinear equations). The design of Zhang dynamics (ZD) is based on the elimination of an indefinite error-function, instead of the elimination of a square-based positive or at least lower-bounded energy-function usually associated with gradient dynamics (GD) and/or Hopfield-type neural networks. In this paper, we generalize, develop, investigate and compare the continuous-time ZD (CTZD) and GD models for online solution of time-varying and static square roots. In addition, a simplified continuous-time ZD (S-CTZD) and discrete-time ZD (DTZD) models are generated for static scalar-valued square roots finding. In terms of such scalar square roots finding problem, the Newton iteration (also termed, Newton-Raphson iteration) is found to be a special case of the DTZD models (by focusing on the static-problem solving, utilizing the linear activation function and fixing the step-size to be 1). Computer-simulation results via a power-sigmoid activation function further demonstrate the efficacy of the ZD solvers for online scalar (time-varying and static) square roots finding, in addition to the DTZD's link and new explanation to Newton-Raphson iteration.

[1]  Yunong Zhang,et al.  Solution of nonlinear equations by continuous- and discrete-time Zhang dynamics and more importantly their links to Newton iteration , 2009, 2009 7th International Conference on Information, Communications and Signal Processing (ICICS).

[2]  Yunong Zhang,et al.  Time-series Gaussian Process Regression Based on Toeplitz Computation of O(N2) Operations and O(N)-level Storage , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[3]  John H. Mathews,et al.  Using MATLAB as a programming language for numerical analysis , 1994 .

[4]  Saeid Abbasbandy,et al.  Improving Newton-Raphson method for nonlinear equations by modified Adomian decomposition method , 2003, Appl. Math. Comput..

[5]  Nicholas J. Higham,et al.  Stable iterations for the matrix square root , 1997, Numerical Algorithms.

[6]  Shuzhi Sam Ge,et al.  Design and analysis of a general recurrent neural network model for time-varying matrix inversion , 2005, IEEE Transactions on Neural Networks.

[7]  Yu-Nong Zhang,et al.  Zhang Neural Network for Linear Time-Varying Equation Solving and its Robotic Application , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[8]  Dongsheng Guo,et al.  Comparison on Zhang neural dynamics and gradient-based neural dynamics for online solution of nonlinear time-varying equation , 2011, Neural Computing and Applications.

[9]  Yunong Zhang.  Towards Piecewise-Linear Primal Neural Networks for Optimization and Redundant Robotics , 2006, 2006 IEEE International Conference on Networking, Sensing and Control.

[10]  Yunong Zhang,et al.  A set of nonlinear equations and inequalities arising in robotics and its online solution via a primal neural network , 2006, Neurocomputing.

[11]  Zhun Cai,et al.  Improved generalized Atkin algorithm for computing square roots in finite fields , 2006, Inf. Process. Lett..

[12]  P. Morse,et al.  Principles of Numerical Analysis , 1954 .

[13]  Yunong Zhang,et al.  Zhang neural network for online solution of time-varying convex quadratic program subject to time-varying linear-equality constraints , 2009 .

[14]  Sanjit K. Mitra,et al.  Digital Signal Processing: A Computer-Based Approach , 1997 .

[15]  Daisuke Takahashi,et al.  Implementation of multiple-precision parallel division and square root on distributed-memory parallel computers , 2000, Proceedings 2000. International Workshop on Parallel Processing.

[16]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[17]  Binghuang Cai,et al.  From Zhang Neural Network to Newton Iteration for Matrix Inversion , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[18]  George Lindfield,et al.  Numerical Methods Using MATLAB , 1998 .

[19]  Yunong Zhang,et al.  Simulation and Comparison of Zhang Neural Network and Gradient Neural Network Solving for Time-Varying Matrix Square Roots , 2008, 2008 Second International Symposium on Intelligent Information Technology Application.

[20]  Mário Basto,et al.  A new iterative method to compute nonlinear equations , 2006, Appl. Math. Comput..

[21]  Jianbo Qian,et al.  How much precision is needed to compare two sums of square roots of integers? , 2006, Inf. Process. Lett..

[22]  Yunong Zhang,et al.  Comparison on Gradient-Based Neural Dynamics and Zhang Neural Dynamics for Online Solution of Nonlinear Equations , 2008, ISICA.

[23]  Yunong Zhang,et al.  Revisit the Analog Computer and Gradient-Based Neural System for Matrix Inversion , 2005, Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005..

[24]  Binghuang Cai,et al.  Zhang neural network without using time-derivative information for constant and time-varying matrix inversion , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[25]  Jun Wang,et al.  A recurrent neural network for solving Sylvester equation with time-varying coefficients , 2002, IEEE Trans. Neural Networks.