From different ZFs to different ZNN models accelerated via Li activation functions to finite-time convergence for time-varying matrix pseudoinversion

In this paper, a special class of recurrent neural network, termed Zhang neural network (ZNN), is investigated for the online solution of the time-varying matrix pseudoinverse. Meanwhile, a novel activation function, named Li activation function, is employed. Then, based on two basic Zhang functions (ZFs) and the intrinsically nonlinear method of ZNN design, two finite-time convergent ZNN models (termed ZNN-1 model and ZNN-2 model) are first proposed and investigated for time-varying matrix pseudoinversion. Such two ZNN models can be accelerated to finite-time convergence to the time-varying theoretical pseudoinverse. The upper bound of the convergence time is also derived analytically via Lyapunov theory. By exploiting the other three simplified ZFs and the extended nonlinearization method, three simplified finite-time convergent ZNN models (termed ZNN-3 model, ZNN-4 model and ZNN-5 model) are sequentially proposed. In addition, the link between the ZNN models and the Getz-Marsden (G-M) dynamic system is discovered and presented in this paper. Computer-simulation results further substantiate the theoretical analysis and demonstrate the effectiveness of ZNN models based on different ZFs for the time-varying matrix pseudoinverse.

[1]  C. C. Lin,et al.  Pseudo-inverse Jacobian control with grey relational analysis for robot manipulators mounted on oscillatory bases ☆ , 2009 .

[2]  Ke Chen,et al.  Performance Analysis of Gradient Neural Network Exploited for Online Time-Varying Matrix Inversion , 2009, IEEE Transactions on Automatic Control.

[3]  Xiaojun Chen,et al.  Smoothing Neural Network for Constrained Non-Lipschitz Optimization With Applications , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Hubert Cardot,et al.  SOM time series clustering and prediction with recurrent neural networks , 2011, Neurocomputing.

[5]  Predrag S. Stanimirovic,et al.  Iterative Method for Computing Moore-penrose Inverse Based on Penrose Equations , 2022 .

[6]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[7]  Jun Wang,et al.  Recurrent Neural Networks for Computing Pseudoinverses of Rank-Deficient Matrices , 1997, SIAM J. Sci. Comput..

[8]  Dennis S. Bernstein,et al.  Finite-Time Stability of Continuous Autonomous Systems , 2000, SIAM J. Control. Optim..

[9]  Milan B. Tasic,et al.  Symbolic computation of weighted Moore-Penrose inverse using partitioning method , 2007, Appl. Math. Comput..

[10]  Jun Wang,et al.  A recurrent neural network for solving a class of generalized convex optimization problems , 2013, Neural Networks.

[11]  Yunong Zhang,et al.  Performance analysis of gradient neural network exploited for online time-varying quadratic minimization and equality-constrained quadratic programming , 2011, Neurocomputing.

[12]  Robert P. W. Duin,et al.  Bagging, Boosting and the Random Subspace Method for Linear Classifiers , 2002, Pattern Analysis & Applications.

[13]  Oussama Khatib,et al.  Springer Handbook of Robotics , 2007, Springer Handbooks.

[14]  Binghuang Cai,et al.  From Zhang Neural Network to Newton Iteration for Matrix Inversion , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  Gene H. Golub,et al.  Tikhonov Regularization and Total Least Squares , 1999, SIAM J. Matrix Anal. Appl..

[16]  Qingshan Liu,et al.  Finite-Time Convergent Recurrent Neural Network With a Hard-Limiting Activation Function for Constrained Optimization With Piecewise-Linear Objective Functions , 2011, IEEE Transactions on Neural Networks.

[17]  Yangming Li,et al.  A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application , 2013, Neural Networks.

[18]  David Sussillo,et al.  A recurrent neural network for closed-loop intracortical brain–machine interface decoders , 2012, Journal of neural engineering.

[19]  Tingzhu Huang,et al.  Method of elementary transformation to compute Moore-Penrose inverse , 2010, Appl. Math. Comput..

[20]  Jerrold E. Marsden,et al.  Dynamical methods for polar decomposition and inversion of matrices , 1997 .

[21]  Jein-Shan Chen,et al.  Recurrent neural networks for solving second-order cone programs , 2011, Neurocomputing.

[22]  Dongsheng Guo,et al.  Zhang neural network, Getz-Marsden dynamic system, and discrete-time algorithms for time-varying matrix inversion with application to robots' kinematic control , 2012, Neurocomputing.

[23]  Witold Pedrycz,et al.  Fuzzy prediction architecture using recurrent neural networks , 2009, Neurocomputing.

[24]  Zhigang Zeng,et al.  Analysis and design of associative memories based on recurrent neural network with discontinuous activation functions , 2012, Neurocomputing.

[25]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[26]  Ying Wang,et al.  Different ZFs Leading to Various ZNN Models Illustrated via Online Solution of Time-Varying Underdetermined Systems of Linear Equations with Robotic Application , 2013, ISNN.

[27]  Shuzhi Sam Ge,et al.  Design and analysis of a general recurrent neural network model for time-varying matrix inversion , 2005, IEEE Transactions on Neural Networks.

[28]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[29]  Pierre Courrieu,et al.  Fast Computation of Moore-Penrose Inverse Matrices , 2008, ArXiv.

[30]  Yunong Zhang,et al.  Different Zhang functions leading to different Zhang-dynamics models illustrated via time-varying reciprocal solving , 2012 .

[31]  Shuzhi Sam Ge,et al.  Face recognition by applying wavelet subband representation and kernel associative memory , 2004, IEEE Transactions on Neural Networks.

[32]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[33]  Arogyaswami Paulraj,et al.  A subspace approach to blind space-time signal processing for wireless communication systems , 1997, IEEE Trans. Signal Process..

[34]  Feng Ding,et al.  Gradient Based Iterative Algorithms for Solving a Class of Matrix Equations , 2005, IEEE Trans. Autom. Control..

[35]  Li-Hong Juang,et al.  Image noise reduction using Wiener filtering with pseudo-inverse , 2010 .

[36]  A. A. Maciejewski,et al.  Repeatable generalized inverse control strategies for kinematically redundant manipulators , 1993, IEEE Trans. Autom. Control..