A New RNN Model With a Modified Nonlinear Activation Function Applied to Complex-Valued Linear Equations

In this paper, an improved Zhang neural network (IZNN) is proposed by using a kind of novel nonlinear activation function to solve the complex-valued systems of linear equation. Compared with the previous ZNN models, the convergence rate of the IZNN model has been accelerated. To do so, a kind of novel nonlinear activation function is first proposed to establish the novel recurrent neural network. Then, the corresponding maximum convergent time is given according to the randomly generated initial error vector, and the theoretical proof is described in detail in this paper. Finally, the experiment results illustrate that the new recurrent neural network using the proposed activation function has higher convergence rate than the previous neural networks using the linear activation function or the tunable activation function.

[1]  Zili Zhang,et al.  Design, verification and robotic application of a novel recurrent neural network for computing dynamic Sylvester equation , 2018, Neural Networks.

[2]  Lin Xiao,et al.  A convergence-accelerated Zhang neural network and its solution application to Lyapunov equation , 2016, Neurocomputing.

[3]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[4]  Yan-Wu Wang,et al.  Solving time-varying quadratic programs based on finite-time Zhang neural networks and their application to robot tracking , 2014, Neural Computing and Applications.

[5]  Ke Chen,et al.  Global exponential convergence and stability of gradient-based neural network for online matrix inversion , 2009, Appl. Math. Comput..

[6]  Davod Khojasteh Salkuyeh,et al.  Two-parameter TSCSP method for solving complex symmetric system of linear equations , 2017, Calcolo.

[7]  Shuai Li,et al.  Nonlinear recurrent neural networks for finite-time solution of general time-varying linear matrix equations , 2018, Neural Networks.

[8]  Lin Xiao,et al.  A nonlinearly activated neural dynamics and its finite-time solution to time-varying nonlinear equation , 2016, Neurocomputing.

[9]  Sundaram Suresh,et al.  A complex-valued neuro-fuzzy inference system and its learning mechanism , 2014, Neurocomputing.

[10]  Jitao Sun,et al.  Stability and Exponential Stability of Complex-valued Discrete Linear Systems with Delay , 2018 .

[11]  Mykel J. Kochenderfer,et al.  Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior , 2017, IEEE Transactions on Intelligent Transportation Systems.

[12]  Jun Wang,et al.  Global stability of a class of discrete-time recurrent neural networks , 2002 .

[13]  Zhenjiang Zhao,et al.  Finite-time stability analysis of fractional-order complex-valued memristor-based neural networks with both leakage and time-varying delays , 2017, Neurocomputing.

[14]  Xi Yang,et al.  A Fully Complex-Valued Neural Network for Rapid Solution of Complex-Valued Systems of Linear Equations , 2015, ISNN.

[15]  Davod Khojasteh Salkuyeh,et al.  A new iterative method for solving a class of complex symmetric system of linear equations , 2016, Numerical Algorithms.

[16]  Dongsheng Guo,et al.  Novel Discrete-Time Zhang Neural Network for Time-Varying Matrix Inversion , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Long Jin,et al.  Discrete-Time Zhang Neural Network for Online Time-Varying Nonlinear Optimization With Application to Manipulator Motion Generation , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Shuai Li,et al.  Design and Analysis of FTZNN Applied to the Real-Time Solution of a Nonstationary Lyapunov Equation and Tracking Control of a Wheeled Mobile Manipulator , 2018, IEEE Transactions on Industrial Informatics.

[19]  R. Venkatesh Babu,et al.  Human action recognition using a fast learning fully complex-valued classifier , 2012, Neurocomputing.

[20]  Lin Xiao,et al.  A new design formula exploited for accelerating Zhang neural network and its application to time-varying matrix inversion , 2016, Theor. Comput. Sci..

[21]  Q. Lin,et al.  A Non-Krylov subspace Method for Solving Large and Sparse Linear System of Equations , 2015, 1511.08042.

[22]  Zhijun Zhang,et al.  Hybrid-Level Joint-Drift-Free Scheme of Redundant Robot Manipulators Synthesized by a Varying-Parameter Recurrent Neural Network , 2018, IEEE Access.

[23]  Yimin Wei,et al.  Two finite-time convergent Zhang neural network models for time-varying complex matrix Drazin inverse , 2017 .

[24]  Yunong Zhang,et al.  Zeroing Dynamics, Gradient Dynamics, and Newton Iterations , 2015 .

[25]  Jie Jin,et al.  A finite-time convergent dynamic system for solving online simultaneous linear equations , 2017, Int. J. Comput. Math..

[26]  Shuai Li,et al.  Nonlinearly Activated Neural Network for Solving Time-Varying Complex Sylvester Equation , 2014, IEEE Transactions on Cybernetics.

[27]  Lin Xiao,et al.  A finite-time convergent neural dynamics for online solution of time-varying linear complex matrix equation , 2015, Neurocomputing.

[28]  D. K. Gupta,et al.  A Family of Iterative Methods with Accelerated Convergence for Restricted Linear System of Equations , 2017 .

[29]  Shuai Li,et al.  Integration-Enhanced Zhang Neural Network for Real-Time-Varying Matrix Inversion in the Presence of Various Kinds of Noises , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Jian Yang,et al.  A new recurrent neural network with noise-tolerance and finite-time convergence for dynamic quadratic minimization , 2018, Neurocomputing.

[31]  Zhi Yang,et al.  Recurrent Neural Networks for Solving Real-Time Linear System of Equations , 2017 .

[32]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[33]  Najim Dehak,et al.  Age Estimation in Short Speech Utterances Based on LSTM Recurrent Neural Networks , 2018, IEEE Access.

[34]  Lin Xiao.  A finite-time recurrent neural network for solving online time-varying Sylvester matrix equation based on a new evolution formula , 2017 .

[35]  Huaguang Zhang,et al.  A Comprehensive Review of Stability Analysis of Continuous-Time Recurrent Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.