A New Varying-Parameter Recurrent Neural-Network for Online Solution of Time-Varying Sylvester Equation

Solving Sylvester equation is a common algebraic problem in mathematics and control theory. Different from the traditional fixed-parameter recurrent neural networks, such as gradient-based recurrent neural networks or Zhang neural networks, a novel varying-parameter recurrent neural network, [called varying-parameter convergent-differential neural network (VP-CDNN)] is proposed in this paper for obtaining the online solution to the time-varying Sylvester equation. With time passing by, this kind of new varying-parameter neural network can achieve super-exponential performance. Computer simulation comparisons between the fixed-parameter neural networks and the proposed VP-CDNN via using different kinds of activation functions demonstrate that the proposed VP-CDNN has better convergence and robustness properties.

[1]  Shuai Li,et al.  Dynamic Neural Networks for Kinematic Redundancy Resolution of Parallel Stewart Platforms , 2016, IEEE Transactions on Cybernetics.

[2]  Charlie Chung-Ping Chen,et al.  Efficient three-dimensional resist profile-driven source mask optimization optical proximity correction based on Abbe-principal component analysis and Sylvester equation , 2014 .

[3]  Yunong Zhang,et al.  MATLAB Simulink modeling and simulation of Zhang neural networks for online time-varying sylvester equation solving , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[4]  Yunong Zhang,et al.  Revisit the Analog Computer and Gradient-Based Neural System for Matrix Inversion , 2005, Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005..

[5]  Martin Stoll,et al.  Fast tensor product solvers for optimization problems with fractional differential equations as constraints , 2016, Appl. Math. Comput..

[6]  Shuai Li,et al.  Modified ZNN for Time-Varying Quadratic Programming With Inherent Tolerance to Noises and Its Application to Kinematic Redundancy Resolution of Robot Manipulators , 2016, IEEE Transactions on Industrial Electronics.

[7]  Ivan V. Oseledets,et al.  From Low-Rank Approximation to a Rational Krylov Subspace Method for the Lyapunov Equation , 2015, SIAM J. Matrix Anal. Appl..

[8]  Yunong Zhang,et al.  Robustness analysis of the Zhang neural network for online time-varying quadratic optimization , 2010 .

[9]  Chunyan Miao,et al.  Comparing the learning effectiveness of BP, ELM, I-ELM, and SVM for corporate credit ratings , 2014, Neurocomputing.

[10]  Guoliang Chen,et al.  On modified HSS iteration methods for continuous Sylvester equations , 2015, Appl. Math. Comput..

[11]  Mohammad Khorsand Zak,et al.  Nested splitting CG-like iterative method for solving the continuous Sylvester equation and preconditioning , 2014, Adv. Comput. Math..

[12]  Lei Wang,et al.  Manifold Regularized Multi-view Subspace Clustering for image representation , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[13]  Jun Wang,et al.  A recurrent neural network for solving Sylvester equation with time-varying coefficients , 2002, IEEE Trans. Neural Networks.

[14]  Jinde Cao,et al.  Synchronization of discrete-time neural networks with delays and Markov jump topologies based on tracker information , 2017, Neural Networks.

[15]  Shiheng Wang,et al.  Gradient-based Neural Network for Online Solution of Lyapunov Matrix Equation with Li Activation Function , 2015, ICIT 2015.

[16]  Shuzhi Sam Ge,et al.  Adaptive Neural Network Control of Robotic Manipulators , 1999, World Scientific Series in Robotics and Intelligent Systems.

[17]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[18]  Sitian Qin,et al.  A One-Layer Recurrent Neural Network for Pseudoconvex Optimization Problems With Equality and Inequality Constraints , 2017, IEEE Transactions on Cybernetics.

[19]  Bertrand Tondu.  Closed-form redundancy solving of serial chain robots with a weak generalized inverse approach , 2015 .

[20]  Shuai Li,et al.  Inverse-Free Extreme Learning Machine With Optimal Information Updating , 2016, IEEE Transactions on Cybernetics.

[21]  Pei-Chun Lin,et al.  High speed and flexible PEB 3D diffusion simulation based on Sylvester equation , 2013, Advanced Lithography.

[22]  Jian Li,et al.  Enhanced discrete-time Zhang neural network for time-variant matrix inversion in the presence of bias noises , 2016, Neurocomputing.

[23]  Thomas Wolf,et al.  ADI iteration for Lyapunov equations: a tangential approach and adaptive shift selection , 2013, ArXiv.

[24]  Zhang Yi,et al.  Trajectory Predictor by Using Recurrent Neural Networks in Visual Tracking , 2017, IEEE Transactions on Cybernetics.

[25]  Shuzhi Sam Ge,et al.  Design and analysis of a general recurrent neural network model for time-varying matrix inversion , 2005, IEEE Transactions on Neural Networks.

[26]  Mohammad Khorsand Zak,et al.  An iterative method for solving the continuous sylvester equation by emphasizing on the skew-hermitian parts of the coefficient matrices , 2017, Int. J. Comput. Math..

[27]  Simon J. Godsill,et al.  R-FUSE: Robust Fast Fusion of Multiband Images Based on Solving a Sylvester Equation , 2016, IEEE Signal Processing Letters.

[28]  Peter Benner,et al.  Computing real low-rank solutions of Sylvester equations by the factored ADI method , 2014, Comput. Math. Appl..

[29]  Karl Meerbergen,et al.  A Sylvester–Arnoldi type method for the generalized eigenvalue problem with two‐by‐two operator determinants , 2014, Numer. Linear Algebra Appl..

[30]  G. Golub,et al.  A Hessenberg-Schur method for the problem AX + XB= C , 1979 .

[31]  Shuai Li,et al.  Nonlinearly Activated Neural Network for Solving Time-Varying Complex Sylvester Equation , 2014, IEEE Transactions on Cybernetics.

[32]  Mohammed Ismail,et al.  Analog VLSI Implementation of Neural Systems , 2011, The Kluwer International Series in Engineering and Computer Science.

[33]  Lin Xiao,et al.  A convergence-accelerated Zhang neural network and its solution application to Lyapunov equation , 2016, Neurocomputing.

[34]  Qingqing Zheng,et al.  On normal and skew-Hermitian splitting iteration methods for large sparse continuous Sylvester equations , 2014, J. Comput. Appl. Math..

[35]  Nadia Magnenat-Thalmann,et al.  Human-Like Behavior Generation Based on Head-Arms Model for Robot Tracking External Targets and Body Parts , 2015, IEEE Transactions on Cybernetics.

[36]  Yuehua Huang,et al.  Finite-Time Stability and Its Application for Solving Time-Varying Sylvester Equation by Recurrent Neural Network , 2014, Neural Processing Letters.

[37]  Yunong Zhang,et al.  Link Between and Comparison and Combination of Zhang Neural Network and Quasi-Newton BFGS Method for Time-Varying Quadratic Minimization , 2013, IEEE Transactions on Cybernetics.

[38]  Stephen D. Shank,et al.  Krylov Subspace Methods for Large-Scale Constrained Sylvester Equations , 2013, SIAM J. Matrix Anal. Appl..

[39]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[41]  Thomas Wolf,et al.  The ADI iteration for Lyapunov equations implicitly performs pseudo-optimal model order reduction , 2013, Int. J. Control.

[42]  Shuyuan Yang,et al.  Local graph regularized coding for salient object detection , 2016 .

[43]  Muhammad H. Rashid,et al.  Power Electronics Handbook: Devices, Circuits and Applications , 2010 .

[44]  Ying Wang,et al.  On exponential convergence of nonlinear gradient dynamics system with application to square root finding , 2015 .

[45]  David L. Kleinman,et al.  Extensions to the Bartels-Stewart algorithm for linear matrix equations , 1978 .

[46]  Changfeng Ma,et al.  The scaling conjugate gradient iterative method for two types of linear matrix equations , 2015, Comput. Math. Appl..

[47]  Hamid Reza Shaker,et al.  Control configuration selection for bilinear systems via generalised Hankel interaction index array , 2015, Int. J. Control.

[48]  Long Jin,et al.  Taylor $O(h^{3})$ Discretization of ZNN Models for Dynamic Equality-Constrained Quadratic Programming With Application to Manipulators , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Na Huang,et al.  Modified conjugate gradient method for obtaining the minimum-norm solution of the generalized coupled Sylvester-conjugate matrix equations , 2016 .

[50]  Lei Shen,et al.  Initial Position Estimation in SRM Using Bootstrap Circuit Without Predefined Inductance Parameters , 2011, IEEE Transactions on Power Electronics.

[51]  Lin Xiao.  A finite-time convergent Zhang neural network and its application to real-time matrix square root finding , 2017, Neural Computing and Applications.

[52]  Zhijun Zhang,et al.  Simulink comparison of varying-parameter convergent-differential neural-network and gradient neural network for solving online linear time-varying equations , 2016, 2016 12th World Congress on Intelligent Control and Automation (WCICA).