Adaptive algorithms for computing the principal Takagi vector of a complex symmetric matrix

Abstract In this paper, we present a unified framework for deriving and analyzing adaptive algorithms for computing the principal Takagi vector of a complex symmetric matrix. Eight systems of complex-valued ordinary differential equations (complex-valued ODEs) are derived and their convergence behavior is analyzed. We prove that the solutions of the complex-valued ODEs are asymptotically stable. The systems can be implemented on neural networks. Finally, we show experimental results to support our analyses.

[1]  Mark D. Plumbley.  Lyapunov functions for convergence of principal component algorithms , 1995, Neural Networks.

[2]  Y. Chauvin,et al.  Principal component analysis by gradient descent on a constrained linear Hebbian cell , 1989, International 1989 Joint Conference on Neural Networks.

[3]  Ken Kreutz-Delgado,et al.  The Complex Gradient Operator and the CR-Calculus ECE275A - Lecture Supplement - Fall 2005 , 2009, 0906.4835.

[4]  Hongyuan Zha,et al.  Low-Rank Matrix Approximation Using the Lanczos Bidiagonalization Process with Applications , 1999, SIAM J. Sci. Comput..

[5]  Fa-Long Luo,et al.  A Minor Component Analysis Algorithm , 1997, Neural Networks.

[6]  Léon Autonne,et al.  Sur les matrices hypohermitiennes et sur les matrices unitaires , 1915 .

[7]  Yimin Wei,et al.  Iterative algorithms for computing US- and U-eigenpairs of complex tensors , 2017, J. Comput. Appl. Math..

[8]  W. Gragg,et al.  Singular value decompositions of complex symmetric matrices , 1988 .

[9]  Zengyun Wang,et al.  Dynamical Behavior of Complex-Valued Hopfield Neural Networks with Discontinuous Activation Functions , 2017, Neural Processing Letters.

[10]  Teiji Takagi,et al.  On an Algebraic Problem Reluted to an Analytic Theorem of Carathéodory and Fejér and on an Allied Theorem of Landau , 1924 .

[11]  Wei Xu,et al.  A twisted factorization method for symmetric SVD of a complex symmetric tridiagonal matrix , 2009, Numer. Linear Algebra Appl..

[12]  Sabine Van Huffel,et al.  Enhanced resolution based on minimum variance estimation and exponential data modeling , 1993, Signal Process..

[13]  Michael J. Shapiro,et al.  Chemical Exchange in Diffusion NMR Experiments , 1998 .

[14]  Gene H. Golub,et al.  Matrix computations , 1983 .

[15]  Alex D. Bain,et al.  Chemical exchange in NMR , 2003 .

[16]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[17]  Zhongxiao Jia,et al.  A Refined Harmonic Lanczos Bidiagonalization Method and an Implicitly Restarted Algorithm for Computing the Smallest Singular Triplets of Large Matrices , 2009, SIAM J. Sci. Comput..

[18]  Zhongxiao Jia,et al.  An Implicitly Restarted Refined Bidiagonalization Lanczos Method for Computing a Partial Singular Value Decomposition , 2003, SIAM J. Matrix Anal. Appl..

[19]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[20]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[21]  M. Bohner,et al.  Global Stability of Complex-Valued Neural Networks on Time Scales , 2011 .

[22]  P. Bouboulis,et al.  Wirtinger's Calculus in general Hilbert Spaces , 2010, ArXiv.

[23]  Akira Hirose,et al.  Complex-Valued Neural Networks: Theories and Applications , 2003 .

[24]  B. A. D. H. Brandwood.  A complex gradient operator and its applica-tion in adaptive array theory , 1983 .

[25]  Glenn Schober,et al.  Univalent Functions - Selected Topics , 1975 .

[26]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[27]  Wei Xu,et al.  Block Lanczos tridiagonalization of complex symmetric matrices , 2005, SPIE Optics + Photonics.

[28]  Huamin Wang,et al.  Exponential Stability of Complex-Valued Memristive Recurrent Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Yimin Wei,et al.  Complex-valued neural networks for the Takagi vector of complex symmetric matrices , 2017, Neurocomputing.

[30]  Tohru Nitta.  Complex-valued Neural Networks: Utilizing High-dimensional Parameters , 2009 .

[31]  Yingbo Hua,et al.  Fast subspace tracking and neural network learning by a novel information criterion , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[32]  Jun Wang,et al.  A Complex-Valued Projection Neural Network for Constrained Optimization of Real Functions in Complex Variables , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Sabine Van Huffel,et al.  The MCA EXIN neuron for the minor component analysis , 2002, IEEE Trans. Neural Networks.

[34]  Igor N. Aizenberg,et al.  Complex-Valued Neural Networks with Multi-Valued Neurons , 2011, Studies in Computational Intelligence.

[35]  Wei Xing Zheng,et al.  A complex-valued neural dynamical optimization approach and its stability analysis , 2015, Neural Networks.

[36]  R. Remmert,et al.  Theory of Complex Functions , 1990 .

[37]  Wei Xu,et al.  A Divide-and-Conquer Method for the Takagi Factorization , 2008, SIAM J. Matrix Anal. Appl..

[38]  L. Trefethen.  Near-circularity of the error curve in complex Chebyshev approximation , 1981 .

[39]  A. Bos.  Complex gradient and Hessian , 1994 .

[40]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .