Neural Network for Computing GSVD and RSVD

Abstract This paper presents the neural dynamical network to compute the generalized and restricted singular value decompositions (GSVD/RSVD) in the regularization methods for ill-posed problems. The neural network model is defined by ordinary differential equations which can be solved by many state-of-the-art techniques. The main purpose of the paper is to develop two neural network models for finding approximations of the GSVD and the RSVD. The globally asymptotic stability analyses are provided and numerical experiments illustrate our theory and methods. For small scale problems, the estimation can be as accurate as O ( 10 - 15 ) . For ill-posed problems, the truncation regularization method implementing the GSVD/RSVD algorithms also produces accurate results.

[1]  Timo Betcke.  The Generalized Singular Value Decomposition and the Method of Particular Solutions , 2008, SIAM J. Sci. Comput..

[2]  Ji-Guang Sun,et al.  Condition Number and Backward Error for the Generalized Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[3]  Haesun Park,et al.  Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[4]  Bart De Moor,et al.  On the Computation of the Restricted Singular Value Decomposition via the Cosine-Sine Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[5]  K. Chu,et al.  Singular value and generalized singular value decompositions and the solution of linear matrix equations , 1987 .

[6]  H. Zha.  The restricted singular value decomposition of matrix triplets , 1991 .

[7]  H. Zha,et al.  Regularization and the general Gauss-Markov linear model , 1990 .

[8]  Charles Van Loan.  Computing the CS and the generalized singular value decompositions , 1985 .

[9]  Zlatko Drmac,et al.  New Accurate Algorithms for Singular Value Decomposition of Matrix Triplets , 2000, SIAM J. Matrix Anal. Appl..

[10]  Andrzej Cichocki,et al.  Neural network for singular value decomposition , 1992 .

[11]  Weichung Yeih,et al.  APPLICATIONS OF THE GENERALIZED SINGULAR-VALUE DECOMPOSITION METHOD ON THE EIGENPROBLEM USING THE INCOMPLETE BOUNDARY ELEMENT FORMULATION , 2000 .

[12]  Xuezhong Wang,et al.  Tensor neural network models for tensor singular value decompositions , 2020, Computational Optimization and Applications.

[13]  Yimin Wei,et al.  Tikhonov Regularization and Randomized GSVD , 2016, SIAM J. Matrix Anal. Appl..

[14]  Andrzej Cichocki,et al.  Neural networks for computing best rank-one approximations of tensors and its applications , 2017, Neurocomputing.

[15]  P. Hansen.  Regularization,GSVD and truncatedGSVD , 1989 .

[16]  C. Paige.  Computing the generalized singular value decomposition , 1986 .

[17]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[18]  Z. Drmač.  A Tangent Algorithm for Computing the Generalized Singular Value Decomposition , 1998 .

[19]  O. Alter,et al.  Platform-Independent Genome-Wide Pattern of DNA Copy-Number Alterations Predicting Astrocytoma Survival and Response to Treatment Revealed by the GSVD Formulated as a Comparative Spectral Decomposition , 2016, PloS one.

[20]  O. Alter,et al.  GSVD- and tensor GSVD-uncovered patterns of DNA copy-number alterations predict adenocarcinomas survival in general and in response to platinum , 2019, APL bioengineering.

[21]  Gene H. Golub,et al.  On a Variational Formulation of the Generalized Singular Value Decomposition , 1997 .

[22]  Saša Singer,et al.  Blocking and parallelization of the Hari-Zimmermann variant of the Falk-Langemeyer algorithm for the generalized SVD , 2015, Parallel Comput..

[23]  King-Wah Eric Chu.  Exclusion theorems and the perturbation analysis of the generalized eigenvalue problem , 1987 .

[24]  James Demmel,et al.  Computing the Generalized Singular Value Decomposition , 1993, SIAM J. Sci. Comput..

[25]  Zheng Bao,et al.  A cross-associative neural network for SVD of non-squared data matrix in signal processing , 2001, IEEE Trans. Neural Networks.

[26]  Hongyuan Zha,et al.  A Numerical Algorithm for Computing the Restricted Singular Value Decomposition of Matrix Triplets. , 1992 .

[27]  Jack J. Dongarra,et al.  The Singular Value Decomposition: Anatomy of Optimizing an Algorithm for Extreme Scale , 2018, SIAM Rev..

[28]  H. Zha,et al.  A tree of generalizations of the ordinary singular value decomposition , 1991 .

[29]  Christopher C. Paige,et al.  The general linear model and the generalized singular value decomposition , 1985 .

[30]  G. Golub,et al.  The restricted singular value decomposition: properties and applications , 1991 .

[31]  Yimin Wei,et al.  Recurrent neural network for computation of generalized eigenvalue problem with real diagonalizable matrix pair and its applications , 2016, Neurocomputing.

[32]  Zhaojun Bai,et al.  A New Preprocessing Algorithm for the Computation of the Generalized Singular Value Decomposition , 1993, SIAM J. Sci. Comput..

[33]  Haesun Park,et al.  Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition , 2003, SIAM J. Matrix Anal. Appl..

[34]  Per Christian Hansen,et al.  Regularization Tools version 4.0 for Matlab 7.3 , 2007, Numerical Algorithms.

[35]  M. Saunders,et al.  Towards a Generalized Singular Value Decomposition , 1981 .

[36]  Yimin Wei,et al.  Ill‐conditioning of the truncated singular value decomposition, Tikhonov regularization and their applications to numerical partial differential equations , 2011, Numer. Linear Algebra Appl..