Orthogonalization Via Deflation: A Minimum Norm Approach for Low-Rank Approximations of a Matrix

In this paper we introduce a new orthogonalization method. Given a real $m \times n$ matrix $A$, the new method constructs an SVD-type decomposition of the form $A = \hat U\hat\Sigma\hat V^T$. The columns of $\hat U$ and $\hat V$ are orthonormal, or nearly orthonormal, while $\hat\Sigma$ is a diagonal matrix whose diagonal entries approximate the singular values of $A$. The method has three versions: a “left-side" orthogonalization scheme in which the columns of $\hat U$ constitute an orthonormal basis of Range$(A)$, a “right-side" orthogonalization scheme in which the columns of $\hat V$ constitute an orthonormal basis of Range$(A^T)$, and a third version in which both $\hat U$ and $\hat V$ have orthonormal columns, but the decomposition is not exact. The new decompositions may substitute the SVD in many applications.

[1]  G. Stewart,et al.  On the Convergence of a New Rayleigh Quotient Method with Applicationsto Large Eigenproblems , 1998 .

[2]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[3]  Trevor Hastie,et al.  Imputing Missing Data for Gene Expression Arrays , 2001 .

[4]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[5]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[6]  B. Parlett.  The Rayleigh Quotient Iteration and Some Generalizations for Nonnormal Matrices , 1974 .

[7]  Achiya Dax,et al.  Computing projections via Householder transformations and Gram–Schmidt orthogonalizations , 2004, Numer. Linear Algebra Appl..

[8]  Shmuel Friedland,et al.  A simultaneous reconstruction of missing data in DNA microarrays , 2006 .

[9]  Gene H. Golub,et al.  Matrix computations , 1983 .

[10]  Achiya Dax,et al.  A modified Gram–Schmidt algorithm with iterative orthogonalization and column pivoting , 2000 .

[11]  Amy Nicole Langville,et al.  A Survey of Eigenvector Methods for Web Information Retrieval , 2005, SIAM Rev..

[12]  David Botstein,et al.  Processing and modeling genome-wide expression data using singular value decomposition , 2001, SPIE BiOS.

[13]  Achiya Dax,et al.  A minimum norm approach for low-rank approximations of a matrix , 2010, J. Comput. Appl. Math..

[14]  Russ B. Altman,et al.  Missing value estimation methods for DNA microarrays , 2001, Bioinform..

[15]  Achiya Dax,et al.  Line search acceleration of iterative methods , 1990 .

[16]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[17]  Gene H. Golub,et al.  A Rank-One Reduction Formula and Its Applications to Matrix Factorizations , 1995, SIAM Rev..

[18]  E. Egerváry.  On rank-diminishing operations and their applications to the solution of linear equations , 1960 .

[19]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[20]  H. Hotelling.  Some New Methods in Matrix Calculation , 1943 .

[21]  A. Ostrowski.  On the convergence of the Rayleigh quotient iteration for the computation of the characteristic roots and vectors. I , 1957 .

[22]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[23]  D. Botstein,et al.  Singular value decomposition for genome-wide expression data processing and modeling. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[24]  G. Stewart.  Introduction to matrix computations , 1973 .

[25]  H. W. Turnbull,et al.  Lectures on Matrices , 1934 .

[26]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[27]  C. D. Meyer,et al.  The Use of the Linear Algebra by Web Search Engines , 2004 .

[28]  G. W. Stewart,et al.  Matrix algorithms , 1998 .