TR-2012006: Randomized Matrix Computations II

It is well and long known that random matrices tend to be well conditioned, and we employ them to advance some fundamental matrix computations. We begin with specifying and proving preconditioning properties of randomized additive preprocessing and randomized augmentation and then apply these results to outline new promising randomized algorithms for such fundamental matrix computations as preconditioning of an ill conditioned matrix that has a small numerical nullity or rank, its 2-by-2 block triangulation, numerical stabilization of Gaussian elimination with no pivoting, and approximation of a matrix by low-rank matrices and by structured matrices. According to both our formal study and numerical tests some of our algorithms significantly accelerate the known ones and improve their output accuracy. This should motivate further effort for advancing the presented approach. Besides the novel techniques of randomized preprocessing and the proof of their preconditioning power, our technical advances of potential independent interest include estimates for the condition numbers of random Toeplitz matrices and extension of the Sherman-Morrison-Woodbury formula. 2000 Math. Subject Classification: 15A52, 15A12, 15A06, 65F22, 65F05, 65F10

[1]  Victor Y. Pan,et al.  Field extension and trilinear aggregating, uniting and canceling for the acceleration of matrix multiplications , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[2]  V. Pan.  New combinations of methods for the acceleration of matrix multiplications , 1981 .

[3]  Shang-Hua Teng,et al.  Smoothed Analysis of the Condition Numbers and Growth Factors of Matrices , 2003, SIAM J. Matrix Anal. Appl..

[4]  Victor Y. Pan,et al.  New homotopic/factorization and symmetrization techniques for Newton's and Newton/structured iteration , 2007, Comput. Math. Appl..

[5]  V.Y. Pan,et al.  Concurrent Iterative Algorithm for Toeplitz-like Linear Systems , 1993, IEEE Trans. Parallel Distributed Syst..

[6]  Siegfried M. Rump,et al.  Accurate Sum and Dot Product , 2005, SIAM J. Sci. Comput..

[7]  Dario Bini,et al.  Tensor Rank and Border Rank of Band Toeplitz Matrices , 1987, SIAM J. Comput..

[8]  Julian D. Laderman,et al.  On practical algorithms for accelerated matrix multiplication , 1992 .

[9]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[10]  Marc Van Barel,et al.  A stabilized superfast solver for indefinite Hankel systems , 1998 .

[11]  I. Gohberg,et al.  On a new class of structured matrices , 1999 .

[12]  Thomas Kailath,et al.  Fast Gaussian elimination with partial pivoting for matrices with displacement structure , 1995 .

[13]  O. Axelsson.  Iterative solution methods , 1995 .

[14]  V. Pan.  On computations with dense structured matrices , 1990 .

[15]  V. Pan,et al.  Additive preconditioning for matrix computations , 2010 .

[16]  Igor E. Kaporin,et al.  The aggregation and cancellation techniques as a practical tool for faster matrix multiplication , 2004, Theor. Comput. Sci..

[17]  Virginia Vassilevska Williams,et al.  Multiplying matrices faster than coppersmith-winograd , 2012, STOC '12.

[18]  A. Edelman.  Eigenvalues and condition numbers of random matrices , 1988 .

[19]  Dario Bini,et al.  Border Rank of a pxqx2 Tensor and the Optimal Approximation od a Pair of Bilinear Forms , 1980, ICALP.

[20]  Richard J. Lipton,et al.  A Probabilistic Remark on Algebraic Program Testing , 1978, Inf. Process. Lett..

[21]  C. Pan.  On the existence and computation of rank-revealing LU factorizations , 2000 .

[22]  V. Pan.  How can we speed up matrix multiplication , 1984 .

[23]  Raf Vandebril,et al.  Matrix Computations and Semiseparable Matrices: Eigenvalue and Singular Value Methods , 2008 .

[24]  Alan Edelman,et al.  Tails of Condition Number Distributions , 2005, SIAM J. Matrix Anal. Appl..

[25]  Dario Andrea Bini.  Tensor and border rank of certain classes of matrices and the fast evaluation of determinant inverse matrix and eigenvalues , 1985 .

[26]  Albrecht Böttcher,et al.  Spectral properties of banded Toeplitz matrices , 1987 .

[27]  Alan J. Laub,et al.  Small-Sample Statistical Condition Estimates for General Matrix Functions , 1994, SIAM J. Sci. Comput..

[28]  A. J. Stothers.  On the complexity of matrix multiplication , 2010 .

[29]  Victor Y. Pan,et al.  Schur aggregation for linear systems and determinants , 2008, Theor. Comput. Sci..

[30]  C. Beltrán.  Estimates on the condition number of random rank-deficient matrices , 2011 .

[31]  Victor Y. Pan.  Decreasing the displacement rank of a matrix , 1993 .

[32]  Zizhong Chen,et al.  Condition Numbers of Gaussian Random Matrices , 2005, SIAM J. Matrix Anal. Appl..

[33]  Victor Y. Pan,et al.  Nearly optimal solution of rational linear systems of equations with symbolic lifting and numerical initialization , 2011, Comput. Math. Appl..

[34]  Peter Kravanja,et al.  Algorithms for solving rational interpolation problems related to fast and superfast solvers for Toeplitz systems , 1999, Optics & Photonics.

[35]  Georg Heinig,et al.  A Stabilized Superfast Solver for Nonsymmetric Toeplitz Systems , 2001, SIAM J. Matrix Anal. Appl..

[36]  M. Morf,et al.  Displacement ranks of matrices and linear equations , 1979 .

[37]  V. Pan,et al.  Randomized Preprocessing of Homogeneous Linear Systems of Equations , 2010 .

[38]  V. Pan,et al.  Methods of aggregation , 1980 .

[39]  J. Demmel.  The Probability That a Numerical, Analysis Problem Is Difficult , 2013 .

[40]  Victor Y. Pan,et al.  Homotopic residual correction processes , 2006, Math. Comput..

[41]  J. Dixon.  Estimating Extremal Eigenvalues and Condition Numbers of Matrices , 1983 .

[42]  Beatrice Meini,et al.  Approximate displacement rank and applications , 2001 .

[43]  V. Pan,et al.  Polynomial and Matrix Computations , 1994, Progress in Theoretical Computer Science.

[44]  Xinmao Wang,et al.  Effect of small rank modification on the condition number of a matrix , 2007, Comput. Math. Appl..

[45]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[46]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[47]  Jacob T. Schwartz,et al.  Fast Probabilistic Algorithms for Verification of Polynomial Identities , 1980, J. ACM.