Generating Extreme-Scale Matrices With Specified Singular Values or Condition Number

A widely used form of test matrix is the randsvd matrix constructed as the product A = USV*, where U and V are random orthogonal or unitary matrices from the Haar distribution and S is a diagonal matrix of singular values. Such matrices are random but have a specified singular value distribution. The cost of forming an m-by-n randsvd matrix is m³ + n³ flops, which is prohibitively expensive at extreme scale; moreover, the randsvd construction requires a significant amount of communication, making it unsuitable for distributed memory environments. By dropping the requirement that U and V be Haar distributed and that both be random, we derive new algorithms for forming A that have cost linear in the number of matrix elements and require a low amount of communication and synchronization. We specialize these algorithms to generating matrices with specified 2-norm condition number. Numerical experiments show that the algorithms have excellent efficiency and scalability.

[1]  Helmert.  Die Genauigkeit der Formel von Peters zur Berechnung des wahrscheinlichen Beobachtungsfehlers directer Beobachtungen gleicher Genauigkeit , 1876 .

[2]  J. O. Irwin.  On the Distribution of a Weighted Estimate of Variance and on Analysis of Variance in Certain Cases of Unequal Weighting , 1942 .

[3]  James Hardy Wilkinson,et al.  Error Analysis of Direct Methods of Matrix Inversion , 1961, JACM.

[4]  F. L. Bauer.  Optimally scaled matrices , 1963 .

[5]  H. O. Lancaster.  The Helmert Matrices , 1965 .

[6]  G. Birkhoff,et al.  Isotropic distributions of test matrices , 1979 .

[7]  Robert D. Skeel,et al.  Scaling for Numerical Stability in Gaussian Elimination , 1979, JACM.

[8]  G. Stewart.  The Efficient Generation of Random Orthogonal Matrices with an Application to Condition Estimators , 1980 .

[9]  Dario Bini,et al.  SPECTRAL AND COMPUTATIONAL PROPERTIES OF BAND SYMMETRIC TOEPLITZ MATRICES , 1983 .

[10]  N. Higham,et al.  Large growth factors in Gaussian elimination with pivoting , 1989 .

[11]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[12]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[13]  Nicholas J. Higham,et al.  Algorithm 694: a collection of test matrices in MATLAB , 1991, TOMS.

[14]  N. Higham.  The Test Matrix Toolbox for MATLAB , 1993 .

[15]  G. Strang.  Wavelet transforms versus Fourier transforms , 1993, math/9304214.

[16]  Paola Favati,et al.  On a Matrix Algebra Related to the Discrete Hartley Transform , 1993, SIAM J. Matrix Anal. Appl..

[17]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[18]  Juan Antonio Cuesta-Albertos,et al.  Some remarks on the condition number of a real random square matrix , 2003, J. Complex..

[19]  F. Mezzadri.  How to generate random matrices from the classical compact groups , 2006, math-ph/0609050.

[20]  Jianqin Zhou,et al.  On discrete cosine transform , 2011, ArXiv.

[21]  Nicholas J. Higham,et al.  The Test Matrix Toolbox for Matlab (Version 3.0) , 2013 .

[22]  Jack J. Dongarra,et al.  Accelerating Numerical Dense Linear Algebra Calculations with GPUs , 2014, Numerical Computations with GPUs.

[23]  Nicholas J. Higham,et al.  Matrix Depot: an extensible test matrix collection for Julia , 2016, PeerJ Comput. Sci..

[24]  Nicholas J. Higham,et al.  Harnessing GPU Tensor Cores for Fast FP16 Arithmetic to Speed up Mixed-Precision Iterative Refinement Solvers , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.

[25]  Nicholas J. Higham,et al.  Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions , 2018, SIAM J. Sci. Comput..

[26]  Nicholas J. Higham,et al.  Random Matrices Generating Large Growth in LU Factorization with Pivoting , 2021, SIAM J. Matrix Anal. Appl..