Searching for Rare Growth Factors Using Multicanonical Monte Carlo Methods

The growth factor of a matrix quantifies the amount of potential error growth possible when a linear system is solved using Gaussian elimination with row pivoting. While it is an easy matter [N. J. Higham and D. J. Higham, SIAM J. Matrix Anal. Appl., 10 (1989), pp. 155-164] to construct examples of $n\times n$ matrices having any growth factor up to the maximum of $2^{n-1}$, the weight of experience and analysis [N. J. Higham, Accuracy and Stability of Numerical Algorithms, SIAM, Philadelphia, 1996], [L. N. Trefethen and R. S. Schreiber, SIAM J. Matrix Anal. Appl., 11 (1990), pp. 335-360], [L. N. Trefethen and I. D. Bau, Numerical Linear Algebra, SIAM, Philadelphia, 1997] suggest that matrices with exponentially large growth factors are exceedingly rare. Here we show how to conduct numerical experiments on random matrices using a multicanonical Monte Carlo method to explore the tails of growth factor probability distributions. Our results suggest, for example, that the occurrence of an $8\times 8$ matrix with a growth factor of 40 is on the order of a once-in-the-age-of-the-universe event.

[1]  Bernd A. Berg,et al.  Introduction to Multicanonical Monte Carlo Simulations , 1999, cond-mat/9909236.

[2]  Persi Diaconis,et al.  Iterated Random Functions , 1999, SIAM Rev..

[3]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[4]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[5]  L. Foster.  Gaussian Elimination with Partial Pivoting Can Fail in Practice , 1994, SIAM J. Matrix Anal. Appl..

[6]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[7]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[8]  Gene H. Golub,et al.  Matrix computations , 1983 .

[9]  N. Higham,et al.  Large growth factors in Gaussian elimination with pivoting , 1989 .

[10]  S. Ulam,et al.  Adventures of a Mathematician , 2019, Mathematics: People · Problems · Results.

[11]  J. Propp,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996 .

[12]  Stephen P. Brooks,et al.  Markov chain Monte Carlo method and its application , 1998 .

[13]  L. Trefethen,et al.  Average-case stability of Gaussian elimination , 1990 .

[14]  Ronald Holzlöhner,et al.  Use of multicanonical Monte Carlo simulations to obtain accurate bit error rates in optical communications systems. , 2003, Optics letters.

[15]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[16]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[17]  Gareth O. Roberts,et al.  Convergence assessment techniques for Markov chain Monte Carlo , 1998, Stat. Comput..

[18]  P. Strevens.  Iii , 1985 .

[19]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[20]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[21]  M. Mahoney,et al.  History of Mathematics , 1924, Nature.

[22]  B. Carlin,et al.  Diagnostics: A Comparative Review , 2022 .

[23]  Jack Dongarra,et al.  LAPACK Users' Guide, 3rd ed. , 1999 .

[24]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[25]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[26]  Stephen J. Wright.  A Collection of Problems for Which Gaussian Elimination with Partial Pivoting is Unstable , 1993, SIAM J. Sci. Comput..

[27]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .