Accurate solution of dense linear systems, part I: Algorithms in rounding to nearest

We investigate how extra-precise accumulation of dot products can be used to solve ill-conditioned linear systems accurately. For a given p-bit working precision, extra-precise evaluation of a dot product means that the products and summation are executed in 2p-bit precision, and that the final result is rounded into the p-bit working precision. Denote by u=2^-^p the relative rounding error unit in a given working precision. We treat two types of matrices: first up to condition number u^-^1, and second up to condition number u^-^2. For both types of matrices we present two types of methods: first for calculating an approximate solution, and second for calculating rigorous error bounds for the solution together with the proof of non-singularity of the matrix of the linear system. In the first part of this paper we present algorithms using only rounding to nearest, in Part II we use directed rounding to obtain better results. All algorithms are given in executable Matlab code and are available from my homepage.

[1]  Tetsuro Yamamoto,et al.  Error bounds for approximate solutions of systems of equations , 1984 .

[2]  Siegfried M. Rump,et al.  Convergence of Rump's method for inverting arbitrarily ill-conditioned matrices , 2007 .

[3]  Siegfried M. Rump,et al.  A Method for the Generation of a Class of Ill-conditioned Matrices , 2008 .

[4]  James Demmel,et al.  Error bounds from extra-precise iterative refinement , 2006, TOMS.

[5]  SIEGFRIED M. RUMP,et al.  VERIFIED SOLUTIONS OF LINEAR SYSTEMS WITHOUT DIRECTED ROUNDING , 2005 .

[6]  James Demmely,et al.  A Reference Implementation for Extended and Mixed Precision Blas , 2007 .

[7]  Siegfried M. Rump,et al.  A method of obtaining verified solutions for linear systems suited for Java , 2007 .

[8]  Wen Lea Pearn,et al.  (Journal of Computational and Applied Mathematics,228(1):274-278)Optimization of the T Policy M/G/1 Queue with Server Breakdowns and General Startup Times , 2009 .

[9]  Donald E. Knuth.  The Art of Computer Programming 2 / Seminumerical Algorithms , 1971 .

[10]  Michael A. Malcolm,et al.  Computer methods for mathematical computations , 1977 .

[11]  M. Porte,et al.  Etude statistique des erreurs dans l'arithmetique des ordinateurs; application au controle des resultats d'algorithmes numeriques , 1974 .

[12]  Guido D. Salvucci,et al.  Ieee standard for binary floating-point arithmetic , 1985 .

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Douglas M. Priest.  On properties of floating point arithmetics: numerical stability and the cost of accurate computations , 1992 .

[15]  Xiaoye S. Li,et al.  ARPREC: An arbitrary precision computation package , 2002 .

[16]  YONG-KANG ZHU,et al.  A New Distillation Algorithm for Floating-Point Summation , 2005, SIAM J. Sci. Comput..

[17]  M. Porte,et al.  Algorithmes numériques : analyse et mise en œuvre , 1974 .

[18]  Siegfried M. Rump,et al.  Accurate Sum and Dot Product , 2005, SIAM J. Sci. Comput..

[19]  Siegfried M. Rump,et al.  INTLAB - INTerval LABoratory , 1998, SCAN.

[20]  Siegfried M. Rump.  Error estimation of floating-point summation and dot product , 2012 .

[21]  Nicholas J. Higham,et al.  Experience with a Matrix Norm Estimator , 1990, SIAM J. Sci. Comput..

[22]  James Demmel,et al.  Accurate and efficient expression evaluation and linear algebra , 2008, Acta Numerica.

[23]  Bernhard Beckermann,et al.  The condition number of real Vandermonde, Krylov and positive definite Hankel matrices , 2000, Numerische Mathematik.

[24]  L. Trefethen,et al.  Average-case stability of Gaussian elimination , 1990 .

[25]  Juan M. Molera,et al.  MULTIPLICATIVE PERTURBATION THEORY AND ACCURATE SOLUTION OF LEAST SQUARES PROBLEMS , 2012 .

[26]  Siegfried M. Rump,et al.  Ultimately Fast Accurate Summation , 2009, SIAM J. Sci. Comput..

[27]  James Demmel.  Accurate Singular Value Decompositions of Structured Matrices , 2000, SIAM J. Matrix Anal. Appl..

[28]  R. Skeel.  Iterative refinement implies numerical stability for Gaussian elimination , 1980 .

[29]  Andreas Frommer,et al.  Proving Conjectures by Use of Interval Arithmetic , 2001, Perspectives on Enclosure Methods.

[30]  Froilán M. Dopico,et al.  Accurate solution of structured linear systems via rank-revealing decompositions , 2012 .

[31]  J. Mixter.  Fast , 2012 .

[32]  Yozo Hida,et al.  Accurate Floating Point Summation , 2002 .

[33]  James Demmel,et al.  Accurate and Efficient Floating Point Summation , 2003, SIAM J. Sci. Comput..

[34]  A. Neumaier.  Rundungsfehleranalyse einiger Verfahren zur Summation endlicher Summen , 1974 .

[35]  L. Collatz.  Einschließungssatz für die charakteristischen Zahlen von Matrizen , 1942 .

[36]  Jean-Michel Muller,et al.  Handbook of Floating-Point Arithmetic (2nd Ed.) , 2018 .

[37]  Siegfried M. Rump.  A class of arbitrarily ill conditioned floating-point matrices , 1991 .

[38]  Ivo Babuska.  Numerical stability in mathematical analysis , 1968, IFIP Congress.

[39]  Jonathan Richard Shewchuk,et al.  Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates , 1997, Discret. Comput. Geom..

[40]  James Demmel,et al.  Design, implementation and testing of extended and mixed precision BLAS , 2000, TOMS.

[41]  Ansi Ieee,et al.  IEEE Standard for Binary Floating Point Arithmetic , 1985 .

[42]  W. Tucker.  The Lorenz attractor exists , 1999 .

[43]  Siegfried M. Rump,et al.  Accurate Floating-Point Summation Part I: Faithful Rounding , 2008, SIAM J. Sci. Comput..

[44]  Siegfried M. Rump,et al.  Inversion of extremely Ill-conditioned matrices in floating-point , 2009 .

[45]  T. J. Dekker,et al.  A floating-point technique for extending the available precision , 1971 .

[46]  Froilán M. Dopico,et al.  Accurate Solution of Structured Least Squares Problems via Rank-Revealing Decompositions , 2013, SIAM J. Matrix Anal. Appl..

[47]  Pedro Alonso,et al.  Growth factors of pivoting strategies associated with Neville elimination , 2011, J. Comput. Appl. Math..

[48]  Axel Facius,et al.  SCAN 2000: GAMM-IMACS International Symposium on Scientific Computing, Computer Arithmetic, and Validated Numerics Interval 2000: International Conference on Interval Methods in Science and Engineering Karlsruhe, Germany, September 18–22, 2000 , 2001, Reliab. Comput..

[49]  James Demmel,et al.  IEEE Standard for Floating-Point Arithmetic , 2008 .

[50]  Siegfried M. Rump,et al.  Accurate Matrix Multiplication by using Level 3 BLAS Operation , 2008 .

[51]  Nicholas J. Higham,et al.  The Accuracy of Floating Point Summation , 1993, SIAM J. Sci. Comput..

[52]  On the generation of very ill-conditioned integer matrices , 2011 .

[53]  Plamen Koev.  Accurate Computations with Totally Nonnegative Matrices , 2007, SIAM J. Matrix Anal. Appl..

[54]  L. Trefethen,et al.  Condition Numbers of Random Triangular Matrices , 1996, SIAM J. Matrix Anal. Appl..

[55]  J. Boothroyd.  Algorithm 274: Generation of Hilbert derived test matrix , 1966, CACM.

[56]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[57]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[58]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[59]  Vincent Lefèvre,et al.  MPFR: A multiple-precision binary floating-point library with correct rounding , 2007, TOMS.

[60]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[61]  Walter Gautschi,et al.  Optimally scaled and optimally conditioned Vandermonde and Vandermonde-like matrices , 2011 .

[62]  Jim Euchner.  Design , 2014, Catalysis from A to Z.

[63]  James Demmel,et al.  Accurate SVDs of Structured Matrices , 1998 .