Exploiting variable precision in GMRES

We describe how variable precision floating point arithmetic can be used in the iterative solver GMRES. We show how the precision of the inner products carried out in the algorithm can be reduced as the iterations proceed, without affecting the convergence rate or final accuracy achieved by the iterates. Our analysis explicitly takes into account the resulting loss of orthogonality in the Arnoldi vectors. We also show how inexact matrix-vector products can be incorporated into this setting.

[1]  Christopher C. Paige,et al.  Properties of a Unitary Matrix Obtained from a Sequence of Normalized Vectors , 2014, SIAM J. Matrix Anal. Appl..

[2]  Valeria Simoncini,et al.  Theory of Inexact Krylov Subspace Methods and Applications to Scientific Computing , 2003, SIAM J. Sci. Comput..

[3]  Miroslav Rozlozník,et al.  Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES , 2006, SIAM J. Matrix Anal. Appl..

[4]  Andrew J. Wathen,et al.  On the choice of preconditioner for minimum residual methods for non-Hermitian matrices , 2013, J. Comput. Appl. Math..

[5]  M. Rozložník,et al.  Numerical stability of GMRES , 1995 .

[6]  Nicholas J. Higham,et al.  Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions , 2018, SIAM J. Sci. Comput..

[7]  Serge Gratton,et al.  Minimizing convex quadratic with variable precision Krylov methods , 2018, ArXiv.

[8]  R. Freund.  Quasi-kernel polynomials and their use in non-Hermitian matrix iterations , 1992 .

[9]  Christopher C. Paige,et al.  Scaled total least squares fundamentals , 2002, Numerische Mathematik.

[10]  Zdenek Strakos,et al.  Bounds for the least squares distance using scaled total least squares , 2002, Numerische Mathematik.

[11]  Valérie Frayssé,et al.  Inexact Matrix-Vector Products in Krylov Methods for Solving Linear Systems: A Relaxation Strategy , 2005, SIAM J. Matrix Anal. Appl..

[12]  Stefan Güttel,et al.  Some observations on weighted GMRES , 2014, Numerical Algorithms.

[13]  Nicholas J. Higham,et al.  Harnessing GPU Tensor Cores for Fast FP16 Arithmetic to Speed up Mixed-Precision Iterative Refinement Solvers , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.

[14]  Walter Gander,et al.  Gram‐Schmidt orthogonalization: 100 years and more , 2013, Numer. Linear Algebra Appl..

[15]  Å. Björck.  Solving linear least squares problems by Gram-Schmidt orthogonalization , 1967 .

[16]  M. Rozložník,et al.  Numerical behaviour of the modified gram-schmidt GMRES implementation , 1997 .

[17]  Nicholas J. Higham,et al.  Simulating Low Precision Floating-Point Arithmetic , 2019, SIAM J. Sci. Comput..

[18]  Christopher C. Paige,et al.  A Useful Form of Unitary Matrix Obtained from Any Sequence of Unit 2-Norm n-Vectors , 2009, SIAM J. Matrix Anal. Appl..

[19]  Jack J. Dongarra,et al.  The Design of Fast and Energy-Efficient Linear Solvers: On the Potential of Half-Precision Arithmetic and Iterative Refinement Techniques , 2018, ICCS.

[20]  Valeria Simoncini,et al.  Recent computational developments in Krylov subspace methods for linear systems , 2007, Numer. Linear Algebra Appl..

[21]  Christopher C. Paige,et al.  Loss and Recapture of Orthogonality in the Modified Gram-Schmidt Algorithm , 1992, SIAM J. Matrix Anal. Appl..

[22]  Jack J. Dongarra,et al.  Investigating half precision arithmetic to accelerate dense linear system solvers , 2017, ScalA@SC.

[23]  Julien Langou,et al.  Convergence in Backward Error of Relaxed GMRES , 2007, SIAM J. Sci. Comput..

[24]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[25]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[26]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[27]  Gerard L. G. Sleijpen,et al.  Inexact Krylov Subspace Methods for Linear Systems , 2004, SIAM J. Matrix Anal. Appl..