ML(n)BiCGStab: Reformulation, Analysis and Implementation

With the aid of index functions, we re-derive the ML(n)BiCGStab algorithm in a paper by Yeung and Chan in 1999 in a more systematic way. It turns out that there are n ways to define the ML(n)BiCGStab residual vector. Each definition will lead to a different ML(n)BiCGStab algorithm. We demonstrate this by presenting a second algorithm which requires less storage. In theory, this second algorithm serves as a bridge that connects the Lanczos-based BiCGStab and the Arnoldi-based FOM while ML(n)BiCG a bridge connecting BiCG and FOM. We also analyze the breakdown situations from the probabilistic point of view and summarize some useful properties of ML(n)BiCGStab. Implementation issues are also addressed.

[1]  R. Fletcher.  Conjugate gradient methods for indefinite systems , 1976 .

[2]  Qiang Ye,et al.  Breakdown-free GMRES for Singular Systems , 2005, SIAM J. Matrix Anal. Appl..

[3]  Gerard L. G. Sleijpen,et al.  Bi-CGSTAB as an induced dimension reduction method , 2010 .

[4]  Y. Saad,et al.  Iterative solution of linear systems in the 20th century , 2000 .

[5]  Yimin Wei,et al.  On the convergence of general stationary iterative methods for range-Hermitian singular linear systems , 2010, Numer. Linear Algebra Appl..

[6]  Jörg Liesen,et al.  Deflated and augmented Krylov subspace methods: Basic Facts and a Breakdown-free deflated MINRES , 2011 .

[7]  Randolph E. Bank,et al.  A composite step bi-conjugate gradient algorithm for nonsymmetric linear systems , 1994, Numerical Algorithms.

[8]  Martin H. Gutknecht,et al.  Look-Ahead Procedures for Lanczos-Type Product Methods Based on Three-Term Lanczos Recurrences , 2000, SIAM J. Matrix Anal. Appl..

[9]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[10]  Takashi Nodera,et al.  Breakdown-Free ML(k)BiCGStab Algorithm for Non-Hermitian Linear Systems , 2005, ICCSA.

[11]  H. Sadok,et al.  A block version of BiCGSTAB for linear systems with multiple right-hand sides. , 2003 .

[12]  H. V. der.  Residual Replacement Strategies for Krylov Subspace Iterative Methods for the Convergence of True Residuals , 2000 .

[13]  Hebing Wu,et al.  Convergence properties of Krylov subspace methods for singular linear systems with arbitrary index , 2000 .

[14]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[15]  Valeria Simoncini,et al.  Recent computational developments in Krylov subspace methods for linear systems , 2007, Numer. Linear Algebra Appl..

[16]  Tony F. Chan,et al.  ML(k)BiCGSTAB: A BiCGSTAB Variant Based on Multiple Lanczos Starting Vectors , 1999, SIAM J. Sci. Comput..

[17]  Gerard L. G. Sleijpen,et al.  Maintaining convergence properties of BiCGstab methods in finite precision arithmetic , 1995, Numerical Algorithms.

[18]  P. Wesseling,et al.  Numerical experiments with a multiple grid and a preconditioned Lanczos type method , 1980 .

[19]  Shao-Liang Zhang,et al.  GPBi-CG: Generalized Product-type Methods Based on Bi-CG for Solving Nonsymmetric Linear Systems , 1997, SIAM J. Sci. Comput..

[20]  Gerard L. G. Sleijpen,et al.  BiCGstab(l) and other hybrid Bi-CG methods , 1994, Numerical Algorithms.

[21]  A. Berman,et al.  Cones and Iterative Methods for Best Least Squares Solutions of Linear Systems , 1974 .

[22]  Daniel B. Szyld,et al.  Comparison theorems for weak splittings of bounded operators , 1990 .

[23]  Man-Chung Yeung,et al.  An introduction to ML(n)BiCGStab , 2011, ArXiv.

[24]  H. V. D. Vorst,et al.  An overview of approaches for the stable computation of hybrid BiCG methods , 1995 .

[25]  R. Freund,et al.  A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides , 1997 .

[26]  Daniel Boley,et al.  Transpose-free multiple Lanczos and its application in Padé approximation , 2005 .

[27]  E. F. Kaasschieter,et al.  Preconditioned conjugate gradients for solving singular systems , 1988 .

[28]  P. Sonneveld,et al.  IDR(s): A family of simple and fast algorithms for solving large nonsymmetric linear systems , 2007 .

[29]  M. B. Van Gijzen,et al.  An elegant IDR(s) variant that efficiently exploits bi-orthogonality properties , 2010 .

[30]  H. Walker,et al.  GMRES On (Nearly) Singular Systems , 1997, SIAM J. Matrix Anal. Appl..

[31]  Martin H. Gutknecht,et al.  Lanczos-type solvers for nonsymmetric linear systems of equations , 1997, Acta Numerica.

[32]  Qiang Ye,et al.  Residual Replacement Strategies for Krylov Subspace Iterative Methods for the Convergence of True Residuals , 2000, SIAM J. Sci. Comput..

[33]  Roland W. Freund,et al.  A Lanczos-type method for multiple starting vectors , 2000, Math. Comput..

[34]  Y. Saad.  The Lanczos Biorthogonalization Algorithm and Other Oblique Projection Methods for Solving Large Unsymmetric Systems , 1982 .

[35]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[36]  Gerard L. G. Sleijpen,et al.  Reliable updated residuals in hybrid Bi-CG methods , 1996, Computing.

[37]  D. R. Fokkema,et al.  BiCGstab(ell) for Linear Equations involving Unsymmetric Matrices with Complex Spectrum , 1993 .

[38]  C. Lanczos.  Solution of Systems of Linear Equations by Minimized Iterations1 , 1952 .

[39]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[40]  P. Sonneveld.  CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[41]  Wayne Joubert,et al.  Lanczos Methods for the Solution of Nonsymmetric Systems of Linear Equations , 1992, SIAM J. Matrix Anal. Appl..

[42]  Martin H. Gutknecht,et al.  Variants of BICGSTAB for Matrices with Complex Spectrum , 1993, SIAM J. Sci. Comput..

[43]  M. Gutknecht.  A Completed Theory of the Unsymmetric Lanczos Process and Related Algorithms. Part II , 1994, SIAM J. Matrix Anal. Appl..

[44]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[45]  Martin H. Gutknecht,et al.  A Completed Theory of the Unsymmetric Lanczos Process and Related Algorithms, Part I , 1992, SIAM J. Matrix Anal. Appl..

[46]  Roland W. Freund,et al.  An Implementation of the Look-Ahead Lanczos Algorithm for Non-Hermitian Matrices , 1993, SIAM J. Sci. Comput..

[47]  W. Joubert.  Generalized conjugate gradient and Lanczos methods for the solution of nonsymmetric systems of linear equations , 1990 .

[48]  A. Edelman.  Eigenvalues and condition numbers of random matrices , 1988 .