Preconditioned Iterative Methods for Solving Linear Least Squares Problems

New preconditioning strategies for solving $m \times n$ overdetermined large and sparse linear least squares problems using the conjugate gradient for least squares (CGLS) method are described. First, direct preconditioning of the normal equations by the balanced incomplete factorization (BIF) for symmetric and positive definite matrices is studied, and a new breakdown-free strategy is proposed. Preconditioning based on the incomplete LU factors of an $n \times n$ submatrix of the system matrix is our second approach. A new way to find this submatrix based on a specific weighted transversal problem is proposed. Numerical experiments demonstrate different algebraic and implementational features of the new approaches and put them into the context of current progress in preconditioning of CGLS. It is shown, in particular, that the robustness demonstrated earlier by the BIF preconditioning strategy transfers into the linear least squares solvers and the use of the weighted transversal helps to improve the LU-...

[1]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[2]  Jacek Gondzio,et al.  A Matrix-Free Preconditioner for Sparse Symmetric Positive Definite Systems and Least-Squares Problems , 2013, SIAM J. Sci. Comput..

[3]  Kincho H. Law,et al.  A robust incomplete factorization based on value and space constraints , 1995 .

[4]  Bora Uçar,et al.  Design, implementation, and analysis of maximum transversal algorithms , 2011, ACM Trans. Math. Softw..

[5]  Z. Zlatev,et al.  SOLVING LARGE AND SPARSE LINEAR LEAST-SQUARES PROBLEMS BY CONJUGATE GRADIENT ALGORITHMS , 1988 .

[6]  Iain S. Duff,et al.  Remarks on implementation of O(n1/2τ) assignment algorithms , 1988, TOMS.

[7]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[8]  James Hardy Wilkinson,et al.  The Least Squares Problem and Pseudo-Inverses , 1970, Comput. J..

[9]  Michele Benzi,et al.  Preconditioning Highly Indefinite and Nonsymmetric Matrices , 2000, SIAM J. Sci. Comput..

[10]  Miroslav Tuma,et al.  Balanced Incomplete Factorization , 2008, SIAM J. Sci. Comput..

[11]  Miroslav Tůma,et al.  The importance of structure in incomplete factorization preconditioners , 2011 .

[12]  Xiaoke Cui.  Approximate Generalized Inverse Preconditioning Methods for Least Squares Problems , 2009 .

[13]  M. Benzi,et al.  A comparative study of sparse approximate inverse preconditioners , 1999 .

[14]  Iain S. Duff,et al.  The Design and Use of Algorithms for Permuting Large Entries to the Diagonal of Sparse Matrices , 1999, SIAM J. Matrix Anal. Appl..

[15]  Gene H. Golub,et al.  A Rank-One Reduction Formula and Its Applications to Matrix Factorizations , 1995, SIAM Rev..

[16]  Miroslav Tuma,et al.  Improved Balanced Incomplete Factorization , 2010, SIAM J. Matrix Anal. Appl..

[17]  Richard M. Karp,et al.  A n^5/2 Algorithm for Maximum Matchings in Bipartite Graphs , 1971, SWAT.

[18]  A. Jennings,et al.  Incomplete Methods for Solving $A^T Ax = b$ , 1984 .

[19]  Nicholas I. M. Gould,et al.  New crash procedures for large systems of linear constraints , 1989, Math. Program..

[20]  Xiaoge Wang.  Incomplete factorization preconditioning for linear least squares problems , 1994 .

[21]  Iain S. Duff,et al.  A Class of Incomplete Orthogonal Factorization Methods. I: Methods and Theories , 1999 .

[22]  Eli Hellerman,et al.  The Partitioned Preassigned Pivot Procedure (P4) , 1972 .

[23]  Y. Saad.  Preconditioning techniques for nonsymmetric and indefinite linear systems , 1988 .

[24]  Michele Benzi,et al.  A Robust Preconditioner with Low Memory Requirements for Large Sparse Least Squares Problems , 2003, SIAM J. Sci. Comput..

[25]  Na Li,et al.  MIQR: A Multilevel Incomplete QR Preconditioner for Large Sparse Least-Squares Problems , 2006, SIAM J. Matrix Anal. Appl..

[26]  Michele Benzi,et al.  A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method , 1996, SIAM J. Sci. Comput..

[27]  Igor E. Kaporin,et al.  High quality preconditioning of a general symmetric positive definite matrix based on its U , 1998 .

[28]  Jeffery L. Kennington,et al.  Recovery from Numerical Instability during Basis Reinversion , 1997, Comput. Optim. Appl..

[29]  Gene H. Golub,et al.  Matrix computations , 1983 .

[30]  Juana Cerdán,et al.  Preconditioning Sparse Nonsymmetric Linear Systems with the Sherman-Morrison Formula , 2003, SIAM J. Sci. Comput..

[31]  V. Eijkhout.  On the existence problem of incomplete factorisation methods, Lapack Working Note 144, UT-CS-99-435 , 1999 .

[32]  Yousef Saad,et al.  On the Relations between ILUs and Factored Approximate Inverses , 2002, SIAM J. Matrix Anal. Appl..

[33]  Olaf Schenk,et al.  Weighted Matchings for Preconditioning Symmetric Indefinite Linear Systems , 2006, SIAM J. Sci. Comput..

[34]  Z. Zlatev.  Computational Methods for General Sparse Matrices , 1991 .

[35]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[36]  Peter Läuchli,et al.  Jordan-Elimination und Ausgleichung nach kleinsten Quadraten , 1961 .

[37]  Å. Björck,et al.  Preconditioners for least squares problems by LU factorization. , 1999 .

[38]  Ken Hayami,et al.  Generalized approximate inverse preconditioners for least squares problems , 2009 .

[39]  M. A. Ajiz,et al.  A robust incomplete Choleski‐conjugate gradient algorithm , 1984 .

[40]  Iain S. Duff,et al.  On Algorithms For Permuting Large Entries to the Diagonal of a Sparse Matrix , 2000, SIAM J. Matrix Anal. Appl..

[41]  William G. Poole,et al.  A Structurally Stable Modification of Hellerman–Rarick’s ${\text{P}}^4 $ Algorithm for Reordering Unsymmetric Sparse Matrices , 1985 .

[42]  Iain S. Duff,et al.  A Class of Incomplete Orthogonal Factorization Methods. II: Implementation and Results , 2005 .

[43]  Xiaoge Wang,et al.  CIMGS: An Incomplete Orthogonal FactorizationPreconditioner , 1997, SIAM J. Sci. Comput..

[44]  Owe Axelsson,et al.  Diagonally compensated reduction and related preconditioning methods , 1994, Numer. Linear Algebra Appl..

[45]  Timothy A. Davis,et al.  Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization , 2011, TOMS.

[46]  Jun-Feng Yin,et al.  Greville’s method for preconditioning least squares problems , 2011, Adv. Comput. Math..

[47]  Jim Euchner.  Design , 2014, Catalysis from A to Z.

[48]  José MarÍn José Mas Rafael Bru,et al.  Balanced Incomplete Factorization , 2008 .

[49]  M. Tismenetsky,et al.  A new preconditioning technique for solving large sparse linear systems , 1991 .

[50]  Wenbin,et al.  A High-Quality Preconditioning Technique for Multi-Length-Scale Symmetric Positive Definite Linear Systems , 2009 .

[51]  Olaf Schenk,et al.  Matching-based preprocessing algorithms to the solution of saddle-point problems in large-scale nonconvex interior-point optimization , 2007, Comput. Optim. Appl..

[52]  Douglas James.  Conjugate Gradient Methods for Constrained Least Squares Problems , 1990 .

[53]  A. Neumaier,et al.  A NEW PIVOTING STRATEGY FOR GAUSSIAN ELIMINATION , 1996 .

[54]  Richard M. Karp,et al.  A n^5/2 Algorithm for Maximum Matchings in Bipartite Graphs , 1971, SWAT.

[55]  Xiao-Wen Chang,et al.  Stopping Criteria for the Iterative Solution of Linear Least Squares Problems , 2009, SIAM J. Matrix Anal. Appl..

[56]  Yousef Saad,et al.  ILUT: A dual threshold incomplete LU factorization , 1994, Numer. Linear Algebra Appl..

[57]  Robert M. Freund,et al.  A note on two block-SOR methods for sparse least squares problems , 1987 .

[58]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[59]  Igor E. Kaporin,et al.  High quality preconditioning of a general symmetric positive definite matrix based on its UTU + UTR + RTU-decomposition , 1998, Numer. Linear Algebra Appl..

[60]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[61]  T. Greville,et al.  Some Applications of the Pseudoinverse of a Matrix , 1960 .

[62]  IAIN S. DUFF,et al.  Towards Stable Mixed Pivoting Strategies for the Sequential and Parallel Solution of Sparse Symmetric Indefinite Systems , 2007, SIAM J. Matrix Anal. Appl..

[63]  Matthias Bollhöfer,et al.  A Robust and Efficient ILU that Incorporates the Growth of the Inverse Triangular Factors , 2003, SIAM J. Sci. Comput..

[64]  R. Kouhia,et al.  Stabilized and block approximate inverse preconditioners for problems in solid and structural mechanics , 2001 .

[65]  Matthias Bollhöfer,et al.  A robust ILU with pivoting based on monitoring the growth of the inverse factors , 2001 .