Numerical Methods for Large Sparse Linear Least Squares Problems

Large sparse least squares problems arise in many applications, including geodetic network adjustments and finite element structural analysis. Although geodesists and engineers have been solving such problems for years, it is only relatively recently that numerical analysts have turned attention to them. In this paper we present a survey of numerical methods for large sparse linear least squares problems, focusing mainly on developments since the last comprehensive surveys of the subject published in 1976. We consider direct methods based on elimination and on orthogonalization, as well as various iterative methods. The ramifications of rank deficiency, constraints, and updating are also discussed.

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  W. Givens.  Computation of Plain Unitary Rotations Transforming a General Matrix to Triangular Form , 1958 .

[3]  Alston S. Householder,et al.  Unitary Triangularization of a Nonsymmetric Matrix , 1958, JACM.

[4]  E. E. Osborne.  On Least Squares Solutions of Linear Equations , 1961, JACM.

[5]  Peter Läuchli,et al.  Jordan-Elimination und Ausgleichung nach kleinsten Quadraten , 1961 .

[6]  G. Golub,et al.  Linear least squares solutions by householder transformations , 1965 .

[7]  J. Rice.  Experiments on Gram-Schmidt orthogonalization , 1966 .

[8]  Å. Björck.  Solving linear least squares problems by Gram-Schmidt orthogonalization , 1967 .

[9]  Åke Björck,et al.  Iterative refinement of linear least squares solutions II , 1967 .

[10]  John K. Reid,et al.  A Note on the Least Squares Solution of a Band System of Linear Equations by Householder Reductions , 1967, Comput. J..

[11]  James Hardy Wilkinson,et al.  The Least Squares Problem and Pseudo-Inverses , 1970, Comput. J..

[12]  M. Saunders.  Large-scale linear programming using the Cholesky factorization , 1972 .

[13]  C. Paige,et al.  An error analysis of a method for solving matrix equations , 1973 .

[14]  A. K. Cline.  An Elimination Method for the Solution of Linear Least Squares Problems , 1973 .

[15]  W. E. Gentleman.  Least Squares Computations by Givens Transformations Without Square Roots , 1973 .

[16]  R. Plemmons.  Linear Least Squares by Elimination and MGS , 1974, JACM.

[17]  Ying-Tung Chen.  Iterative methods for linear least squares problems. , 1974 .

[18]  S. Hammarling.  A Note on Modifications to the Givens Plane Rotation , 1974 .

[19]  W. Gentleman.  Error analysis of QR decompositions by Givens transformations , 1975 .

[20]  G. Stewart.  The economical storage of plane rotations , 1976 .

[21]  I. Duff,et al.  A Comparison of Some Methods for the Solution of Sparse Overdetermined Systems of Linear Equations , 1976 .

[22]  R. E. Cline,et al.  $l_2 $-Solutions to Underdetermined Linear Systems , 1976 .

[23]  Philip E. Gill,et al.  THE ORTHOGONAL FACTORIZATION OF A LARGE SPARSE MATRIX , 1976 .

[24]  W. Morven Gentleman,et al.  Row elimination for solving sparse linear systems and least squares problems , 1976 .

[25]  George A. F. Seber,et al.  Linear regression analysis , 1977 .

[26]  G B Kolata,et al.  Geodesy: dealing with an enormous computer task. , 1978, Science.

[27]  Åke Björck,et al.  Comment on the Iterative Refinement of Least-Squares Solutions , 1978 .

[28]  J. Goodnight.  A Tutorial on the SWEEP Operator , 1979 .

[29]  L. Delves,et al.  A Fast Direct Method for the Least Squares Solution of Slightly Overdetermined Sets of Linear Equations , 1979 .

[30]  Å. Björck,et al.  A direct method for the solution of sparse linear least squares problems , 1980 .

[31]  N. Munksgaard,et al.  Solving Sparse Symmetric Sets of Linear Equations by Preconditioned Conjugate Gradients , 1980, TOMS.

[32]  Iain S. Duff,et al.  MA28 --- A set of Fortran subroutines for sparse unsymmetric linear equations , 1980 .

[33]  A. George,et al.  Solution of sparse linear least squares problems using givens rotations , 1980 .

[34]  T. Manteuffel.  An incomplete factorization technique for positive definite linear systems , 1980 .

[35]  Michael T. Heath,et al.  Solution of Large-Scale Sparse Least Squares Problems Using Auxiliary Storage , 1981 .

[36]  Alan George,et al.  Computer Solution of Large Sparse Positive Definite , 1981 .

[37]  Franklin T. Luk,et al.  A Block Lanczos Method for Computing the Singular Values and Corresponding Singular Vectors of a Matrix , 1981, TOMS.

[38]  Stanley C. Eisenstat,et al.  Algorithms and Data Structures for Sparse Symmetric Gaussian Elimination , 1981 .

[39]  D. O’Leary,et al.  A Bidiagonalization-Regularization Procedure for Large Scale Discretizations of Ill-Posed Problems , 1981 .

[40]  M. Cox.  The Least Squares Solution of Overdetermined Linear Equations Having Band or Augmented Band Structure , 1981 .

[41]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[42]  Iain S. Duff,et al.  MA27 -- A set of Fortran subroutines for solving sparse symmetric sets of linear equations , 1982 .

[43]  Michael A. Saunders,et al.  Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems , 1982, TOMS.

[44]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[45]  M. Heath.  Some Extensions of an Algorithm for Sparse Linear Least Squares Problems , 1982 .

[46]  Z. Zlatev.  Comparison of two pivotal strategies in sparse plane rotations , 1980 .

[47]  A. George,et al.  On Row and Column Orderings for Sparse Least Squares Problems , 1983 .

[48]  A. George,et al.  A Comparison of Some Methods for Solving Sparse Linear Least-Squares Problems , 1983 .

[49]  Michael T. Heath,et al.  Solution of Sparse Underdetermined Systems of Linear Equations , 1984 .

[50]  Å. Björck.  A General Updating Algorithm for Constrained Linear Least Squares Problems , 1984 .