Extra-Precise Iterative Refinement for Overdetermined Least Squares Problems

We present the algorithm, error bounds, and numerical results for extra-precise iterative refinement applied to overdetermined linear least squares (LLS) problems. We apply our linear system refinement algorithm to Björck’s augmented linear system formulation of an LLS problem. Our algorithm reduces the forward normwise and componentwise errors to <i>O</i>(<i>ϵ</i><sub>w</sub>), where <i>ϵ</i><sub>w</sub> is the working precision, unless the system is too ill conditioned. In contrast to linear systems, we provide two separate error bounds for the solution <i>x</i> and the residual <i>r</i>. The refinement algorithm requires only limited use of extra precision and adds only <i>O</i>(<i>mn</i>) work to the <i>O</i>(<i>mn</i><sup>2</sup>) cost of QR factorization for problems of size <i>m</i>-by-<i>n</i>. The extra precision calculation is facilitated by the new extended-precision BLAS standard in a portable way, and the refinement algorithm will be included in a future release of LAPACK and can be extended to the other types of least squares problems.

[1]  James Demmel,et al.  Design, implementation and testing of extended and mixed precision BLAS , 2000, TOMS.

[2]  G. Forsythe,et al.  Computer solution of linear algebraic systems , 1969 .

[3]  Ji-Guang Sun,et al.  Optimal backward perturbation bounds for the linear least squares problem , 1995, Numer. Linear Algebra Appl..

[4]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[5]  J. Uhlig.  C. Forsythe and C. B. Moler, Computer Solution of Linear Algebraic Systems. (Series in Automatic Computation) XI + 148 S. Englewood Cliffs, N.J. 1967. Prentice-Hall, Inc. Preis geb. 54 s. net , 1972 .

[6]  Åke Björck,et al.  Iterative refinement of linear least squares solutions II , 1967 .

[7]  J. L. Rigal,et al.  On the Compatibility of a Given Solution With the Data of a Linear System , 1967, JACM.

[8]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[9]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[10]  Roy H. Wampler.  A Report on the Accuracy of Some Widely Used Least Squares Computer Programs , 1970 .

[11]  G. Forsythe,et al.  Computer solution of linear algebraic systems , 1969 .

[12]  Nicholas J. Higham,et al.  FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation , 1988, TOMS.

[13]  J. Dongarra,et al.  Exploiting the Performance of 32 bit Floating Point Arithmetic in Obtaining 64 bit Accuracy (Revisiting Iterative Refinement for Linear Systems) , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[14]  Åke Björck.  Iterative refinement of linear least squares solutions I , 1967 .

[15]  Yimin Wei,et al.  On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems , 2006, Math. Comput..

[16]  Mårten Gulliksson.  Iterative refinement for constrained and weighted linear least squares , 1994 .

[17]  G. Stewart.  On the Perturbation of Pseudo-Inverses, Projections and Linear Least Squares Problems , 1977 .

[18]  James Demmel,et al.  Error bounds from extra-precise iterative refinement , 2006, TOMS.

[19]  Serge Gratton,et al.  A Partial Condition Number for Linear Least Squares Problems , 2007, SIAM J. Matrix Anal. Appl..

[20]  Å. Björck.  Component-wise perturbation analysis and error bounds for linear least squares solutions , 1991 .

[21]  W. Prager,et al.  Compatibility of approximate solution of linear equations with given error bounds for coefficients and right-hand sides , 1964 .

[22]  J. K. Reid,et al.  Implicit Scaling of Linear Least Squares Problems , 2000 .