Tensor Methods for Large, Sparse Nonlinear Least Squares Problems

This paper introduces tensor methods for solving large, sparse nonlinear least squares problems where the Jacobian either is analytically available or is computed by finite difference approximations. Tensor methods have been shown to have very good computational performance for small to medium-sized, dense nonlinear least squares problems. In this paper we consider the application of tensor methods to large, sparse nonlinear least squares problems where a sparse factorization of the Jacobian can be stored. This involves an entirely new way of solving the tensor model that is efficient for sparse problems. A number of interesting linear algebraic implementation issues are addressed. The test results of the tensor method applied to a set of sparse nonlinear least squares problems compared with those of the standard Gauss--Newton method reveal that the tensor method is significantly more robust and efficient than the standard Gauss--Newton method.

[1]  Åke Björck.  Iterative refinement of linear least squares solutions I , 1967 .

[2]  N. S. Barnett,et al.  Private communication , 1969 .

[3]  James Hardy Wilkinson,et al.  The Least Squares Problem and Pseudo-Inverses , 1970, Comput. J..

[4]  J. Bunch.  Partial Pivoting Strategies for Symmetric Matrices , 1972 .

[5]  W. E. Gentleman.  Least Squares Computations by Givens Transformations Without Square Roots , 1973 .

[6]  I. Duff,et al.  A Comparison of Some Methods for the Solution of Sparse Overdetermined Systems of Linear Equations , 1976 .

[7]  Å. Björck,et al.  A direct method for the solution of sparse linear least squares problems , 1980 .

[8]  Iain S. Duff,et al.  MA28 --- A set of Fortran subroutines for sparse unsymmetric linear equations , 1980 .

[9]  John E. Dennis,et al.  An Adaptive Nonlinear Least-Squares Algorithm , 1977, TOMS.

[10]  J. J. Moré,et al.  Estimation of sparse jacobian matrices and graph coloring problems , 1983 .

[11]  John D. Ramsdell,et al.  Estimation of Sparse Jacobian Matrices , 1983 .

[12]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[13]  Thomas F. Coleman,et al.  Algorithm 618: FORTRAN subroutines for estimating sparse Jacobian Matrices , 1984, TOMS.

[14]  Thomas F. Coleman,et al.  Software for estimating sparse Jacobian matrices , 1984, ACM Trans. Math. Softw..

[15]  R. Schnabel,et al.  Tensor Methods for Nonlinear Equations. , 1984 .

[16]  Bobby Schnabel,et al.  A modular system of algorithms for unconstrained minimization , 1985, TOMS.

[17]  M. Al-Baali,et al.  Variational Methods for Non-Linear Least-Squares , 1985 .

[18]  I. Duff,et al.  On the augmented system approach to sparse least-squares problems , 1989 .

[19]  A. Bouaricha.  Solving large sparse systems of nonlinear equations and nonlinear least squares problems using tensor methods on sequential and parallel computers , 1992 .

[20]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[21]  Dan Feng,et al.  Local convergence analysis of tensor methods for nonlinear equations , 1993, Math. Program..

[22]  Andreas Griewank,et al.  Computing Large Sparse Jacobian Matrices Using Automatic Differentiation , 1994, SIAM J. Sci. Comput..

[23]  Bobby Schnabel,et al.  Algorithm 768: TENSOLVE: a software package for solving systems of nonlinear equations and nonlinear least-squares problems using tensor methods , 1997, TOMS.

[24]  Bobby Schnabel,et al.  Tensor methods for large sparse systems of nonlinear equations , 1996, Math. Program..