Algorithm 873: LSTRS: MATLAB software for large-scale trust-region subproblems and regularization

A MATLAB 6.0 implementation of the LSTRS method is presented. LSTRS was described in Rojas et al. [2000]. LSTRS is designed for large-scale quadratic problems with one norm constraint. The method is based on a reformulation of the trust-region subproblem as a parameterized eigenvalue problem, and consists of an iterative procedure that finds the optimal value for the parameter. The adjustment of the parameter requires the solution of a large-scale eigenvalue problem at each step. LSTRS relies on matrix-vector products only and has low and fixed storage requirements, features that make it suitable for large-scale computations. In the MATLAB implementation, the Hessian matrix of the quadratic objective function can be specified either explicitly, or in the form of a matrix-vector multiplication routine. Therefore, the implementation preserves the matrix-free nature of the method. A description of the LSTRS method and of the MATLAB software, version 1.2, is presented. Comparisons with other techniques and applications of the method are also included. A guide for using the software and examples are provided.

[1]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[2]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM Rev..

[3]  Gene H. Golub,et al.  Matrix computations , 1983 .

[4]  G. Golub,et al.  Quadratically constrained least squares and quadratic problems , 1991 .

[5]  L. Eldén.  Algorithms for the regularization of ill-conditioned least squares problems , 1977 .

[6]  Le Thi Hoai An,et al.  A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem , 1998, SIAM J. Optim..

[7]  Wolfgang Osten,et al.  Introduction to Inverse Problems in Imaging , 1999 .

[8]  Arnold Neumaier,et al.  Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on Regularization , 1998, SIAM Rev..

[9]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[10]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[11]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[12]  Per Christian Hansen,et al.  Regularization methods for large-scale problems , 1993 .

[13]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[14]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM J. Matrix Anal. Appl..

[15]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[16]  W. Gander,et al.  A D.C. OPTIMIZATION ALGORITHM FOR SOLVING THE TRUST-REGION SUBPROBLEM∗ , 1998 .

[17]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[18]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[19]  Danny C. Sorensen,et al.  Implicit Application of Polynomial Filters in a k-Step Arnoldi Method , 1992, SIAM J. Matrix Anal. Appl..

[20]  Henry Wolkowicz,et al.  Regularization Using a Parameterized Trust Region Subproblem , 2022 .

[21]  Henry Wolkowicz,et al.  The trust region subproblem and semidefinite programming , 2004, Optim. Methods Softw..

[22]  D. Sorensen,et al.  A large-scale trust-region approach to the regularization of discrete ill-posed problems , 1999 .

[23]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[24]  Keh-Shew Lu,et al.  DIGITAL FILTER DESIGN , 1973 .

[25]  A. Carasso.  Determining Surface Temperatures from Interior Observations , 1982 .

[26]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[27]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[28]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[29]  P. Hansen.  The discrete picard condition for discrete ill-posed problems , 1990 .

[30]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[31]  Danny C. Sorensen,et al.  A Trust-Region Approach to the Regularization of Large-Scale Discrete Forms of Ill-Posed Problems , 2001, SIAM J. Sci. Comput..

[32]  Trond Steihaug,et al.  An interior-point trust-region-based method for large-scale non-negative regularization , 2002 .

[33]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[34]  W. Symes.  A Differential Semblance Criterion for Inversion of Multioffset Seismic Reflection Data , 1993 .

[35]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[36]  F. Natterer.  The Mathematics of Computerized Tomography , 1986 .

[37]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[38]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[39]  Guust Nolet,et al.  Seismic tomography : with applications in global seismology and exploration geophysics , 1987 .

[40]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[41]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[42]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[43]  Per Christian Hansen,et al.  Minimization of Linear Functionals Defined on Solutions of Large-Scale Discrete Ill-Posed Problems , 2005 .

[44]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[45]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[46]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[47]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[48]  Laura Palagi,et al.  On Some Properties of Quadratic Programs with a Convex Quadratic Constraint , 1998, SIAM J. Optim..