On the Generalized Lanczos Trust-Region Method

The so-called trust-region subproblem gets its name in the trust-region method in optimization and also plays a vital role in various other applications. Several numerical algorithms have been proposed in the literature for solving small-to-medium size dense problems as well as for large-scale sparse problems. The generalized Lanczos trust-region (GLTR) method proposed by [N. I. M. Gould, S. Lucidi, M. Roma and P. L. Toint, SIAM J. Optim., 9 (1999), pp. 561--580] is a natural extension of the classical Lanczos method for the linear system to the trust-region subproblem. In this paper, we first analyze the convergence of GLTR to reveal its convergence behavior in theory and then propose new stopping criteria that can be integrated into GLTR for better numerical performance. Specifically, we develop a priori upper bounds for the convergence to both the optimal objective value as well as the optimal solution and argue that these bounds can be efficiently estimated numerically and serve as stopping criteria f...

[1]  Danny C. Sorensen,et al.  A Trust-Region Approach to the Regularization of Large-Scale Discrete Forms of Ill-Posed Problems , 2001, SIAM J. Sci. Comput..

[2]  W. Yang,et al.  Error bounds of Lanczos approach for trust-region subproblem , 2018 .

[3]  William W. Hager,et al.  Graph Partitioning and Continuous Quadratic Programming , 1999, SIAM J. Discret. Math..

[4]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[5]  Galahad , 1906 .

[6]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[7]  Ren-Cang Li,et al.  Sharpness in rates of convergence for the symmetric Lanczos method , 2010, Math. Comput..

[8]  Y. Saad.  On the Rates of Convergence of the Lanczos and the Block-Lanczos Methods , 1980 .

[9]  Ren-Cang Li,et al.  Vandermonde matrices with Chebyshev nodes , 2008 .

[10]  Danny C. Sorensen,et al.  Algorithm 873: LSTRS: MATLAB software for large-scale trust-region subproblems and regularization , 2008, TOMS.

[11]  Ren-Cang Li,et al.  On Meinardus' examples for the conjugate gradient method , 2008, Math. Comput..

[12]  Nicholas I. M. Gould,et al.  On solving trust-region and other regularised subproblems in optimization , 2010, Math. Program. Comput..

[13]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[14]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[15]  Ren-Cang Li,et al.  Bounding the spectrum of large Hermitian matrices , 2011 .

[16]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[17]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[18]  G. Golub,et al.  Quadratically constrained least squares and quadratic problems , 1991 .

[20]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[21]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[22]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[23]  E. Cheney.  Introduction to approximation theory , 1966 .

[24]  Ren-Cang Li,et al.  Convergence of the block Lanczos method for eigenvalue clusters , 2014, Numerische Mathematik.

[25]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[26]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[27]  Ya-Xiang Yuan,et al.  On the truncated conjugate gradient method , 2000, Math. Program..

[28]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[29]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[30]  G. Meinardus.  Approximation of Functions: Theory and Numerical Methods , 1967 .

[31]  David S. Watkins.  Iterative Methods for Linear Systems , 2005 .

[32]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[33]  Maxim A. Olshanskii.  Iterative Methods For Linear Systems , 2017 .

[34]  L. Lukšan,et al.  On Lagrange multipliers of trust-region subproblems , 2008 .

[35]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[36]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[37]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[38]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .