On solving L-SR1 trust-region subproblems

In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact representation of L-SR1 matrices. Our approach makes use of both an orthonormal basis for the eigenspace of the L-SR1 matrix and the Sherman–Morrison–Woodbury formula to compute global solutions to trust-region subproblems. To compute the optimal Lagrange multiplier for the trust-region constraint, we use Newton’s method with a judicious initial guess that does not require safeguarding. A crucial property of this solver is that it is able to compute high-accuracy solutions even in the so-called hard case. Additionally, the optimal solution is determined directly by formula, not iteratively. Numerical experiments demonstrate the effectiveness of this solver.

[1]  採編典藏組.  Society for Industrial and Applied Mathematics(SIAM) , 2008 .

[2]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[3]  David R. Kincaid,et al.  Numerical analysis: mathematics of scientific computing (2nd ed) , 1996 .

[4]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[5]  Henry Wolkowicz,et al.  Measures for Symmetric Rank-One Updates , 1994, Math. Oper. Res..

[6]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[7]  Nicholas I. M. Gould,et al.  Convergence of quasi-Newton matrices generated by the symmetric rank one update , 1991, Math. Program..

[8]  Richard H. Byrd,et al.  Analysis of a Symmetric Rank-One Trust Region Method , 1996, SIAM J. Optim..

[9]  Ekkehard W. Sachs,et al.  Local Convergence of the Symmetric Rank-One Iteration , 1995, Comput. Optim. Appl..

[10]  Roummel F. Marcia,et al.  On Efficiently Computing the Eigenvalues of Limited-Memory Quasi-Newton Matrices , 2014, SIAM J. Matrix Anal. Appl..

[11]  Danny C. Sorensen,et al.  Algorithm 873: LSTRS: MATLAB software for large-scale trust-region subproblems and regularization , 2008, TOMS.

[12]  Maximilian Bayer,et al.  Numerical Analysis Mathematics Of Scientific Computing , 2016 .

[13]  Roummel F. Marcia,et al.  Algorithm 943 , 2014 .

[14]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[15]  XU JAMESV.BURKEANDREASWIEGMANNLIANG.  LIMITED MEMORY BFGS UPDATING IN A TRUST – REGION FRAMEWORK , 1996 .

[16]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[17]  S. Nash,et al.  Linear and Nonlinear Optimization , 2008 .

[18]  William W. Hager,et al.  Global convergence of SSM for minimizing a quadratic over a sphere , 2004, Math. Comput..

[19]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[20]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[21]  Phillipp Kaestner,et al.  Linear And Nonlinear Programming , 2016 .

[22]  Richard H. Byrd,et al.  A Theoretical and Experimental Study of the Symmetric Rank-One Update , 1993, SIAM J. Optim..

[23]  Philip E. Gill,et al.  Iterative Methods for Finding a Trust-region Step , 2009, SIAM J. Optim..

[24]  Philip E. Gill,et al.  A Subspace Minimization Method for the Trust-Region Step , 2009, SIAM J. Optim..

[25]  Ya-Xiang Yuan,et al.  On efficiently combining limited-memory and trust-region techniques , 2017, Math. Program. Comput..

[26]  Nicholas I. M. Gould,et al.  On solving trust-region and other regularised subproblems in optimization , 2010, Math. Program. Comput..

[27]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..