Solving the quadratic trust-region subproblem in a low-memory BFGS framework

We present a new matrix-free method for the large-scale trust-region subproblem, assuming that the approximate Hessian is updated by the L-BFGS formula with m=1 or 2. We determine via simple formulas the eigenvalues of these matrices and, at each iteration, we construct a positive definite matrix whose inverse can be expressed analytically, without using factorization. Consequently, a direction of negative curvature can be computed immediately by applying the inverse power method. The computation of the trial step is obtained by performing a sequence of inner products and vector summations. Furthermore, it immediately follows that the strong convergence properties of trust region methods are preserved. Numerical results are also presented.

[1]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[2]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[3]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[4]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[5]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[6]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[7]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[8]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[9]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[10]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[11]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[12]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[13]  Ilse C. F. Ipsen.  Computing an Eigenvector with Inverse Iteration , 1997, SIAM Rev..

[14]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[15]  W. Menke.  Geophysical data analysis , 1984 .

[16]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[17]  Adam Ouorou.  Implementing a proximal algorithm for some nonlinear multicommodity flow problems , 2007, Networks.

[18]  William W. Hager,et al.  Graph Partitioning and Continuous Quadratic Programming , 1999, SIAM J. Discret. Math..

[19]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[20]  Adam Ouorou,et al.  A proximal subgradient projection algorithm for linearly constrained strictly convex problems , 2007, Optim. Methods Softw..

[21]  P. Borwein,et al.  Polynomials and Polynomial Inequalities , 1995 .

[22]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[23]  Danny C. Sorensen,et al.  Algorithm 873: LSTRS: MATLAB software for large-scale trust-region subproblems and regularization , 2008, TOMS.

[24]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[25]  W. Menke.  Geophysical data analysis : discrete inverse theory , 1984 .

[26]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[27]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[28]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[29]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[30]  William M. Faucette.  A Geometric Interpretation of the Solution of the General Quartic Polynomial , 1996 .