A practical method for solving large-scale TRS

We present a nearly-exact method for the large scale trust region subproblem (TRS) based on the properties of the minimal-memory BFGS method. Our study is concentrated in the case where the initial BFGS matrix can be any scaled identity matrix. The proposed method is a variant of the Moré–Sorensen method that exploits the eigenstructure of the approximate Hessian B, and incorporates both the standard and the hard case. The eigenvalues of B are expressed analytically, and consequently a direction of negative curvature can be computed immediately by performing a sequence of inner products and vector summations. Thus, the hard case is handled easily while the Cholesky factorization is completely avoided. An extensive numerical study is presented, for covering all the possible cases arising in the TRS with respect to the eigenstructure of B. Our numerical experiments confirm that the method is suitable for very large scale problems.

[1]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[2]  Panayiotis E. Pintelas,et al.  Solving the quadratic trust-region subproblem in a low-memory BFGS framework , 2008, Optim. Methods Softw..

[3]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[4]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[5]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[6]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[7]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[8]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[9]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .

[10]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[11]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[12]  William W. Hager,et al.  Graph Partitioning and Continuous Quadratic Programming , 1999, SIAM J. Discret. Math..

[13]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[14]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[15]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[16]  J. Charles,et al.  LIBOPT – An environment for testing solvers on heterogeneous collections of problems – The manual , version 2 . 0 – , 2009 .

[17]  Richard H. Byrd,et al.  Approximate solution of the trust region problem by minimization over two-dimensional subspaces , 1988, Math. Program..

[18]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[19]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[20]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[21]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[22]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[23]  Ya-Xiang Yuan.  Advances in Nonlinear Programming , 1998 .

[24]  I. Duff.  Sparse Matrices and Their Uses. , 1983 .

[25]  Pablo Moscato,et al.  Handbook of Applied Optimization , 2000 .

[26]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[27]  Adam Ouorou.  Implementing a proximal algorithm for some nonlinear multicommodity flow problems , 2007, Networks.

[28]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[29]  W. Menke.  Geophysical data analysis : discrete inverse theory , 1984 .

[30]  Panos M. Pardalos,et al.  Interior Point Methods for Global Optimization , 1996 .

[31]  Ilse C. F. Ipsen.  Computing an Eigenvector with Inverse Iteration , 1997, SIAM Rev..

[32]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[33]  Jean Charles Gilbert,et al.  LIBOPT - An environment for testing solvers on heterogeneous collections of problems - Version 1.0 , 2007, ArXiv.

[34]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[35]  José Mario Martínez,et al.  Structured minimal-memory inexact quasi-Newton method and secant preconditioners for augmented Lagrangian optimization , 2008, Comput. Optim. Appl..