Recursive Trust-Region Methods for Multiscale Nonlinear Optimization

A class of trust-region methods is presented for solving unconstrained nonlinear and possibly nonconvex discretized optimization problems, like those arising in systems governed by partial differential equations. The algorithms in this class make use of the discretization level as a means of speeding up the computation of the step. This use is recursive, leading to true multilevel/multiscale optimization methods reminiscent of multigrid methods in linear algebra and the solution of partial differential equations. A simple algorithm of the class is then described and its numerical performance is shown to be numerically promising. This observation then motivates a proof of global convergence to first-order stationary points on the fine grid that is valid for all algorithms in the class.

[1]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[2]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[3]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[4]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..

[5]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[6]  P. Toint,et al.  Local convergence analysis for partitioned quasi-Newton updates , 1982 .

[7]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[8]  Gene H. Golub,et al.  Matrix computations , 1983 .

[9]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[10]  W. Hackbusch,et al.  Analysis of a damped nonlinear multilevel method , 1989 .

[11]  E. Gelman,et al.  On multilevel iterative methods for optimization problems , 1990, Math. Program..

[12]  P. Wesseling.  An Introduction to Multigrid Methods , 1992 .

[13]  W. Hackbusch.  Iterative Solution of Large Sparse Systems of Equations , 1993 .

[14]  Annick Sartenaer,et al.  Automatic Determination of an Initial Trust Region in Nonlinear Programming , 1997, SIAM J. Sci. Comput..

[15]  Philippe L. Toint,et al.  Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints , 1997, Math. Program..

[16]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[17]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[18]  S. Lucidi,et al.  A Linesearch Algorithm with Memory for Unconstrained Optimization , 1998 .

[19]  P. Gill,et al.  Combination trust-region line-search methods for unconstrained optimization , 1999 .

[20]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[21]  M. Ulbrich.  Non-Monotone Trust-Region Methods for Bound-Constrained Semismooth Equations with Applications to No , 1999 .

[22]  E. Sachs,et al.  Trust-region proper orthogonal decomposition for flow control , 2000 .

[23]  S. Nash.  A multigrid approach to discretized optimization problems , 2000 .

[24]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[25]  R. Lewis,et al.  An Overview of First-Order Model Management for Engineering Optimization , 2001 .

[26]  P. A. Newman,et al.  Approximation and Model Management in Aerodynamic Optimization with Variable-Fidelity Models , 2001 .

[27]  Stephen G. Nash,et al.  Practical Aspects of Multiscale Optimization Methods for VLSICAD , 2003 .

[28]  John T. Betts,et al.  Optimal Low Thrust Trajectories to the Moon , 2003, SIAM J. Appl. Dyn. Syst..

[29]  Marco Fahl,et al.  Reduced Order Modelling Approaches to PDE-Constrained Optimization Based on Proper Orthogonal Decomposition , 2003 .

[30]  Lois Curfman McInnes,et al.  Scalable Algorithms in Optimization: Computational Experiments , 2004 .

[31]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[32]  Nicholas I. M. Gould,et al.  A Filter-Trust-Region Method for Unconstrained Optimization , 2005, SIAM J. Optim..

[33]  Stephen G. Nash,et al.  Model Problems for the Multigrid Optimization of Systems Governed by Differential Equations , 2005, SIAM J. Sci. Comput..

[34]  Yi Xiao,et al.  Nonmonotone trust region methods with curvilinear path in unconstrained optimization , 1992, Computing.

[35]  Irad Yavneh,et al.  A Multilevel Nonlinear Method , 2006, SIAM J. Sci. Comput..

[36]  Serge,et al.  SECOND-ORDER CONVERGENCE PROPERTIES OF TRUST-REGION METHODS USING INCOMPLETE CURVATURE INFORMATION, WITH AN APPLICATION TO MULTIGRID OPTIMIZATION * , 2006 .

[37]  Karl Kunisch,et al.  A globalization strategy for the multigrid solution of elliptic optimal control problems , 2006, Optim. Methods Softw..

[38]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.