Approximate invariant subspaces and quasi-newton optimization methods

New approximate secant equations are shown to result from the knowledge of (problem dependent) invariant subspace information, which in turn suggests improvements in quasi-Newton methods for unconstrained minimization. A new limited-memory Broyden–Fletcher–Goldfarb–Shanno using approximate secant equations is then derived and its encouraging behaviour illustrated on a small collection of multilevel optimization examples. The smoothing properties of this algorithm are considered next, and automatic generation of approximate eigenvalue information demonstrated. The use of this information for improving algorithmic performance is finally investigated on the same multilevel examples.

[1]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[2]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[3]  William C. Davidon,et al.  Variable Metric Method for Minimization , 1959, SIAM J. Optim..

[4]  S. Nash.  A multigrid approach to discretized optimization problems , 2000 .

[5]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[6]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[7]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[8]  Stephen M. Griffies,et al.  Fundamentals of Ocean Climate Models , 2004 .

[9]  Stephen G. Nash,et al.  Model Problems for the Multigrid Optimization of Systems Governed by Differential Equations , 2005, SIAM J. Sci. Comput..

[10]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[11]  DONALD GOLDFARB,et al.  A LINE SEARCH MULTIGRID METHOD FOR LARGE-SCALE CONVEX OPTIMIZATION , 2007 .

[12]  Serge,et al.  SECOND-ORDER CONVERGENCE PROPERTIES OF TRUST-REGION METHODS USING INCOMPLETE CURVATURE INFORMATION, WITH AN APPLICATION TO MULTIGRID OPTIMIZATION * , 2006 .

[13]  P. Courtier,et al.  Correlation modelling on the sphere using a generalized diffusion equation , 2001 .

[14]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .

[15]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[16]  J. Demmel,et al.  The strong stability of algorithms for solving symmetric linear systems , 1989 .

[17]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[18]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[19]  Serge Gratton,et al.  Recursive Trust-Region Methods for Multiscale Nonlinear Optimization , 2008, SIAM J. Optim..

[20]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[21]  Richard Asselin,et al.  Frequency Filter for Time Integrations , 1972 .

[22]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[23]  P. Toint,et al.  A recursive trust-region method in infinity norm for bound-constrained nonlinear optimization , 2007 .

[24]  P. Toint,et al.  A recursive ℓ∞-trust-region method for bound-constrained nonlinear optimization , 2008 .

[25]  Stephen G. Nash,et al.  Practical Aspects of Multiscale Optimization Methods for VLSICAD , 2003 .

[26]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[27]  David J. Thuente,et al.  Line search algorithms with guaranteed sufficient decrease , 1994, TOMS.

[28]  J. Haslinger,et al.  Solution of Variational Inequalities in Mechanics , 1988 .

[29]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[30]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[31]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..