On variable-metric methods for sparse Hessians

The relationship between variable-metric methods derived by norm minimization and those derived by symmetrization of rank-one updates for sparse systems is studied, and an analogue of Dennis's nonsparse symmetrization formula derived. A new method of using norm minimization to produce a sparse analogue of any nonsparse variable-metric method is proposed. The sparse BFGS generated by this method is tested against the sparse PSB and variable-memory conjugate gradient methods, with computational experience uniformly favoring the sparse BFGS.

[1]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[2]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[3]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[4]  Lenhart K. Schubert.  Modification of a quasi-Newton method for nonlinear equations with a sparse Jacobian , 1970 .

[5]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[6]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[7]  J. Greenstadt.  Variations on variable-metric methods. (With discussion) , 1970 .

[8]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[9]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[10]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[11]  J. Greenstadt.  Variations on Variable-Metric Methods , 1970 .

[12]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .

[13]  J. Dennis.  On Some Methods Based on Broyden's Secant Approximation to the Hessian , 1971 .

[14]  M. D. Hebden,et al.  An algorithm for minimization using exact second derivatives , 1973 .

[15]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[16]  Robert B. Schnabel,et al.  Analyzing and Improving Quasi-Newton Methods for Unconstrained Optimization , 1977 .

[17]  Donald Goldfarb,et al.  Generating conjugate directions without line searches using factorized variable metric updating formulas , 1977, Math. Program..

[18]  P. Toint.  On sparse and symmetric matrix updating subject to a linear equation , 1977 .

[19]  P. Toint.  Some numerical results using a sparse matrix updating formula in unconstrained optimization , 1978 .

[20]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[21]  R. Schnabel,et al.  Least Change Secant Updates for Quasi-Newton Methods , 1978 .

[22]  D. Shanno,et al.  Numerical comparison of several variable-metric algorithms , 1978 .

[23]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..

[24]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.