A diagonal quasi-Newton updating method for unconstrained optimization

A diagonal quasi-Newton updating algorithm is presented. The elements of the diagonal matrix approximating the Hessian are determined by minimizing both the size of the change from the previous estimate and the trace of the update, subject to the weak secant equation. Under mild classical assumptions, the convergence of the algorithm is proved to be linear. The diagonal quasi-Newton update satisfies the bounded deterioration property. Numerical experiments with 80 unconstrained optimization test problems of different structures and complexities prove that the suggested algorithm is more efficient and more robust than the steepest descent, Cauchy with Oren and Luenberger scaling algorithm in its complementary form and classical Broyden-Fletcher-Goldfarb-Shanno algorithm.

[1]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[2]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[3]  June Leong Wah,et al.  A new diagonal gradient-type method for large scale unconstrained optimization , 2013 .

[4]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[5]  M. Zhu,et al.  The Quasi-Cauchy Relation and Diagonal Updating , 1999, SIAM J. Optim..

[6]  J. Dennis,et al.  Sizing and least-change secant methods , 1993 .

[7]  Philip E. Gill,et al.  Practical optimization , 1981 .

[8]  John E. Dennis,et al.  On the Local and Superlinear Convergence of Quasi-Newton Methods , 1973 .

[9]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[10]  M. Farid,et al.  Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method , 2010 .

[11]  Jorge Nocedal,et al.  Theory of algorithms for unconstrained optimization , 1992, Acta Numerica.

[12]  Neculai Andrei,et al.  Continuous Nonlinear Optimization for Engineering Applications in GAMS Technology , 2017 .

[13]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[14]  Siam Rfview,et al.  CONVERGENCE CONDITIONS FOR ASCENT METHODS , 2016 .

[15]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[16]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[17]  S. Nash.  Preconditioning of Truncated-Newton Methods , 1985 .

[18]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[19]  David G. Luenberger,et al.  Linear and Nonlinear Programming: Second Edition , 2003 .

[20]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[21]  William C. Davidon,et al.  Variable Metric Method for Minimization , 1959, SIAM J. Optim..

[22]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[23]  Keyvan Amini,et al.  A new structured quasi-Newton algorithm using partial information on Hessian , 2010, J. Comput. Appl. Math..

[24]  Michael Bartholomew-Biggs,et al.  Nonlinear Optimization with Engineering Applications , 2008 .