Compact representation of the full Broyden class of quasi-Newton updates

In this paper, we present the compact representation for matrices belonging to the the Broyden class of quasi-Newton updates, where each update may be either rank-one or rank-two. This work extends previous results solely for the restricted Broyden class of rank-two updates. In this article, it is not assumed the same Broyden update is used each iteration; rather, different members of the Broyden class may be used each iteration. Numerical experiments suggest that a practical implementation of the compact representation is able to accurately represent matrices belonging to the Broyden class of updates. Furthermore, we demonstrate how to compute the compact representation for the inverse of these matrices, as well as a practical algorithm for solving linear systems with members of the Broyden class of updates. We demonstrate through numerical experiments that the proposed linear solver is able to efficiently solve linear systems with members of the Broyden class of matrices to high accuracy. As an immediate consequence of this work, it is now possible to efficiently compute the eigenvalues of any limited-memory member of the Broyden class of matrices, allowing for the computation of condition numbers and the ability perform sensitivity analysis.

[1]  Roummel F. Marcia,et al.  Trust-region methods for nonconvex sparse recovery optimization , 2016, 2016 International Symposium on Information Theory and Its Applications (ISITA).

[2]  Roummel F. Marcia,et al.  On solving large-scale limited-memory quasi-Newton equations , 2015, 1510.06378.

[3]  J. Burke,et al.  Notes On Limited Memory BFGS Updating In A Trust-Region Framework , 1996 .

[4]  Ya-Xiang Yuan,et al.  On efficiently combining limited-memory and trust-region techniques , 2017, Math. Program. Comput..

[5]  XU JAMESV.BURKEANDREASWIEGMANNLIANG.  LIMITED MEMORY BFGS UPDATING IN A TRUST – REGION FRAMEWORK , 1996 .

[6]  Jan Vlcek,et al.  Recursive form of general limited memory variable metric methods , 2013, Kybernetika.

[7]  Ignacio E. Grossmann,et al.  On the solution of nonconvex cardinality Boolean quadratic programming problems: a computational study , 2017, Comput. Optim. Appl..

[8]  Roummel F. Marcia,et al.  On solving L-SR1 trust-region subproblems , 2015, Comput. Optim. Appl..

[9]  Gene H. Golub,et al.  Matrix computations , 1983 .

[10]  Roummel F. Marcia,et al.  On Efficiently Computing the Eigenvalues of Limited-Memory Quasi-Newton Matrices , 2014, SIAM J. Matrix Anal. Appl..

[11]  Wen Huang,et al.  A Broyden Class of Quasi-Newton Methods for Riemannian Optimization , 2015, SIAM J. Optim..

[12]  Jennifer B. Erway,et al.  Shape-Changing L-SR1 Trust-Region Methods , 2016 .

[13]  Jorge Nocedal,et al.  On the Behavior of Broyden's Class of Quasi-Newton Methods , 1992, SIAM J. Optim..

[14]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[15]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[16]  Roummel F. Marcia,et al.  Limited-memory trust-region methods for sparse relaxation , 2017, Optical Engineering + Applications.

[17]  Chuanhai Liu,et al.  Statistical Quasi-Newton: A New Look at Least Change , 2007, SIAM J. Optim..

[18]  R. Tewarson,et al.  Quasi-Newton Algorithms with Updates from the Preconvex Part of Broyden's Family , 1988 .