On solving large-scale limited-memory quasi-Newton equations

We consider the problem of solving linear systems of equations arising with limited-memory members of the restricted Broyden class of updates and the symmetric rank-one (SR1) update. In this paper, we propose a new approach based on a practical implementation of the compact representation for the inverse of these limited-memory matrices. Numerical results suggest that the proposed method compares favorably in speed and accuracy to other algorithms and is competitive with several update-specific methods available to only a few members of the Broyden class of updates. Using the proposed approach has an additional benefit: The condition number of the system matrix can be computed efficiently.

[1]  Ya-Xiang Yuan,et al.  On efficiently combining limited-memory and trust-region techniques , 2017, Math. Program. Comput..

[2]  Jorge Nocedal,et al.  On the Behavior of Broyden's Class of Quasi-Newton Methods , 1992, SIAM J. Optim..

[3]  Roummel F. Marcia,et al.  On Efficiently Computing the Eigenvalues of Limited-Memory Quasi-Newton Matrices , 2014, SIAM J. Matrix Anal. Appl..

[4]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[5]  Phillipp Kaestner,et al.  Linear And Nonlinear Programming , 2016 .

[6]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[7]  Jennifer B. Erway,et al.  Shape-Changing L-SR1 Trust-Region Methods , 2016 .

[8]  採編典藏組.  Society for Industrial and Applied Mathematics(SIAM) , 2008 .

[9]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[10]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[11]  Roummel F. Marcia,et al.  Limited-memory BFGS Systems with Diagonal Updates , 2011, 1112.6060.

[12]  Roummel F. Marcia,et al.  On solving L-SR1 trust-region subproblems , 2015, Comput. Optim. Appl..

[13]  Chuanhai Liu,et al.  Statistical Quasi-Newton: A New Look at Least Change , 2007, SIAM J. Optim..

[14]  Jorge Nocedal,et al.  Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..

[15]  R. Tewarson,et al.  Quasi-Newton Algorithms with Updates from the Preconvex Part of Broyden's Family , 1988 .

[16]  Wen Huang,et al.  A Broyden Class of Quasi-Newton Methods for Riemannian Optimization , 2015, SIAM J. Optim..

[17]  S. Nash,et al.  Linear and Nonlinear Programming , 1987 .

[18]  K. Miller.  On the Inverse of the Sum of Matrices , 1981 .

[19]  D. O’Leary,et al.  The linear algebra of block quasi-newton algorithms , 1993 .

[20]  Roummel F. Marcia,et al.  Shifted limited-memory DFP systems , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[21]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[22]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[23]  Gene H. Golub,et al.  Matrix computations , 1983 .