Issues on the use of a modified Bunch and Kaufman decomposition for large scale Newton’s equation

In this work, we deal with Truncated Newton methods for solving large scale (possibly nonconvex) unconstrained optimization problems. In particular, we consider the use of a modified Bunch and Kaufman factorization for solving the Newton equation, at each (outer) iteration of the method. The Bunch and Kaufman factorization of a tridiagonal matrix is an effective and stable matrix decomposition, which is well exploited in the widely adopted SYMMBK (Bunch and Kaufman in Math Comput 31:163–179, 1977; Chandra in Conjugate gradient methods for partial differential equations, vol 129, 1978; Conn et al. in Trust-region methods. MPS-SIAM series on optimization, Society for Industrial Mathematics, Philadelphia, 2000; HSL, A collection of Fortran codes for large scale scientific computation, http://www.hsl.rl.ac.uk/ ; Marcia in Appl Numer Math 58:449–458, 2008) routine. It can be used to provide conjugate directions, both in the case of $$1\times 1$$ and $$2\times 2$$ pivoting steps. The main drawback is that the resulting solution of Newton’s equation might not be gradient–related, in the case the objective function is nonconvex. Here we first focus on some theoretical properties, in order to ensure that at each iteration of the Truncated Newton method, the search direction obtained by using an adapted Bunch and Kaufman factorization is gradient–related. This allows to perform a standard Armijo-type linesearch procedure, using a bounded descent direction. Furthermore, the results of an extended numerical experience using large scale CUTEst problems is reported, showing the reliability and the efficiency of the proposed approach, both on convex and nonconvex problems.

[1]  G. Fasano.  Planar Conjugate Gradient Algorithm for Large-Scale Unconstrained Optimization, Part 1: Theory , 2005 .

[2]  S. Nash,et al.  Assessing a search direction within a truncated-newton method , 1990 .

[3]  G. Fasano.  Lanczos Conjugate-Gradient Method and Pseudoinverse Computation on Indefinite and Singular Systems , 2007 .

[4]  Roummel F. Marcia,et al.  On solving sparse symmetric linear systems whose definiteness is unknown , 2008 .

[5]  G. Fasano.  Lanczos-Conjugate Gradient method and pseudoinverse computation, in unconstrained optimization , 2004 .

[6]  Massimo Roma,et al.  Data article Title : Data and performance profiles applying an adaptive truncation criterion , within linesearch-based truncated Newton methods , in large scale nonconvex optimization , 2017 .

[7]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[8]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[9]  S. Nash.  A survey of truncated-Newton methods , 2000 .

[10]  Massimo Roma,et al.  Iterative computation of negative curvature directions in large scale optimization , 2007, Comput. Optim. Appl..

[11]  Josef Stoer,et al.  Solution of Large Linear Systems of Equations by Conjugate Gradient Type Methods , 1982, ISMP.

[12]  R. Chandra.  Conjugate gradient methods for partial differential equations. , 1978 .

[13]  Massimo Roma,et al.  Preconditioning Newton–Krylov methods in nonconvex large scale optimization , 2013, Comput. Optim. Appl..

[14]  L. Grippo,et al.  A truncated Newton method with nonmonotone line search for unconstrained optimization , 1989 .

[15]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[16]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[17]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[18]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[19]  Trond Steihaug,et al.  Truncated-newtono algorithms for large-scale unconstrained optimization , 1983, Math. Program..

[20]  Massimo Roma,et al.  An adaptive truncation criterion, for linesearch-based truncated Newton methods in large scale nonconvex optimization , 2018, Oper. Res. Lett..

[21]  G. Fasano.  Planar Conjugate Gradient Algorithm for Large-Scale Unconstrained Optimization, Part 2: Application , 2005 .

[22]  Yaroslav D. Sergeyev,et al.  Planar methods and grossone for the Conjugate Gradient breakdown in nonlinear programming , 2018, Comput. Optim. Appl..

[23]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .