Inexact trust region PGC method for large sparse unconstrained optimization

We present an algorithm, partitioning group correction (PGC) algorithm based on trust region and conjugate gradient method, for large-scale sparse unconstrained optimization. In large sparse optimization, computing the whole Hessian matrix and solving the Newton-like equations at each iteration can be considerably expensive when a trust region method is adopted. The method depends on a symmetric consistent partition of the columns of the Hessian matrix and an inaccurate solution to the Newton-like equations by conjugate gradient method. And we allow that the current direction exceeds the trust region bound if it is a good descent direction. Besides, we studies a method dealing with some sparse matrices having a dense structure part. Some good convergence properties are kept and we contrast the computational behavior of our method with that of other algorithms. Our numerical tests show that the algorithm is promising and quite effective, and that its performance is comparable to or better than that of other algorithms available.

[1]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[2]  J. Dennis,et al.  Two new unconstrained optimization algorithms which use function and gradient values , 1979 .

[3]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[4]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[5]  G. Y. Li.  Successive element correction algorithms for sparse unconstrained optimization , 1993 .

[6]  Trond Steihaug,et al.  On the successive projections approach to least-squares problems , 1986 .

[7]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[8]  Stephen Wolfram,et al.  The Mathematica Book , 1996 .

[9]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[10]  Ya-Xiang Yuan.  Advances in Nonlinear Programming , 1998 .

[11]  Trond Steihaug,et al.  Truncated-newtono algorithms for large-scale unconstrained optimization , 1983, Math. Program..

[12]  Richard J. Fateman,et al.  Automatic Differentiation of Algorithms: Theory, Implementation, and Application (Andreas Griewank and George F. Corliss, eds.) , 1993, SIAM Rev..

[13]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[14]  Yanfei Wang,et al.  A new trust region algorithm for image restoration , 2005 .

[15]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[16]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[17]  Ya-Xiang Yuan,et al.  A trust region-CG algorithm for deblurring problem in atmospheric image reconstruction , 2002 .

[18]  Thomas F. Coleman,et al.  Software for estimating sparse Hessian matrices , 1985, TOMS.

[19]  A. Griewank,et al.  Automatic differentiation of algorithms : theory, implementation, and application , 1994 .

[20]  M. D. Hebden,et al.  An algorithm for minimization using exact second derivatives , 1973 .

[21]  Thomas F. Coleman,et al.  Estimation of sparse hessian matrices and graph coloring problems , 1982, Math. Program..

[22]  M. Powell,et al.  On the Estimation of Sparse Hessian Matrices , 1979 .

[23]  Junxiang Li,et al.  Truncated partitioning group correction algorithms for large-scale sparse unconstrained optimization , 2007, Appl. Math. Comput..

[24]  Anne Greenbaum,et al.  Iterative Solution Methods for Linear Systems , 2014 .

[25]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[26]  H. W. Zhang,et al.  On the convergence of partitioning group correction algorithms , 2007, Appl. Math. Comput..

[27]  Stephen Wolfram,et al.  The Mathematica book (3rd ed.) , 1996 .

[28]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[29]  J. Michael Harrison,et al.  A Method for Staffing Large Call Centers Based on Stochastic Fluid Models , 2005, Manuf. Serv. Oper. Manag..

[30]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .