Newton-2EGSOR Method for Unconstrained Optimization Problems with a Block Diagonal Hessian

We find that finding the unconstrained optimizer for large-scale problems with the Newton method can be very expensive. Therefore, in this paper, we developed a new efficient method for solving large-scale unconstrained optimization problems with block diagonal Hessian matrices to reduce the cost of computing Newton’s direction. This method is a combination of the Newton method and the 2-point explicit group successive-over relaxation (2EGSOR) block iterative method. To calculate the performance of the developed method, we used a combination of the Newton method with Gauss–Seidel (Newton-GS) point iteration and the Newton method with successive-over relaxation (Newton-SOR) point iteration as reference methods. The numerical experiment has proven that the developed algorithms generate results that are more efficient compared to the reference methods with less execution time and fewer iterations. Through 90 test cases, we observe that the speedup ratio for our proposed method is up to 659.87 times faster compared to the Newton-SOR method and up to 1963.57 times more rapidly than the Newton-GS method.

[1]  Abdul Rahman Abdullah.  The four point explicit decoupled group (EDG) Method: a fast poisson solver , 1991, Int. J. Comput. Math..

[2]  J. Killingbeck,et al.  A Gauss elimination method for resonances , 2009, 0906.4420.

[3]  Musa Mammadov,et al.  Globally convergent algorithms for solving unconstrained optimization problems , 2015 .

[4]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[5]  Jumat Sulaiman,et al.  Numerical solutions of nonlinear second-order two-point boundary value problems using half-sweep SOR with Newton Method , 2013 .

[6]  D. J. Evans,et al.  Explicit group over-relaxation methods for solving elliptic partial differential equations , 1986 .

[7]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[8]  Yosza Dasril,et al.  Application of Newton-4EGSOR Iteration for Solving Large Scale Unconstrained Optimization Problems with a Tridiagonal Hessian Matrix , 2019 .

[9]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[10]  Akbar Hashemi Borzabadi,et al.  An adaptive nonmonotone global Barzilai–Borwein gradient method for unconstrained optimization , 2017 .

[11]  Yosza Dasril,et al.  Newton method with explicit group iteration for solving large scale unconstrained optimization problems , 2018 .

[12]  Leandro dos Santos Coelho,et al.  A hybrid shuffled complex evolution approach with pattern search for unconstrained optimization , 2011, Math. Comput. Simul..

[13]  A Comparison of Newton’s Method and Two of its Modifications for the Solution of Nonlinear Optimization Problems , 2017 .

[14]  Hao Zhang,et al.  A new regularized quasi-Newton method for unconstrained optimization , 2018, Optim. Lett..

[15]  T. N. Grapsa,et al.  A modified Newton direction for unconstrained optimization , 2014 .

[16]  Emin Kahya.  A class of exponential quadratically convergent iterative formulae for unconstrained optimization , 2007, Appl. Math. Comput..

[17]  D. J. Evans,et al.  Group explicit iterative methods for solving large linear systems , 1985 .

[18]  Yixun Shi,et al.  Globally Convergent Algorithms for Unconstrained Optimization , 2000, Comput. Optim. Appl..

[19]  I. Moghrabi.  MINIMIZATION OF EXTENDED QUADRATIC FUNCTIONS WITH INEXACT LINE SEARCHES , 2005 .

[20]  Xiao-Wei Gao,et al.  The application of simultaneous elimination and back-substitution method (SEBSM) in finite element method , 2016 .

[21]  A. U. Moyi,et al.  A sufficient descent three-term conjugate gradient method via symmetric rank-one update for large-scale optimization , 2016 .

[22]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[23]  David J. Evans,et al.  Explicit De-coupled Group Iterative Methods and their Parallel Implementations , 1995, Parallel Algorithms Appl..

[24]  Sergey N. Savenkov,et al.  Incomplete active polarimetry: Measurement of the block-diagonal scattering matrix , 2011 .

[25]  Sanjeev Kumar,et al.  Geometrically Constructed Families of Newton's Method for Unconstrained Optimization and Nonlinear Equations , 2011, Int. J. Math. Math. Sci..

[26]  Jumat Sulaiman,et al.  Fourth-order solutions of nonlinear two-point boundary value problems by Newton-HSSOR iteration , 2014 .

[27]  Jing Zhang,et al.  An extended nonmonotone line search technique for large-scale unconstrained optimization , 2018, J. Comput. Appl. Math..

[28]  M. Othman,et al.  An efficient four points modified explicit group poisson solver , 2000, Int. J. Comput. Math..