Application of Newton-4EGSOR Iteration for Solving Large Scale Unconstrained Optimization Problems with a Tridiagonal Hessian Matrix

Solving the unconstrained optimization problems using Newton method will lead to the need to solve linear system. Further, the Explicit Group iteration is one of the numerical methods that has an advantage of the efficient block iterative method for solving any linear system. Thus, in this paper to reduce the cost of solving large linear system, we proposed a combination between Newton method with four-point Explicit Group (4-point EG) block iterative method for solving large scale unconstrained optimization problems where the Hessian of the Newton direction is tridiagonal matrices. For the purpose of comparison, we used combination of Newton method with basic iterative method namely successive-over relaxation (SOR) point iteration and Newton method with two-point Explicit Group (2-point EG) block iterative method as reference method. The proposed method shows that the numerical results were more superior compared to the reference methods in term of execution time and number of iteration.

[1]  Zhen-Jun Shi,et al.  Step-size estimation for unconstrained optimization methods , 2005 .

[2]  S. Kaniel,et al.  A Modified Newton's Method for Unconstrained Minimization , 1979 .

[3]  Jumat Sulaiman,et al.  Fourth-order solutions of nonlinear two-point boundary value problems by Newton-HSSOR iteration , 2014 .

[4]  F. S. Sisser.  A modified Newton's method for minimizing factorable functions , 1982 .

[5]  David J. Evans,et al.  A Parallel four points modified explicit group algorithm on shared memory multiprocessors , 2004, Parallel Algorithms Appl..

[6]  Mohd,et al.  Newton-EGMSOR Methods for Solution of Second Order Two-Point Nonlinear Boundary Value Problems , 2012 .

[7]  Philip E. Gill,et al.  Newton-type methods for unconstrained and linearly constrained optimization , 1974, Math. Program..

[8]  Ting-Zhu Huang,et al.  On the inverses of general tridiagonal matrices , 2010 .

[9]  Kayode James Adebayo,et al.  On Quasi-Newton Method for Solving Unconstrained Optimization Problems , 2015 .

[10]  Ali Emrouznejad,et al.  Big Data Optimization: Recent Developments and Challenges , 2016 .

[11]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[12]  Ali Emrouznejad,et al.  Big Data: Who, What and Where? Social, Cognitive and Journals Map of Big Data Publications with Focus on Optimization , 2016 .

[13]  Desmond J. Higham,et al.  Trust Region Algorithms and Timestep Selection , 1999, SIAM J. Numer. Anal..

[14]  D. J. Evans,et al.  Explicit group over-relaxation methods for solving elliptic partial differential equations , 1986 .

[15]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[16]  D. Young.  Iterative methods for solving partial difference equations of elliptic type , 1954 .

[17]  David J. Evans,et al.  Explicit De-coupled Group Iterative Methods and their Parallel Implementations , 1995, Parallel Algorithms Appl..

[18]  Rafael Martí,et al.  Experimental Testing of Advanced Scatter Search Designs for Global Optimization of Multimodal Functions , 2005, J. Glob. Optim..

[19]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[20]  Trond Steihaug,et al.  Please Scroll down for Article Optimization Methods and Software on Large-scale Unconstrained Optimization Problems and Higher Order Methods on Large-scale Unconstrained Optimization Problems and Higher Order Methods , 2022 .

[21]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[22]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[23]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[24]  Neculai Andrei.  An adaptive conjugate gradient algorithm for large-scale unconstrained optimization , 2016, J. Comput. Appl. Math..

[25]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[26]  Jumat Sulaiman,et al.  Numerical solutions of nonlinear second-order two-point boundary value problems using half-sweep SOR with Newton Method , 2013 .

[27]  M. Othman,et al.  An efficient four points modified explicit group poisson solver , 2000, Int. J. Comput. Math..

[28]  Yu. P. Laptin.  An approach to the solution of nonlinear unconstrained optimization problems , 2009 .

[29]  Jorge Nocedal,et al.  Theory of algorithms for unconstrained optimization , 1992, Acta Numerica.

[30]  Shin-Yeu Lin,et al.  An efficient method for unconstrained optimization problems of nonlinear large mesh-interconnected systems , 1995, IEEE Trans. Autom. Control..

[31]  Jonathan H. Manton,et al.  Optimization algorithms exploiting unitary constraints , 2002, IEEE Trans. Signal Process..

[32]  D. J. Evans,et al.  Group explicit iterative methods for solving large linear systems , 1985 .

[33]  Abdul Rahman Abdullah.  The four point explicit decoupled group (EDG) Method: a fast poisson solver , 1991, Int. J. Comput. Math..

[34]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[35]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .