A Globally Convergent Trust-Region Method for Large-Scale Symmetric Nonlinear Systems

This study presents a novel adaptive trust-region method for solving symmetric nonlinear systems of equations. The new method uses a derivative-free quasi-Newton formula in place of the exact Jacobian. The global convergence and local quadratic convergence of the new method are established without the nondegeneracy assumption of the exact Jacobian. Using the compact limited memory BFGS, we adapt a version of the new method for solving large-scale problems and develop the dogleg scheme for solving the associated trust-region subproblems. The sufficient decrease condition for the adapted dogleg scheme is established. While the efficiency of the present trust-region approach can be improved by using adaptive radius techniques, utilizing the compact limited memory BFGS adjusts this approach to handle large-scale symmetric nonlinear systems of equations. Preliminary numerical results for both medium- and large-scale problems are reported.

[1]  Jorge Nocedal,et al.  A Numerical Study of the Limited Memory BFGS Method and the Truncated-Newton Method for Large Scale Optimization , 1991, SIAM J. Optim..

[2]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[3]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[4]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[5]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[6]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[7]  Gonglin Yuan,et al.  Limited memory BFGS method with backtracking for symmetric nonlinear equations , 2011, Math. Comput. Model..

[8]  P. Gill,et al.  Algorithms for the Solution of the Nonlinear Least-Squares Problem , 1978 .

[9]  M. Fukushima,et al.  On the Rate of Convergence of the Levenberg-Marquardt Method , 2001 .

[10]  Masoud Ahookhosh,et al.  Two globally convergent nonmonotone trust-region methods for unconstrained optimization , 2015, 1501.02011.

[11]  Masoud Ahookhosh,et al.  An efficient nonmonotone trust-region method for unconstrained optimization , 2011, Numerical Algorithms.

[12]  Jorge Nocedal,et al.  Combining Trust Region and Line Search Techniques , 1998 .

[13]  Jinyan Fan,et al.  On the modified trust region algorithm for nonlinear equations , 2015, Optim. Methods Softw..

[14]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..

[15]  Jorge Nocedal,et al.  Remark on “algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization” , 2011, TOMS.

[16]  Bobby Schnabel,et al.  Tensor methods for large sparse systems of nonlinear equations , 1996, Math. Program..

[17]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[18]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Gauss-Newton-Based BFGS Method for Symmetric Nonlinear Equations , 1999, SIAM J. Numer. Anal..

[19]  Ya-Xiang Yuan,et al.  Recent advances in numerical methods for nonlinear equations andnonlinear least squares , 2011 .

[20]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[21]  Masoud Ahookhosh,et al.  A hybrid of adjustable trust-region and nonmonotone algorithms for unconstrained optimization , 2014 .

[22]  Jorge Nocedal,et al.  Enriched Methods for Large-Scale Unconstrained Optimization , 2002, Comput. Optim. Appl..

[23]  Natasa Krejic,et al.  Practical Quasi-Newton algorithms for singular nonlinear systems , 2010, Numerical Algorithms.

[24]  Masoud Ahookhosh,et al.  A nonmonotone trust-region line search method for large-scale unconstrained optimization , 2012 .

[25]  Zhenjun Shi,et al.  A new trust region method for unconstrained optimization , 2008 .

[26]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[27]  XU JAMESV.BURKEANDREASWIEGMANNLIANG.  LIMITED MEMORY BFGS UPDATING IN A TRUST – REGION FRAMEWORK , 1996 .

[28]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[29]  Marco Sciandrone,et al.  A Truncated Nonmonotone Gauss-Newton Method for Large-Scale Nonlinear Least-Squares Problems , 2006, Comput. Optim. Appl..

[30]  Liqun Qi,et al.  On the Convergence of a Trust-Region Method for Solving Constrained Nonlinear Equations with Degenerate Solutions , 2004 .

[31]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[32]  LongHei.  A SELF—ADAPTIVE TRUST REGION ALGORITHM , 2003 .

[33]  Xiwen Lu,et al.  A BFGS trust-region method for nonlinear equations , 2011, Computing.

[34]  Annick Sartenaer,et al.  Automatic Determination of an Initial Trust Region in Nonlinear Programming , 1997, SIAM J. Sci. Comput..

[35]  Ya-Xiang Yuan,et al.  Subspace methods for large scale nonlinear equations and nonlinear least squares , 2009 .

[36]  Linda Kaufman.  Reduced Storage, Quasi-Newton Trust Region Approaches to Function Optimization , 1999, SIAM J. Optim..

[37]  Yong Wang,et al.  A new trust region method for nonlinear equations , 2003, Math. Methods Oper. Res..

[38]  Anderas Griewank.  The “global” convergence of Broyden-like methods with suitable line search , 1986, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[39]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[40]  Masoud Ahookhosh,et al.  An effective trust-region-based approach for symmetric nonlinear systems , 2013, Int. J. Comput. Math..

[41]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[42]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[43]  R. Schnabel,et al.  Tensor Methods for Nonlinear Equations. , 1984 .

[44]  C. G. Broyden,et al.  The convergence of an algorithm for solving sparse nonlinear systems , 1971 .

[45]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[46]  Jinhua Guo,et al.  A new trust region method with adaptive radius , 2008, Comput. Optim. Appl..

[47]  John E. Dennis,et al.  On the Convergence of Broyden's Method for Nonlinear Systems of Equations , 1971 .

[48]  Jinyan Fan.  Convergence Rate of The Trust Region Method for Nonlinear Equations Under Local Error Bound Condition , 2006, Comput. Optim. Appl..

[49]  Morteza Kimiaei,et al.  A new adaptive trust-region method for system of nonlinear equations , 2014 .

[50]  L. Lukšan,et al.  Truncated Trust Region Methods Based on Preconditioned Iterative Subalgorithms for Large Sparse Systems of Nonlinear Equations , 1997 .

[51]  Qingna Li,et al.  A class of derivative-free methods for large-scale nonlinear monotone equations , 2011 .

[52]  Jinyan Fan,et al.  An improved trust region algorithm for nonlinear equations , 2011, Comput. Optim. Appl..

[53]  Masoud Ahookhosh,et al.  Computers and Mathematics with Applications a Nonmonotone Trust Region Method with Adaptive Radius for Unconstrained Optimization Problems , 2022 .

[54]  Jośe Mario Martinez,et al.  A family of quasi-Newton methods for nonlinear equations with direct secant updates of matrix factorizations , 1990 .

[55]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[56]  Dong-Hui Li,et al.  Descent Directions of Quasi-Newton Methods for Symmetric Nonlinear Equations , 2002, SIAM J. Numer. Anal..