Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems

The spectral gradient method has proved to be effective for solving large-scale optimization problems. In this work we extend the spectral approach to solve nonlinear systems of equations. We consider a strategy based on nonmonotone line search techniques to guarantee global convergence. and discuss implementation details for solving large-scale problems. We compare the performance of our new method with recent implementations of inexact Newton schemes based on Krylov subspace inner iterative methods for the linear systems. Our numerical experiments indicate that the spectral approach for solving nonlinear systems competes favorably with well-established numerical methods.

[1]  Jośe Mario Martinez,et al.  A family of quasi-Newton methods for nonlinear equations with direct secant updates of matrix factorizations , 1990 .

[2]  Roland W. Freund,et al.  A Transpose-Free Quasi-Minimal Residual Algorithm for Non-Hermitian Linear Systems , 1993, SIAM J. Sci. Comput..

[3]  M. Ferris,et al.  Nonmonotone stabilization methods for nonlinear equations , 1994 .

[4]  M. Fukushima,et al.  A derivative-free line search and global convergence of Broyden-like method for nonlinear equations , 2000 .

[5]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[6]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[7]  Yuhong Dai.  On the Nonmonotone Line Search , 2002 .

[8]  Stefania Bellavia,et al.  A Globally Convergent Newton-GMRES Subspace Method for Systems of Nonlinear Equations , 2001, SIAM J. Sci. Comput..

[9]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[10]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[11]  A. H. Sherman.  On Newton-Iterative Methods for the Solution of Systems of Nonlinear Equations , 1978 .

[12]  L. T. Santos,et al.  Restricted optimization: a clue to a fast and accurate implementation of the Common Reflection Surface Stack method , 1999 .

[13]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[14]  José Mario Martínez,et al.  Triangular Decomposition Methods for Solving Reducible Nonlinear Systems of Equations , 1994, SIAM J. Optim..

[15]  C. Kelley.  Iterative Methods for Linear and Nonlinear Equations , 1987 .

[16]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[17]  Homer F. Walker,et al.  Globally Convergent Inexact Newton Methods , 1994, SIAM J. Optim..

[18]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[19]  José Mario Martínez,et al.  Comparing Algorithms for Solving Sparse Nonlinear Systems of Equations , 1992, SIAM J. Sci. Comput..

[20]  Lenhart K. Schubert.  Modification of a quasi-Newton method for nonlinear equations with a sparse Jacobian , 1970 .

[21]  Ernesto G. Birgina,et al.  Automatic differentiation and spectral projected gradient methods for optimal control problems , 1998 .

[22]  Glenn M. Fung,et al.  A fast and global two point low storage optimization technique for tracing rays in 2D and 3D isotropic media , 2000 .

[23]  Maria Grazia Gasparo,et al.  A nonmonotone hybrid method for nonlinear systems , 2000 .

[24]  Yousef Saad,et al.  Convergence Theory of Nonlinear Newton-Krylov Algorithms , 1994, SIAM J. Optim..

[25]  Yousef Saad,et al.  Hybrid Krylov Methods for Nonlinear Systems of Equations , 1990, SIAM J. Sci. Comput..

[26]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[27]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[28]  E. Birgin,et al.  Determination of thickness and optical constants of amorphous silicon films from transmittance data , 2000 .

[29]  H. Walker,et al.  Convergence Theorems for Least-Change Secant Update Methods, , 1981 .

[30]  Marcos Raydan,et al.  Low Cost Optimization Techniques for Solving the Nonlinear Seismic Reflection Tomography Problem , 2000 .

[31]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[32]  E. Birgin,et al.  Estimation of the optical constants and the thickness of thin films using unconstrained optimization , 2006 .

[33]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[34]  J. M. Martínez,et al.  Local convergence theory of inexact Newton methods based on structured least change updates , 1990 .