Nonlinearly Preconditioned Krylov Subspace Methods for Discrete Newton Algorithms

We propose an algorithm for implementing Newton's method for a general nonlinear system $f(x) = 0$ where the linear systems that arise at each step of Newton's method are solved by a preconditioned Krylov subspace iterative method. The algorithm requires only function evaluations and does not require the evaluation or storage of the Jacobian matrix. Matrix-vector products involving the Jacobian matrix are approximated by directional differences. We develop a framework for constructing preconditionings for this inner iterative method which do not reference the Jacobian matrix explicitly. We derive a nonlinear SSOR type preconditioning which numerical experiments show to be as effective as the linear SSOR preconditioning that uses the Jacobian explicitly.

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[3]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[4]  R. P. Kendall,et al.  An Approximate Factorization Procedure for Solving Self-Adjoint Elliptic Difference Equations , 1968 .

[5]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[6]  O. Axelsson.  A generalized SSOR method , 1972 .

[7]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[8]  M. Powell,et al.  On the Estimation of Sparse Jacobian Matrices , 1974 .

[9]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[10]  T. Manteuffel.  The Tchebychev iteration for nonsymmetric linear systems , 1977 .

[11]  I. Gustafsson.  A class of first order factorization methods , 1978 .

[12]  R. Tapia,et al.  QDN: a variable storage algorithm for unconstrained optimization , 1978 .

[13]  R. Chandra.  Conjugate gradient methods for partial differential equations. , 1978 .

[14]  M. Powell,et al.  On the Estimation of Sparse Hessian Matrices , 1979 .

[15]  Hans D. Mittelmann,et al.  On the efficient solution of nonlinear finite element equations I , 1980 .

[16]  B. Parlett.  A new look at the Lanczos algorithm for solving symmetric systems of linear equations , 1980 .

[17]  P. Gill,et al.  A Procedure for Computing Forward-Difference Intervals for Numerical Optimization. , 1981 .

[18]  Dianne P. O'Leary,et al.  A discrete Newton algorithm for minimizing a function of many variables , 1982, Math. Program..

[19]  H. Elman.  Iterative methods for large, sparse, nonsymmetric systems of linear equations , 1982 .

[20]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[21]  S. Nash.  Truncated-Newton methods , 1982 .

[22]  Structural analysis of large sparse systems of nonlinear equations with application to fire modeling , 1982 .

[23]  S. Thomas McCormick,et al.  Optimal approximation of sparse hessians and its equivalence to a graph coloring problem , 1983, Math. Program..

[24]  J. J. Moré,et al.  Estimation of sparse jacobian matrices and graph coloring problems , 1983 .

[25]  C. W. Gear,et al.  Iterative Solution of Linear Equations in ODE Codes , 1983 .

[26]  Y. Saad,et al.  Practical Use of Some Krylov Subspace Methods for Solving Indefinite and Nonsymmetric Linear Systems , 1984 .