Convergence Analysis for Anderson Acceleration

Anderson($m$) is a method for acceleration of fixed point iteration which stores m+1 prior evaluations of the fixed point map and computes the new iteration as a linear combination of those evaluations. Anderson(0) is fixed point iteration. In this paper we show that Anderson($m$) is locally r-linearly convergent if the fixed point map is a contraction and the coefficients in the linear combination remain bounded. Without assumptions on the coefficients, we prove q-linear convergence of Anderson(1) and, in the case of linear problems, Anderson($m$). We observe that the optimization problem for the coefficients can be formulated and solved in nonstandard ways and report on numerical experiments which illustrate the ideas.

[1]  Cornelis W. Oosterlee,et al.  KRYLOV SUBSPACE ACCELERATION FOR NONLINEAR MULTIGRID SCHEMES , 1997 .

[2]  Homer F. Walker,et al.  Globally Convergent Inexact Newton Methods , 1994, SIAM J. Optim..

[3]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[4]  G. Kresse,et al.  Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set , 1996 .

[5]  Reinhold Schneider,et al.  An analysis for the DIIS acceleration method used in quantum chemistry calculations , 2011 .

[6]  N. S. Barnett,et al.  Private communication , 1969 .

[7]  R. Schneider,et al.  DIRECT MINIMIZATION FOR CALCULATING INVARIANT SUBSPACES IN DENSITY FUNCTIONAL COMPUTATIONS OF THE ELECTRONIC STRUCTURE , 2008, 0805.1190.

[8]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[9]  Michael C. Ferris,et al.  Linear programming with MATLAB , 2007, MPS-SIAM series on optimization.

[10]  Gene H. Golub,et al.  Matrix computations , 1983 .

[11]  Keith Miller.  Nonlinear Krylov and moving nodes in the method of lines , 2005 .

[12]  Yousef Saad,et al.  Numerical Methods for Electronic Structure Calculations of Materials , 2010, SIAM Rev..

[13]  Keith Miller,et al.  Design and Application of a Gradient-Weighted Moving Finite Element Code I: in One Dimension , 1998, SIAM J. Sci. Comput..

[14]  Chao Yang,et al.  Elliptic Preconditioner for Accelerating the Self-Consistent Field Iteration in Kohn-Sham Density Functional Theory , 2012, SIAM J. Sci. Comput..

[15]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[16]  Thomas F. Coleman,et al.  A Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on Some of the Variables , 1992, SIAM J. Optim..

[17]  C. Kelley.  Iterative Methods for Linear and Nonlinear Equations , 1987 .

[18]  Homer F. Walker,et al.  Anderson Acceleration for Fixed-Point Iterations , 2011, SIAM J. Numer. Anal..

[19]  C. Kelley,et al.  Newton’s Method at Singular Points. I , 1980 .

[20]  Florian Potra,et al.  A characterization of the behavior of the Anderson acceleration on linear problems , 2011, 1102.0796.

[21]  Yousef Saad,et al.  Two classes of multisecant methods for nonlinear acceleration , 2009, Numer. Linear Algebra Appl..

[22]  Carl Tim Kelley,et al.  Solution by iteration of H‐equations in multigroup neutron transport , 1978 .

[23]  P. Pulay.  Convergence acceleration of iterative sequences. the case of scf iteration , 1980 .

[24]  C. Kelley,et al.  Inexact newton methods for singular problems , 1993 .

[25]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[26]  G. Rybicki.  Radiative transfer , 2019, Climate Change and Terrestrial Ecosystem Modeling.

[27]  Stephen P. Boyd,et al.  Recent Advances in Learning and Control , 2008, Lecture Notes in Control and Information Sciences.

[28]  Donald G. M. Anderson.  Iterative Procedures for Nonlinear Integral Equations , 1965, JACM.

[29]  C. Kelley.  Solving Nonlinear Equations with Newton's Method , 1987 .

[30]  I. W. Busbridge.  The Mathematics of Radiative Transfer. , 1960 .

[31]  Z. Q. XUENorth.  Inexact Newton Methods for Singular Problems , 2007 .

[32]  T. W. Mullikin.  Some probability distributions for neutron transport in a half-space , 1968 .

[33]  P. Pulay.  Improved SCF convergence acceleration , 1982 .