A Control-Theoretic Approach to the Design of Zero Finding Numerical Methods

In this paper, it is shown how standard iterative methods for solving linear and nonlinear equations can be designed from the point of view of control. Appropriate choices of control Lyapunov functions (CLFs) lead to both continuous and discrete-time versions of the Newton-Raphson and conjugate gradient algorithms as well as new variants.

[1]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[2]  J. Hurt.  Some Stability Theorems for Ordinary Difference Equations , 1967 .

[3]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[4]  F. H. Branin.  Widely convergent method for finding multiple solutions of simultaneous nonlinear equations , 1972 .

[5]  J. Ortega.  Stability of Difference Equations and Convergence of Iterative Processes , 1973 .

[6]  K. Chao,et al.  Optimally controlled iterative schemes for obtaining the solution of a non-linear equation† , 1973 .

[7]  I︠a︡. Z. T︠S︡ypkin,et al.  Foundations of the theory of learning systems , 1973 .

[8]  Kumpati S. Narendra,et al.  Adaptation and learning in automatic systems , 1974 .

[9]  S. K. Korovin,et al.  Using sliding modes in static optimization and nonlinear programming , 1974, Autom..

[10]  V. G. Zhadan,et al.  Application of the method of Lyapunov functions to the study of the convergence of numerical methods , 1975 .

[11]  P. Boggs.  The convergence of the Ben-Israel iteration for nonlinear least squares problems , 1976 .

[12]  M. V. Rybashov,et al.  The method of lyapunov functions in the study of continuous algorithms of mathematical programming , 1977 .

[13]  Eduardo Sontag.  A Lyapunov-Like Characterization of Asymptotic Controllability , 1983, SIAM Journal on Control and Optimization.

[14]  Z. Artstein.  Stabilization with relaxed controls , 1983 .

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  Dimitri P. Bertsekas,et al.  Distributed asynchronous computation of fixed points , 1983, Math. Program..

[17]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[18]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[19]  K. Gustafsson,et al.  API stepsize control for the numerical solution of ordinary differential equations , 1988 .

[20]  M. Chu.  On the Continuous Realization of Iterative Processes , 1988 .

[21]  Aleksej F. Filippov,et al.  Differential Equations with Discontinuous Righthand Sides , 1988, Mathematics and Its Applications.

[22]  M. A. Krasnoselʹskii,et al.  Positive Linear Systems, the Method of Positive Operators , 1989 .

[23]  R. Brockett.  Least squares matching problems , 1989 .

[24]  Eduardo Sontag.  A universal construction of Artstein's theorem on nonlinear stabilization , 1989 .

[25]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[26]  R. Brockett.  Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems , 1991 .

[27]  Lloyd N. Trefethen,et al.  The Definition of Numerical Analysis , 1992 .

[28]  Vadim I. Utkin,et al.  Sliding Modes in Control and Optimization , 1992, Communications and Control Engineering Series.

[29]  Ali H. Sayed,et al.  Robustness of Gauss - Newton recursive methods: A deterministic feedback analysis , 1996, Signal Process..

[30]  A. R. Humphries,et al.  Dynamical Systems And Numerical Analysis , 1996 .

[31]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[32]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[33]  T. Vincent,et al.  Nonlinear and Optimal Control Systems , 1997 .

[34]  P. J. Zufiria,et al.  Weak Singularities and the Continuous Newton Method , 1999 .

[35]  J. Neuberger.  Continuous newton’s method for polynomials , 1999 .

[36]  Y. Saad,et al.  Iterative solution of linear systems in the 20th century , 2000 .

[37]  Henk A. van der Vorst.  Krylov subspace iteration , 2000, Comput. Sci. Eng..

[38]  U. Helmke,et al.  Controllability of matrix eigenvalue algorithms: the inverse power method☆ , 2000 .

[39]  Christian E. Schaerer,et al.  The shooting method for the solution of ordinary differential equations: A control-theoretical perspective , 2001, Int. J. Syst. Sci..

[40]  James R. Bunch,et al.  A conceptual framework for consistency, conditioning, and stability issues in signal processing , 2001, IEEE Trans. Signal Process..

[41]  C. Brezinski.  Dynamical systems and sequence transformations , 2001 .

[42]  S. Campbell.  Numerical analysis and systems theory , 2001 .

[43]  Uwe Helmke,et al.  On controllability of the real shifted inverse power iteration , 2001, Syst. Control. Lett..

[44]  Pedro J. Zufiria,et al.  On the role of singularities in Branin's method from dynamic and continuation perspectives , 2002, Appl. Math. Comput..

[45]  L. Grüne.  Asymptotic Behavior of Dynamical and Control Systems under Perturbation and Discretization , 2002 .

[46]  M. Krstić,et al.  Real-Time Optimization by Extremum-Seeking Control , 2003 .

[47]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[48]  J. Bunch,et al.  Applying numerical linear algebra techniques to analyzing algorithms in signal processing , 2003 .

[49]  Liqun Qi,et al.  Neurodynamical Optimization , 2004, J. Glob. Optim..

[50]  Eugenius Kaszkurewicz,et al.  Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method , 2004, Neural Networks.

[51]  Miroslav Krstic,et al.  Real-Time Optimization by Extremum-Seeking Control: Ariyur/Extremum Seeking , 2004 .

[52]  Amit Bhaya,et al.  Adaptive filtering algorithms designed using control Liapunov functions , 2006, IEEE Signal Processing Letters.

[53]  Kenji Kashima,et al.  System theory for numerical analysis , 2007, Autom..

[54]  Paul Tseng.  Control perspectives on numerical algorithms and matrix problems , 2008, Math. Comput..