Control-Theoretic Forward Error Analysis of Iterative Numerical Algorithms

It has been known for at least five decades that control theory can be used to study iterative algorithms. However, little work can be found in the control systems literature on numerical algorithms, especially on the study of finite precision effects. In this technical note, we consider numerical iterative algorithms in finite precision as dynamical systems and study the effects of finite precision using control theory. By using the control tools of input-to-state stability and results from the study of quantization in control systems, we present new systematic ways to find bounds on the forward error for iterative algorithms. The advantages of the proposed schemes are shown by applying them to find bounds for the classical iterative methods for solving a system of linear equations.

[1]  Hernan Haimovich.  Quantisation Issues in Feedback Control | NOVA. The University of Newcastle's Digital Repository , 2006 .

[2]  Hernan Haimovich,et al.  Quantisation Issues in Feedback Control , 2006 .

[3]  Amit Bhaya,et al.  A Study of the Robustness of Iterative Methods for Linear Systems , 2009 .

[4]  Moody T. Chu,et al.  Linear algebra algorithms as dynamical systems , 2008, Acta Numerica.

[5]  Eric C. Kerrigan,et al.  More Flops or More Precision? Accuracy Parameterizable Linear Equation Solvers for Model Predictive Control , 2009, 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines.

[6]  J. Ortega.  Stability of Difference Equations and Convergence of Iterative Processes , 1973 .

[7]  Eric C. Kerrigan,et al.  Quantization in Control Systems and Forward Error Analysis of Iterative Numerical Algorithms , 2010 .

[8]  A. Troelstra.  Non-linear systems analysis in electro-retinography , 1964 .

[9]  Kevin Skadron,et al.  Accelerating Compute-Intensive Applications with GPUs and FPGAs , 2008, 2008 Symposium on Application Specific Processors.

[10]  Gilbert Strang,et al.  Computational Science and Engineering , 2007 .

[11]  R. Brockett.  Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems , 1991 .

[12]  Jacques Laminie,et al.  Differential equations and solution of linear systems , 2005, Numerical Algorithms.

[13]  Eugenius Kaszkurewicz,et al.  A Control-Theoretic Approach to the Design of Zero Finding Numerical Methods , 2007, IEEE Transactions on Automatic Control.

[14]  María M. Seron,et al.  A systematic method to obtain ultimate bounds for perturbed systems , 2007, Int. J. Control.

[15]  L. Grüne.  Asymptotic Behavior of Dynamical and Control Systems under Perturbation and Discretization , 2002 .

[16]  Uwe Helmke,et al.  A control theory approach to linear equation solvers , 2006 .

[17]  Akila Gothandaraman,et al.  Comparing Hardware Accelerators in Scientific Applications: A Case Study , 2011, IEEE Transactions on Parallel and Distributed Systems.

[18]  J. Hurt.  Some Stability Theorems for Ordinary Difference Equations , 1967 .

[19]  Ammar Hasan.  Control theoretic analysis and design of numerical algorithms , 2012 .

[20]  Alessandro Astolfi,et al.  Stability of Dynamical Systems - Continuous, Discontinuous, and Discrete Systems (by Michel, A.N. et al.; 2008) [Bookshelf] , 2007, IEEE Control Systems.

[21]  Eugenius Kaszkurewicz,et al.  Control Perspectives on Numerical Algorithms And Matrix Problems (Advances in Design and Control) (Advances in Design and Control 10) , 2006 .

[22]  George A. Constantinides,et al.  Tutorial paper: Parallel architectures for model predictive control , 2009, 2009 European Control Conference (ECC).

[23]  Kenji Kashima,et al.  System theory for numerical analysis , 2007, Autom..

[24]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[25]  Gene H. Golub,et al.  Matrix computations , 1983 .

[26]  A. Michel,et al.  Stability of Dynamical Systems — Continuous , Discontinuous , and Discrete Systems , 2008 .

[27]  A. Michel,et al.  Quantization and overflow effects in digital implementations of linear dynamic controllers , 1988 .

[28]  Eduardo Sontag,et al.  Input-to-state stability for discrete-time nonlinear systems , 1999, at - Automatisierungstechnik.