High order algorithms in robust least-squares estimation with SDD information matrix: Redesign, simplification and unification

This paper describes new high order algorithms in the least-squares problem with harmonic regressor and SDD (Strictly Diagonally Dominant) information matrix. Estimation accuracy and the number of steps to achieve this accuracy are controllable in these algorithms. Simplified forms of the high order matrix inversion algorithms and the high order algorithms of direct calculation of the parameter vector are found. The algorithms are presented as recursive procedures driven by estimation errors multiplied by the gain matrices, which can be seen as preconditioners. A simple and recursive (with respect to order) algorithm for update of the gain matrix, which is associated with Neumann series is found. It is shown that the limiting form of the algorithm (algorithm of infinite order) provides perfect estimation. A new form of the gain matrix is also a basis for unification method of high order algorithms. New combined and fast convergent high order algorithms of recursive matrix inversion and algorithms of direct calculation of the parameter vector are presented. The stability of algorithms is proved and explicit transient bound on estimation error is calculated. New algorithms are simple, fast and robust with respect to round-off error accumulation.

[1]  Alexander Stotsky,et al.  Blade root moment sensor failure detection based on multibeam LIDAR for fault‐tolerant individual pitch control of wind turbines , 2014 .

[2]  C. Kelley Iterative Methods for Linear and Nonlinear Equations , 1987 .

[3]  A new frequency domain system identification method , 2011 .

[4]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[5]  Fazlollah Soleymani,et al.  A Rapid Numerical Algorithm to Compute Matrix Inversion , 2012, Int. J. Math. Math. Sci..

[6]  Dirk T. M. Slock Backward consistency concept and round-off error propagation dynamics in recursive least-squares algorithms , 1992 .

[7]  A. Stotsky Recursive trigonometric interpolation algorithms , 2010 .

[8]  Zeyuan Allen Zhu,et al.  A simple, combinatorial algorithm for solving SDD systems in nearly-linear time , 2013, STOC '13.

[9]  G. Schulz Iterative Berechung der reziproken Matrix , 1933 .

[10]  Qiang Ye Computing singular values of diagonally dominant matrices to high relative accuracy , 2008, Math. Comput..

[11]  E. V. Krishnamurthy,et al.  Numerical algorithms : computations in science and engineering , 2008 .

[12]  Victor Y. Pan,et al.  Newton-Like Iteration Based on a Cubic Polynomial for Structured Matrices , 2004, Numerical Algorithms.

[13]  L. Richardson,et al.  On the Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations, with an Application to the Stresses in a Masonry Dam , 1910 .

[14]  Yong Zhang,et al.  Chebyshev-type methods and preconditioning techniques , 2011, Appl. Math. Comput..

[15]  Phillip A. Regalia,et al.  On the numerical stability and accuracy of the conventional recursive least squares algorithm , 1999, IEEE Trans. Signal Process..

[16]  K. Chen,et al.  Matrix preconditioning techniques and applications , 2005 .

[17]  Laura Schweitzer Matrix Preconditioning Techniques And Applications , 2016 .

[18]  Weiguo Li,et al.  A family of iterative methods for computing the approximate inverse of a square matrix and inner inverse of a non-square matrix , 2010, Appl. Math. Comput..

[19]  Fehmi Cirak,et al.  ZEITSCHRIFT FÜR ANGEWANDTE MATHEMATIK UND MECHANIK , 1952, Über Stammfaktoren bei ternären quadratischen Formen.

[20]  David S. Bayard,et al.  A general theory of linear time-invariant adaptive feedforward systems with harmonic regressors , 2000, IEEE Trans. Autom. Control..

[21]  Alexander Stotsky,et al.  Harmonic regressor: Robust solution to least-squares problem , 2013, J. Syst. Control. Eng..

[22]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[23]  Adi Ben-Israel,et al.  An iterative method for computing the generalized inverse of an arbitrary matrix , 1965 .

[24]  Lennart Ljung,et al.  Error propagation properties of recursive least-squares adaptation algorithms , 1985, Autom..

[25]  L. Richardson The Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations, with an Application to the Stresses in a Masonry Dam , 1911 .

[26]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .