A comparative analysis of various least-squares identification algorithms

The purpose of this paper is to clarify the relations and to provide some selection guides among several time-series identification algorithms that appear in the literature under different names but which are essentially least-squares identification algorithms where only the numerical solution of the least-squares estimation problem is different. Such algorithms are, apart from the batch and the sequential forms of direct least-squares, the PARCOR (partial correlation) algorithm, (which may be in the Durbin, the Levinson or the autocorrelation form), the lattice or the ladder algorithm, also known as the Markel-Gray algorithm, the square-root algorithm, the equation-error algorithm, and related algorithms. Further to the above, we shall discuss why certain such algorithms differ in performance from the direct least-squares forms, in terms of convergence, convergence-rate, computational effort (speed) per iteration, and in terms of robustness to computational errors, such as arise when using short word-length computers.

[1]  Mandyam D. Srinath,et al.  Sequential Estimation Technique for Enhancement of Noisy Images , 1977, IEEE Transactions on Computers.

[2]  C. Lawson,et al.  Extensions and applications of the Householder algorithm for solving linear least squares problems , 1969 .

[3]  L. R. Nardizzi,et al.  Noninvasive radioisotope cardiographic images with digital prefiltering and boundary detection for measuring left ventricular parameters , 1977 .

[4]  A. Gray,et al.  On autocorrelation equations as applied to speech analysis , 1973 .

[5]  J. Makhoul.  Spectral analysis of speech by linear prediction , 1973 .

[6]  P. Fenwick,et al.  Mathematical simulation of the electroencephalogram using an autoregressive series. , 1971, International journal of bio-medical computing.

[7]  Daniel Graupe,et al.  On convergence of least-squares identifiers of autoregressive models having stable and unstable roots , 1980 .

[8]  A. Wald,et al.  On the Statistical Treatment of Linear Stochastic Difference Equations , 1943 .

[9]  Daniel Graupe,et al.  Identification of autoregressive moving-average parameters of time series , 1975 .

[10]  James Durbin,et al.  The fitting of time series models , 1960 .

[11]  S. Merhav,et al.  Unbiased parameter estimation by means of autocorrelation functions , 1975 .

[12]  Karl Johan Åström,et al.  BOOK REVIEW SYSTEM IDENTIFICATION , 1994, Econometric Theory.

[13]  Daniel Graupe,et al.  Functional Separation of EMG Signals via ARMA Identification Methods for Prosthesis Control Purposes , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[14]  D. Faddeev,et al.  Computational Methods of Linear Algebra , 1959 .

[15]  John E. Markel,et al.  Linear Prediction of Speech , 1976, Communication and Cybernetics.

[16]  R. Mehra.  On-line identification of linear dynamic systems with applications to Kalman filtering , 1971 .

[17]  N. Levinson.  The Wiener (Root Mean Square) Error Criterion in Filter Design and Prediction , 1946 .

[18]  Jerry D. Gibson,et al.  On reflection coefficients and the Cholesky decomposition , 1977 .

[19]  M. Srinath,et al.  Sequential algorithm for identification of parameters of an autoregressive process , 1975 .

[20]  Jerry D. Gibson,et al.  Unified development of algorithms used for linear predictive coding of speech signals , 1976 .

[21]  B. Atal,et al.  Speech analysis and synthesis by linear prediction of the speech wave. , 1971, The Journal of the Acoustical Society of America.

[22]  Fumitada Itakura,et al.  An Audio Response Unit Based on Partial Autocorrelation , 1972, IEEE Trans. Commun..

[23]  J. P. Burg,et al.  Maximum entropy spectral analysis. , 1967 .

[24]  A. Gray,et al.  Digital lattice and ladder filter synthesis , 1973 .

[25]  Waveforms Hisashi Wakita.  Direct Estimation of the Vocal Tract Shape by Inverse Filtering of Acoustic Speech , 1973 .

[26]  Daniel Graupe,et al.  Identification of Systems , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[27]  Alston S. Householder,et al.  The Theory of Matrices in Numerical Analysis , 1964 .

[28]  M. Pagano.  An Algorithm for Fitting Autoregressive Schemes , 1972 .

[29]  G. Bierman.  Factorization methods for discrete sequential estimation , 1977 .

[30]  J. Makhoul.  Stable and efficient lattice methods for linear prediction , 1977 .

[31]  W. Gersch.  Spectral analysis of EEG's by autoregressive decomposition of time series , 1970 .

[32]  J. Doob.  Stochastic processes , 1953 .

[33]  E. A. Flinn.  Comments on “Speech Analysis and Synthesis by Linear Prediction of the Speech Wave” [B. S. Atal and S. L. Hanauer, J. Acoust. Soc. Amer. 50, 637–655 (1971)] , 1972 .

[34]  P. Lion.  Rapid identification of linear and nonlinear systems. , 1967 .