EMPIRICAL TIME SERIES ANALYSIS and MAXIMUM LIKELIHOOD ESTIMATION

Maximum likelihood (ML) estimation maximizes the likelihood function and is a celebrated principle in linear regression analysis. Asymptotically, the Cramer-Rao lower bound for the covariance matrix of unbiased estimated parameters is reached by the maximum likelihood estimator. With asymptotic arguments, it has been proved that this principle can also be applied to autoregression and to the more general autoregressive moving average (ARMA) models in time series analysis. It is at least suggested in textbooks that a closer approximation of the exact likelihood in the maximization will produce a better estimate for time series models. In contrast, the finite sample practice often shows differently. Some finite sample facts and their estimation implications are discussed.

[1]  Piet M. T. Broersen,et al.  Accurate ARMA models with Durbin's second method , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[2]  Piet M. T. Broersen,et al.  Autoregressive model order selection by a finite sample estimator for the Kullback-Leibler discrepancy , 1998, IEEE Trans. Signal Process..

[3]  Piet M. T. Broersen,et al.  The quality of models for ARMA processes , 1998, IEEE Trans. Signal Process..

[4]  P. Broersen,et al.  Facts and fiction in spectral analysis , 1998, IMTC/98 Conference Proceedings. IEEE Instrumentation and Measurement Technology Conference. Where Instrumentation is Going (Cat. No.98CH36222).

[5]  Piet M.T. Broersen,et al.  The ABC of Autoregressive Order Selection Criteria , 1997 .

[6]  Piet M. T. Broersen,et al.  The best order of long autoregressive models for moving average estimation , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[7]  Benjamin Friedlander,et al.  A general lower bound for parametric spectrum estimation , 1984 .

[8]  D. B. Preston.  Spectral Analysis and Time Series , 1983 .

[9]  J. Gooijer,et al.  On the maximum likelihood estimation of the parameters of a Gaussian moving average process , 1982 .

[10]  Johannes Ledolter,et al.  Small-sample properties of the maximum likelihood estimator in the first-order moving average model , 1981 .

[11]  S.M. Kay,et al.  Spectrum analysis—A modern perspective , 1981, Proceedings of the IEEE.

[12]  James Davidson,et al.  Problems with the estimation of moving average processes , 1981 .

[13]  P. Balestra.  A note on the exact transformation associated with the first-order moving average process , 1980 .

[14]  E. Godolphin.  An Invariance Property for the Maximum Likelihood Estimator of the Parameters of a Gaussian Moving Average Process , 1980 .

[15]  G. Jenkins,et al.  Time Series Analysis: Forecasting and Control , 1978 .

[16]  R. P. Mentz.  Estimation in the first-order moving average model through the finite autoregressive approximation : Some asymptotic results , 1977 .

[17]  H. Akaike.  A new look at the statistical model identification , 1974 .

[18]  Charles R. Nelson,et al.  The first-order moving average process: Identification, estimation and prediction , 1974 .

[19]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[20]  A. M. Walker.  Large-sample estimation of parameters for movingaverage models , 1961 .

[21]  J. Durbin.  EFFICIENT ESTIMATION OF PARAMETERS IN MOVING-AVERAGE MODELS , 1959 .

[22]  P. Whittle,et al.  Estimation and information in stationary time series , 1953 .

[23]  A. Wald,et al.  On the Statistical Treatment of Linear Stochastic Difference Equations , 1943 .

[24]  Piet M. T. Broersen,et al.  On Finite Sample Theory for Autoregressive Model Order Selection , 1993, IEEE Trans. Signal Process..

[25]  Steven Kay,et al.  Modern Spectral Estimation: Theory and Application , 1988 .

[26]  B. Hofmann-Wellenhof,et al.  Introduction to spectral analysis , 1986 .

[27]  James Durbin,et al.  The fitting of time series models , 1960 .